DEMIS: Electron Microscopy Image Stitching using Deep Learning Features and Global Optimisation

Electron Microscopy, Whole Slide Imaging, Image Stitching, Neural Networks.

Accurate stitching of overlapping image tiles is essential for reconstructing large-scale Electron Microscopy (EM) images during Whole Slide Imaging. Current stitching approaches rely on handcrafted features and translation-only global alignment based on Minimum Spanning Tree (MST) construction. This results in suboptimal global alignment since it neglects rotational errors and works only with transformations estimated from pairwise feature matches, discarding valuable information tied to individual features. Moreover, handcrafted features may have trouble with repetitive textures. Motivated by the limitations of current methods and recent advancements in deep learning, we propose DEMIS, a novel EM image stitching method. DEMIS uses Local Feature TRansformer (LoFTR) for image matching, and optimises translational and rotational parameters directly at the level of individual features. For evaluation and training, we create EM424, a synthetic dataset generated by splitting high-resolution EM images into arrays of overlapping image tiles. Furthermore, to enable evaluation on unannotated real-world data, we design a no-reference stitching quality metric based on optical flow. Experiments that use the new metric show that DEMIS can improve the average results from 32.11 to 2.28 compared to current stitching techniques (a 1408\% improvement).