Deep Neural Networks for Landmark Detection on 3D Models

Landmark detection is a frequent step during medical data analysis. More and more often, these data are represented in the form of 3D models. An example is a 3D intraoral scan of dentition. Deep neural networks are an appropriate way of detecting landmarks in images. In terms of 3D data, the processing comes with high memory requirements and computational time, which does not meet the needs of medical applications. In this work, I propose a method that eliminates this problem and detects landmarks on the surface of polygonal models of jaws. Different architectures of neural networks, all of which are based on the U-Net architecture, are used in this work. The multi-view approach transforms the task into a 2D domain, where the suggested networks detect landmarks by heatmap regression from several viewpoints. Using a consensus method, final estimates from multiple views are predicted in 3D space. This work introduces experiments with two consensus methods: a centroid of predictions and a geometric approach based on the RANSAC algorithm and least-squares fit. Experiments have shown that a combination of Attention U-Net, 100 viewpoints, and RANSAC consensus method, is able to detect landmarks with an error of 1.20 +- 1.81 mm, while 94.01% of landmarks is predicted with an error of less than 2 mm.

landmark detection in 3D, polygonal models, multi-view neural networks, RANSAC, U-Net, heatmap regression