3D Reconstruction using Prior Knowledge

Produktbeschreibung

This dissertation investigates how prior knowledge can improve optical reconstruction systems that enable the localizationand measurement of objects in three-dimensional space. Reconstruction algorithms are usually subject to information lossduring the initial image acquisition, affecting the results’ quality. This fundamental problem can be mitigated by exploiting prior knowledge about the scene. Four reconstructionsystems are presented in this dissertation, demonstrating theeffective utilization of prior knowledge. First, a deep convolutional neural network is used for matting dynamic scenes, exploiting synchronizedbackground color changes to reconstruct transparent foregrounds, even with imprecise backgroundcolors. Second, the exact positions of cochlear implant electrodes are localized using Markov random fields, utilizing priorknowledge of electrode distances and minimal bending radii, significantly improving positioning accuracy. Third, electroluminescent wires woven into hair help reconstruct braided hairstyles for special effects by using active curves to track guidehairs and create realistic 3D braid models. Finally, 3D reconstruction of the human spine during movement is achievedby applying perforated kinesiology tape to the athlete’s back. Prior knowledge of the placement and spacing of the tape’sholes aids the reconstruction process, resulting in high marker density, accurate localization, and robustness to occlusion.The four reconstruction systems are each presented through a comprehensive description of the entire pipeline, includingdata acquisition, algorithm details, scientific evaluation, and discussion of the results.

Contents
1 introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 What is a Prior? . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Industrial Image Processing and 3D Reconstruction . . . . 7
1.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 fundamentals 17
2.1 Camera Model . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.1 External Camera Matrix . . . . . . . . . . . . . . . . 18
2.1.2 Internal Camera Matrix . . . . . . . . . . . . . . . . 19
2.1.3 Camera Calibration . . . . . . . . . . . . . . . . . . 20
2.2 Markov Random Fields . . . . . . . . . . . . . . . . . . . . 22
2.3 Linear Programming . . . . . . . . . . . . . . . . . . . . . . 25
2.4 Active Contours . . . . . . . . . . . . . . . . . . . . . . . . . 27
3 temporal duplex alpha matting 32
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.3.1 Time Duplex System . . . . . . . . . . . . . . . . . . 40
3.3.2 Multiple Backdrop Matting . . . . . . . . . . . . . . 42
3.3.3 Implementation Details . . . . . . . . . . . . . . . . 48
3.4 Experiments and Results . . . . . . . . . . . . . . . . . . . . 50
3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4 hair el 68
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.3.1 Dataset Acquisition . . . . . . . . . . . . . . . . . . 72
4.3.2 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.3.3 Extraction of Braid Strand Trajectories . . . . . . . 75
4.4 Experiments and Results . . . . . . . . . . . . . . . . . . . . 80
4.4.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.4.2 Comparisons . . . . . . . . . . . . . . . . . . . . . . 83
4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5 cochlear implant localization 90
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.3 Synthetic and real CBCT Datasets . . . . . . . . . . . . . . 92
5.4 Electrode Array Localization . . . . . . . . . . . . . . . . . 94
5.5 Experiments and Results . . . . . . . . . . . . . . . . . . . . 97
5.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6 spine motion capture 100
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6.3.1 Linear Programming . . . . . . . . . . . . . . . . . 106
6.3.2 Alignment Markov Random Field . . . . . . . . . . 108
6.3.3 Refinement Markov Random Field . . . . . . . . . 110
6.4 Experiments and Results . . . . . . . . . . . . . . . . . . . . 112
6.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
7 conclusion and outlook 121
7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.2 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
bibliography 125

Keywords: Computer Vision, 3D-Rekonstruktion, Vorwissen, Optimierung, Computer Vision, 3D Reconstruction, Prior Knowledge, Optimization