This project was completed as part of my bachelor thesis in computer science. Over the course of several months, I explored the potential of point cloud registration techniques for achieving accurate and scalable localization in Augmented and Mixed Reality. Using LiDAR data from the Apple Vision Pro, I systematically evaluated the performance of different registration algorithms under varying environmental conditions.
TOOLS USED
VSC, Overleaf, JASP
ROLE
UX Reseacher, UX/ UI Designer, Developer
This project was completed as part of my bachelor thesis in computer science. Over the course of several months, I explored the potential of point cloud registration techniques for achieving accurate and scalable localization in Augmented and Mixed Reality. Using LiDAR data from the Apple Vision Pro, I systematically evaluated the performance of different registration algorithms under varying environmental conditions.
TOOLS USED
VSC, Overleaf, JASP
ROLE
UX Reseacher, UX/ UI Designer, Developer
3D Registration Algorithm Evaluation
Problem and Motivation
With the rise of consumer-grade LiDAR and depth sensors in headsets like the Apple Vision Pro, leveraging 3D spatial information for localization has become increasingly feasible. Point cloud-based methods present a promising alternative to traditional vision-based approaches, offering improved robustness and accuracy in complex indoor environments. In this thesis, I systematically evaluated various registration algorithms and scene conditions to understand their impact on localization performance - providing valuable insights for building more reliable and scalable spatial computing systems in AR/MR.
Research Question
The thesis tackles the challenge of global localization in AR/MR by systematically evaluating point cloud registration pipelines under realistically simulated indoor conditions. The goal was to identify which combinations of algorithms, refinement techniques, and point cloud characteristics deliver the most robust and accurate results.
The research was guided by the central question:
To what extent do algorithmic design choices and point cloud characteristics influence the performance of 3D registration methods for large-scale indoor localization in AR/MR applications?
Problem and Motivation
With the rise of consumer-grade LiDAR and depth sensors in headsets like the Apple Vision Pro, leveraging 3D spatial information for localization has become increasingly feasible. Point cloud-based methods present a promising alternative to traditional vision-based approaches, offering improved robustness and accuracy in complex indoor environments. In this thesis, I systematically evaluated various registration algorithms and scene conditions to understand their impact on localization performance - providing valuable insights for building more reliable and scalable spatial computing systems in AR/MR.
Research Question
The thesis tackles the challenge of global localization in AR/MR by systematically evaluating point cloud registration pipelines under realistically simulated indoor conditions. The goal was to identify which combinations of algorithms, refinement techniques, and point cloud characteristics deliver the most robust and accurate results.
The research was guided by the central question:
To what extent do algorithmic design choices and point cloud characteristics influence the performance of 3D registration methods for large-scale indoor localization in AR/MR applications?
Independent Variables
Registration Algorithm: RANSAC, FGR, TEASER++
ICP Refinement: With ICP, Without ICP
Point Cloud Composition: Structural point cloud, Furnished point cloud, Complete point cloud
Scene Variability: 0 cm, 5 cm, 10 cm displacement
Point Cloud Complexity: Low complexity (Apartment, 52 m²), High complexity (Office, 322 m²)
Dependent Variables
Translation Error
Rotation Error
Root Mean Square Error
Computation Time
Success Rate
Independent Variables
Registration Algorithm: RANSAC, FGR, TEASER++
ICP Refinement: With ICP, Without ICP
Point Cloud Composition: Structural point cloud, Furnished point cloud, Complete point cloud
Scene Variability: 0 cm, 5 cm, 10 cm displacement
Point Cloud Complexity: Low complexity (Apartment, 52 m²), High complexity (Office, 322 m²)
Dependent Variables
Translation Error
Rotation Error
Root Mean Square Error
Computation Time
Success Rate
Simulation Process
The simulation pipeline mimics the interaction between an AR/MR headset and a backend registration system using a client-server model. A Python-based client simulates the Apple Vision Pro, generating partial point clouds by sampling random viewpoints within a floor plan, applying realistic field-of-view constraints, occlusion modeling, and random transformations. These point clouds are sent to a server, which runs different registration algorithms (RANSAC, FGR, TEASER++, with optional ICP). The server returns alignment results, which the client then evaluates using metrics like translation error, rotation error, RMSE, and computation time. This setup enabled consistent, repeatable testing across varied conditions.
Simulation Process
The simulation pipeline mimics the interaction between an AR/MR headset and a backend registration system using a client-server model. A Python-based client simulates the Apple Vision Pro, generating partial point clouds by sampling random viewpoints within a floor plan, applying realistic field-of-view constraints, occlusion modeling, and random transformations. These point clouds are sent to a server, which runs different registration algorithms (RANSAC, FGR, TEASER++, with optional ICP). The server returns alignment results, which the client then evaluates using metrics like translation error, rotation error, RMSE, and computation time. This setup enabled consistent, repeatable testing across varied conditions.
Lessons Learned
CHALLENGES & OPPORTUNITIES
Designing and evaluating 3D registration pipelines revealed important trade-offs between accuracy, robustness, and computation time. Choosing the right algorithm - especially FGR - proved critical for performance. Incorporating ICP improved reliability but increased runtime. Unexpectedly, complex scenes enhanced registration success despite higher error and processing costs, showing potential for leveraging rich geometry in real-world applications.
OUTLOOK
The results provide practical insights for building more reliable localization systems in AR/MR. Future work should focus on real-world validation, more advanced descriptors, and combining LiDAR with vision-based methods to improve adaptability and robustness in dynamic environments.
Lessons Learned
CHALLENGES & OPPORTUNITIES
Designing and evaluating 3D registration pipelines revealed important trade-offs between accuracy, robustness, and computation time. Choosing the right algorithm - especially FGR - proved critical for performance. Incorporating ICP improved reliability but increased runtime. Unexpectedly, complex scenes enhanced registration success despite higher error and processing costs, showing potential for leveraging rich geometry in real-world applications.
OUTLOOK
The results provide practical insights for building more reliable localization systems in AR/MR. Future work should focus on real-world validation, more advanced descriptors, and combining LiDAR with vision-based methods to improve adaptability and robustness in dynamic environments.
Let's Work Together.
