Towards Integration of Augmented Reality into the Interventional Radiology Suite
Abstract
The groundbreaking advancements in medical imaging technology throughout the 20th century have laid the groundwork for the field of interventional radiology, where a wide range of minimally-invasive procedures are performed. Despite their benefits over open surgery, these procedures impose significant cognitive demands due to the mental mapping required between 2D imaging (e.g., ultrasound, fluoroscopy) and the 3D patient and tools. Additionally, the optimal positioning of displays is constrained by physical equipment, leading to a large visual-motor field disparity.
Augmented reality surgical navigation systems have recently gained attention as a means to address these challenges, particularly with the emergence of optical see-through head-mounted displays, allowing clinicians to retain a natural view of the operating environment. However, technical challenges in calibration, tool tracking, depth perception, and workflow integration have thus far limited their widespread clinical adoption.
This thesis develops and validates a comprehensive AR infrastructure tailored towards the HoloLens 2. First, virtual displays are introduced to minimize visual-motor field disparity without a significant change to the current workflow, improving clinical feasibility. A user study demonstrates a preference for virtual displays among novices, with no significant impact on procedure time or accuracy.
Next, tracking accuracy is assessed, revealing that monoscopic vision-based tracking falls short of interventional radiology requirements due to depth errors, line-of-sight constraints, and the difficulty of rigidly attaching optical markers to surgical tools. To address this, a novel hand-eye calibration method for the HoloLens 2 is developed, enabling seamless integration of magnetic tracking into the AR system. This solution resolves prior limitations and achieves sub-2 mm accuracy in under a minute—suitable for clinical deployment.
Finally, the thesis tackles depth perception challenges in medical AR. Because occlusion is a critical depth cue, a "black hole" projector is introduced to effectively remove real-world light, enhancing the illusion of virtual objects beneath the patient’s surface. A user study confirms improved needle targeting accuracy and reduced mental demand under this approach compared to standard surgical lighting.
This work advances AR integration in interventional radiology by addressing core challenges in visualization, tracking, and clinical feasibility, paving the way for future adoption in surgical navigation.