STL to Fluoroscopy Image Registration for Autonomous Navigation in Vascular Interventions

In this Directed Research at the Future Therapeutic Unlimited Robotics Engineering (FuTURE) Lab at WPI, I implemented an interface to perform 2D/3D image registration between a fluoroscopy image and the respective STL derived volume, or CT/MRI preoperative scan. The pipeline leverages xvr (X-ray to Volume Registration), which uses DiffDRR in the backend as the render model to simulate realistic, differentiable Digital Reconstructed Radiographs (DRR).
The developed framework is divided into three main phases:
(i) STL to Volume Generation: Converting the STL model into a Housfield Unit (HU)-assigned scalar volume through 3D Slicer.
(ii) XVR GUI: Interfacing with the Preoperative and Intraoperative Stages.
(iii) Experimentation: Testing registration with a fluoroscopy video of a 3D printed anatomy. This requires initial pre-processing to extract a Region of Interest (ROI), remove catheter pixels to balance image intensity, and convert the output to DICOM format. After registration, the performance is evaluated through a manual landmark detection.