As part of the INSPIRATION project, led by Assoc. Prof. Filip Šuligoj, junior researchers and doctoral students Ines Frajtag and Roč Stilinović spent two months on a research stay at the Surgical Robotics Laboratory at the University of Twente in the Netherlands.
During her research stay at the Surgical Robotics Laboratory at the University of Twente, PhD student Ines Frajtag worked on the development of a computer vision-based tracking system for magnetically controlled microrobots. Such microrobots have great potential in minimally invasive medical procedures, for example for precise positioning in confined spaces, targeted delivery of therapy or performing tasks in hard-to-reach places, where classical instruments have limitations. A key prerequisite for the reliable application of such systems is the ability to stably and precisely determine the position of the microrobot in real time, which is a challenge due to their small dimensions, changing lighting conditions and reflections. As part of the work, special emphasis was placed on the design and configuration of the experimental setup so that the entire work area was continuously visible to the cameras and to ensure repeatability of measurements. After assembling the setup, activities related to data collection and development of the detection method were carried out, using approaches based on neural networks to recognize microrobots of different sizes and shapes. Based on multi-view detections, a procedure for generating 3D position data was developed, which represents the basis for later magnetic field control of microrobots (closed loop: detection - position estimation - control). The obtained results contribute to the development of a robust tracking system that is applicable in research and future clinically relevant scenarios.
During his research stay at the Surgical Robotics Laboratory at the University of Twente, PhD student Roč Stilinović worked on studying and unifying methods for registering surgical instruments in robotic-assisted, image-guided surgery. The activities focused on comparing marker-based and marker-free approaches, while reviewing the sensors and imaging modalities used in data acquisition (e.g. ultrasound, fluoroscopy/CBCT, CT/MR, endoscopic images, and optical and electromagnetic tracking systems). The particular importance of such precise registration is evident in branches of medicine that require extreme precision, such as dental surgery, neurosurgery, and cardiovascular surgery. It was also important to consider possible challenges and problems that make reliable instrument registration difficult in real-world operating conditions: line-of-sight obstruction and reflection in optical tracking, magnetic field distortion in electromagnetic trackers, drift in IMU sensors, or degradation of image-based registration due to blurring, smoke, blood, low contrast, and artifacts caused by the instrument itself. Practical challenges of calibration between sensors and cameras (eg hand-eye), real-time time synchronization and latency, as well as the influence of soft tissue deformations on rigid registration assumptions are also discussed.
We would like to thank the entire Surgical Robotics Laboratory team for their support and excellent cooperation.
The project is funded by the European Union - NextGenerationEU.
The project was co-financed by the European Union from the European Regional Development Fund
The website was co-financed by the European Union from the European Regional Development Fund.
The content of the website is the sole responsibility of the Faculty of Mechanical Engineering and Naval Architecture.