Computer science meets medicine: Immersive assistance for surgery

The IEEE VR took place in Saint-Malo, France, from March 7 to 13. Mats Ole Ellenberg and Katja Krug from the Interactive Media Lab Dresden presented Endomersion - an immersive "remote guidance and feedback" system for robot-assisted minimally invasive surgery. The project shows how computer science research can contribute to the development of innovative solutions for high-precision medical interventions.

Endomersion was developed as part of the CeTI (Centre for Tactile Internet with Human-in-the-Loop) Cluster of Excellence and is a good example of interdisciplinary collaboration between computer science, medical technology and surgery. In cooperation with the Department of Translational Surgical Oncology at the National Center for Tumor Diseases Dresden, the Clinic for Visceral, Thoracic and Vascular Surgery at the University Hospital Carl Gustav Carus Dresden, the Chair of Industrial Design Engineering at the Faculty of Mechanical Science and Engineering and our Interactive Media Lab, this system has grown over the years and has been iteratively improved through feedback from the surgeons and technical experts involved.

Now Endomersion was presented to visitors of the IEEE VR in a realistic medical scenario: They slipped into the role of a remote expert who provides real-time support during an operation in Dresden. With the help of a mixed reality head-mounted display, they were able to view the stereoscopic camera image from the endoscope camera and augment it in real time. They were able to place visual annotations, share additional medical images and even control the robotic arm holding the endoscope camera - over a distance of almost 1500 km. In the future, this form of telemedical consultation could help to make surgical expertise available regardless of location and enable both verbal and non-verbal communication.