Augmented ​ Reality and ​ Artificial ​ Intelligence supported Laparoscopic ​Imagery in ​ Surgery​ ​ (ARAILIS)

Project title

Augmented ​ Reality and ​ Artificial ​ Intelligence supported Laparoscopic ​Imagery in ​ Surgery​ ​ (ARAILIS)

Description

Project duration: 01.02.2020 - 31.12.2022

Minimally invasive surgery using stereo endoscopes reduces the necessary care for the patients after surgery. However, due to the limited visibility and lack of haptic feedback, these surgeries can be more challenging for physicians. Therefore, the goal of this project is to support surgeons by displaying information in the camera image.

Surgical interventions are often rely on CT, MRI or ultrasound data which are recorded before an operation, to determine the location of tumors and to plan the surgery. The goal of this project is to display this data as an overlay in front of the camera image during the operation to support navigation. In addition, hidden vessels and the tumors can be made visible. The particular scientific challenge here lies, among other things, in the deformation of the human body between the recording of the preoperative data and the subsequent intervention, as well as during the actual operation.

The tasks of photogrammetry in the project are the development of a concept for the calibration of the used stereo endoscope, taking into account the constraints and routines in the operating room. In addition, improvements in spatial reconstruction of the dynamic environment are developed.

 A short demonstration of the overall system shows the possible process:

Addional information

• Professur für Multimedia-Technologie (MT)
• Professur für Softwaretech­nologie (SWT)

Publications

Docea, R., Pfeiffer, M., Müller, J., Krug, K., Hardner, M., Riedel, P., Menzel, M., Kolbinger, F. R., Frohneberg, L., Weitz, J., & Speidel, S. (2022). A Laparoscopic Liver Navigation Pipeline with Minimal Setup Requirements. In BioCAS 2022 - IEEE Biomedical Circuits and Systems Conference: Intelligent Biomedical Systems for a Better Future, Proceedings: Intelligent Biomedical Systems for a Better Future, Proceedings (pp. 578-582). [9948587] IEEE Xplore. https://doi.org/10.1109/BioCAS54905.2022.9948587

Müller, J., Docea, R., Hardner, M., Krug, K., Riedel, P., & Tetzlaff, R. (2022). Fast High-Resolution Disparity Estimation for Laparoscopic Surgery. In BioCAS 2022 - IEEE Biomedical Circuits and Systems Conference: Intelligent Biomedical Systems for a Better Future, Proceedings: Intelligent Biomedical Systems for a Better Future, Proceedings (pp. 573-577). [9948563] IEEE Xplore. https://doi.org/10.1109/BioCAS54905.2022.9948563

Hardner, M., Docea, R., & Schneider, D. (2022). Guided Calibration of Medical Stereo Endoscopes. In International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives (XLIII-B2-2022 ed., Vol. 43, pp. 679-686). The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences Vol. XLIII-B2-2022 https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-679-2022 

Project partner

• Klinik und Poliklinik für Viszeral-, Thorax- und Gefäßchirurgie (VTG)

• Abteilung für Translationale Chirurgie Onkologie am Nationalen Centrum für Tumorerkrankungen Dresden (NCT)

• TU Dresden, Professur für Softwaretech­nologie (SWT)

• TU Dresden, Professur für Grundlagen der Elektrotechnik (ETIT)

• TU Dresden, Professur für Multimedia-Technologie (MT)

Contact

• M.Sc. Matthias Hardner (project work)
• Prof. Dr. Danilo Schneider (project management)

Funding

This project is co-financed from tax revenues on the basis of the budget decided by the members of the Saxon state parliament. TG70 / SAB