Augmented Reality and Artificial Intelligence supported Laparoscopic Imagery in Surgery (ARAILIS)
Project title
Augmented Reality and Artificial Intelligence supported Laparoscopic Imagery in Surgery (ARAILIS)
Description
Project duration: 01.02.2020 - 31.12.2022
Minimally invasive surgery using stereo endoscopes reduces the necessary care for the patients after surgery. However, due to the limited visibility and lack of haptic feedback, these surgeries can be more challenging for physicians. Therefore, the goal of this project is to support surgeons by displaying information in the camera image.
Surgical interventions are often rely on CT, MRI or ultrasound data which are recorded before an operation, to determine the location of tumors and to plan the surgery. The goal of this project is to display this data as an overlay in front of the camera image during the operation to support navigation. In addition, hidden vessels and the tumors can be made visible. The particular scientific challenge here lies, among other things, in the deformation of the human body between the recording of the preoperative data and the subsequent intervention, as well as during the actual operation.
The tasks of photogrammetry in the project are the development of a concept for the calibration of the used stereo endoscope, taking into account the constraints and routines in the operating room. In addition, improvements in spatial reconstruction of the dynamic environment are developed.
Demo of the developed calibration method.
A short demonstration of the overall system shows the possible process:
Demo of the developed prototype © NCT / UCC Dresden
Addional information
Publications
Docea, R., Pfeiffer, M., Müller, J., Krug, K., Hardner, M., Riedel, P., Menzel, M., Kolbinger, F. R., Frohneberg, L., Weitz, J., & Speidel, S. (2022). A Laparoscopic Liver Navigation Pipeline with Minimal Setup Requirements. In BioCAS 2022 - IEEE Biomedical Circuits and Systems Conference: Intelligent Biomedical Systems for a Better Future, Proceedings: Intelligent Biomedical Systems for a Better Future, Proceedings (pp. 578-582). [9948587] IEEE Xplore. https://doi.org/10.1109/BioCAS54905.2022.9948587
Müller, J., Docea, R., Hardner, M., Krug, K., Riedel, P., & Tetzlaff, R. (2022). Fast High-Resolution Disparity Estimation for Laparoscopic Surgery. In BioCAS 2022 - IEEE Biomedical Circuits and Systems Conference: Intelligent Biomedical Systems for a Better Future, Proceedings: Intelligent Biomedical Systems for a Better Future, Proceedings (pp. 573-577). [9948563] IEEE Xplore. https://doi.org/10.1109/BioCAS54905.2022.9948563
Hardner, M., Docea, R., & Schneider, D. (2022). Guided Calibration of Medical Stereo Endoscopes. In International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives (XLIII-B2-2022 ed., Vol. 43, pp. 679-686). The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences Vol. XLIII-B2-2022 https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-679-2022
Project partner
-
Klinik und Poliklinik für Viszeral-, Thorax- und Gefäßchirurgie (VTG)
-
TU Dresden, Professur für Softwaretechnologie (SWT)
-
TU Dresden, Professur für Grundlagen der Elektrotechnik (ETIT)
-
TU Dresden, Professur für Multimedia-Technologie (MT)
Contact
- M.Sc. Matthias Hardner (project work)
- Prof. Dr. Danilo Schneider (project management)
Funding
This project is co-financed from tax revenues on the basis of the budget decided by the members of the Saxon state parliament. TG70 / SAB
