Development of an augmented reality simulator for a robotic system used in single incision laparoscopic surgery
Keywords:
robotic assisted SILS, augmented reality, kinematics, AR simulator, Hololens 2Abstract
Augmented Reality (AR) became a very useful tool for various domains, with multiple applications in different medical fields, where this technology is integrated in medical training, image-based diagnosis, simulation of medical tasks, or in guiding systems for surgical procedures. The introduction of AR technology in robotic assisted surgery is relatively new and challenging considering the required precision for this type of medical task. The objective of this study is to develop an AR simulator for robotic assisted Single Incision Laparoscopic Surgery (SILS) where the robot performing the procedure is embedded in a virtual environment, providing both an interactive training and a simulation environment. The architecture and particularities of the proposed robot for SILS are analyzed for a better understanding of the structure, to obtain the validation of the kinematic model. The final result is represented in the form of an application which can be operated with Hololens 2 device.
References
CHEN, Y., Wang, Q., CHEN, H., SONG, X., TANG, H., & TIAN, M., An overview of augmented reality technology, Journal of Physics: Conference Series, 1237, 2, 022082, 2019.
BILLINGHURST, M., CLARK, A., LEE, G., A survey of augmented reality, Foundations and Trends® in Human-Computer Interaction, 8, 2-3, pp. 73-272, 2015.
BOAS, Yuri Antonio, GONÇALVES, V., Overview of virtual reality Ttchnologies, 2012.
GEORGE, E.I., BRAND, T.C., LAPORTA, A., MARESCAUX, J., SATAVA, R.M., Origins of robotic surgery: From skepticism to standard of care, JSLS, 22, 4, e2018.00039, 2018.
PAILHÉ, R., Total knee arthroplasty: Latest robotics implantation techniques, Orthop Traumatol Surg Res., 2021 Feb., 107, 1S, 102780, 2021.
GHERMAN, B., VAIDA, C., PISLA, D., PLITEA, N., GYURKA, B., LESE, D., GLOGOVEANU, M., Singularities and Workspace Analysis for a Parallel Robot for Minimally Invasive Surgery, 2010 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR), pp. 1-6, doi: 10.1109/AQTR.2010.5520866, 2010.
KARASIN, B., HARDINGE, T., ESKUCHEN, L., WATKINSON, J., Care of the Patient Undergoing Robotic-Assisted Brain Biopsy With Stereotactic Navigation: An Overview, AORN J. 2022 Mar, 115, 3, pp. 223-236, 2022.
VAIDA, C., PISLA, D., SCHADLBAUER, J., HUSTY, M., PLITEA, N., Kinematic Analysis of an Innovative Medical Parallel Robot Using Study Parameters, In: Wenger, P., Chevallereau, C., Pisla, D., Bleuler, H., Rodic, A. (eds), New Trends in Medical and Service Robots. Mechanisms and Machine Science, 39, Springer, Cham., pp. 85-99, 2016.
PISLA, D., VAIDA, C., BIRLESCU, I., HAJJAR, N.A., GHERMAN, B., RADU, C., PLITEA, N., Risk Management for the Reliability of Robotic Assisted Treatment of Non-resectable Liver Tumors, Appl. Sci., 10, 52, 2020.
SAIDY, MARYAM N et al., Single-incision laparoscopic surgery--hype or reality: a historical control study, The Permanente journal, 16, 1, 2012.
PISLA, D., GHERMAN, B., TUCAN, P., BIRLESCU, I., PUSCA, A., RUS, G., PISLA, A., VAIDA, C., Application oriented modelling and simulation of an innovative parallel robot for single incision laparoscopic surgery, Proceedings of the ASME, IDETC/CIE2022 August 14-17, St. Louis, Missouri, 2022.
RIVAS-BLANCO .I., P. del Saz-Orozco, I. GARCIA-MORALES, MUNOZ, V., Robotic system for single incision laparoscopic surgery, IECON 2012 - 38th Annual Conference on IEEE Industrial Electronics Society, pp. 2774-2779, 2012.
TUCAN, P., GHERMAN, B., ANDRAS, I., VAIDA, C., PISLA, D., Kinematic Modelling of a Parallel Robot Used in Single Incision Laparoscopic Surgery, In: Kecskeméthy, A., Parenti-Castelli, V. (eds) ROMANSY 24 - Robot Design, Dynamics and Control, ROMANSY, CISM International Centre for Mechanical Sciences, 606, Springer, Cham., pp. 115-122, 2022.
VAIDA, C., PLITEA, N., PISLA, D. et al., Orientation module for surgical instruments-a systematical approach, Meccanica, 48, pp. 145-158, 2013.
YANG Y, Li J, KONG K, WANG S., Design of a dexterous robotic surgical instrument with a novel bending mechanism, Int J Med Robot, 2022 Feb., 18, 1, e2334, 2022.
PISLA, D., CARAMI, D., GHERMAN, B., SOLETI, G., ULINICI, I., VAIDA C., A novel control architecture for robotic-assisted single incision laparoscopic surgery, The Romanian Journal of Technical Sciences. Applied Mechanics, 66, 2, 2021.
VAIDA, C., CRISAN, N., BIRLESCU, I., ANDRAS, I., PISLA, D., Preliminary assessment of artificial intelligence agents for a SILS robotic system, 2021 International Conference on e-Health and Bioengineering (EHB), pp. 1-6, 2021.
OYAMADA Y., A ‘Look Into’ Medical augmented reality, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2014.
JO Y.-J., CHOI J.-S., KIM, J., KIM, H.-J,, MOON, S.-Y., Virtual Reality (VR) Simulation and Augmented Reality (AR) Navigation in Orthognathic Surgery: A Case Report, Applied Sciences, 11, 12, 5673, 2021.
ROESSEL, J., KNOELL, M., HOFMANN, J., BUETTNER, R., A Systematic Literature Review of Practical Virtual and Augmented Reality Solutions in Surgery, IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC), pp. 489-498, 2020.
VAVRA, P., ROMAN, J., ZONCA, P., IHNAT, P., NEMEC, M., KUMAR, J., HABIB, N., EL-GENDI, A., Recent Development of Augmented Reality in Surgery: A Review, Journal of Healthcare Engineering, 2017, pp. 1-9, 2017.
CHOI, H., CHO, B., MASAMUNE, K., HASHIZUME, M., HONG, J., An effective visualization technique for depth perception in augmented reality-based surgical navigation, Int J Med Robot, 12, 1, pp. 62-72, 2016.
COLLINS J., CHAND S., VANDERKOP A. and HOWARD D., A Review of Physics Simulators for Robotic Applications, in IEEE Access, 9, pp. 51416-51431, 2021.
IVALDI, S., PADOIS, V., NORI, F., Tools for dynamics simulation of robots: a survey based on user feedback, 2014.
NX-https://www.plm.automation.siemens.com/global/en/products/nx/ - 13/10/2022.
PISLA, D., BIRLESCU, I., PUSCA, A., TUCAN, P., GHERMAN, B., VAIDA, C., Singularity analysis and geometric optimization of a 6-DOF parallel robot for SILS, Proceedings of the Romanian Academy, Series A, 2022, accepted for publication.
Published
Issue
Section
Copyright (c) 2023 The Romanian Journal of Technical Sciences. Applied Mechanics.
This work is licensed under a Creative Commons Attribution 4.0 International License.