Computer vision techniques for laparoscopic surgery training

Researchers from Biomedical Engineering and Telemedicine group at Universidad Politécnica de Madrid (UPM) are participating, along with Jesús Usón Minimally Invasive Centre, Delft University of Technology and Leiden University Medical Centre in the development of technologies to perform more objectively assessment processes of surgeons in simulation environments. In this context, researchers have developed EVA Tracking System, a tracking system of surgical instruments that enables the study and assessment of instrumental movements within a simulator, this system has become the main source of objective measure for surgical expertise.

Unlike other tracking systems based on sensors, EVA uses computer vision techniques to obtain spatial location of laparoscopic instruments only using the endoscopic camera image. These features give as a result a low cost portable tracking system that does not require instrumental modification. Besides, this system is able to accurately predict the expertise of a surgeon in the simulator.

Laparoscopic surgery allows us to lessen the post operative trauma and favoring a faster recovery of patients. The training of the required skills is complex and the surgeon has to learn indirect surgery through a monitor and without physical contact with the inner body. In order to prevent accidents that can put patients at risk, it is required to develop environments that allow surgeons to acquire skills outside the operating room. An example of this are simulators, both physical and virtual.

One of these simulators was used as a test environment to validate the monitoring system for surgical instruments developed by UPM researchers. From the obtained measures, EVA Tracking System can estimate diverse objective evaluation metrics such as the surgical expertise in a simulation task. These metrics are, for example, total travel path, speed, average acceleration, volume occupied or motion smoothness.

Final paths measured in a simulator task. Each color means a different tool. / UPM.

The research carried out in this field so far, apart from the development of EVA Tracking System, has described the integration of laparoscopic physical simulator and its usage with laparoscopic surgeons. Firstly, researchers showed that the estimated metrics allow us to discern the expertise of novice and expert surgeons and setting the correlation of the mentioned metrics with the ones obtained by other optical tracking system. Secondly, the metrics data were used to train a classifier that predicts the skills of a surgeon in the simulator with a success rate of up to 83.3%. This result shows a direct correlation between the previous experience of the surgeon and the simulation task performing.

Today, the EVA system is included in an application that will allow us to store and manage data of surgeons when monitoring their progress. Besides, researchers are working to include EVA as a control device for virtual simulation environments or serious games all this within the framework of the KT European project.

OROPESA I, SÁNCHEZ-MARGALLO JA, SÁNCHEZ-GONZÁLEZ P, PAGADOR JB, TSOLKAS G, SKARMEAS N, SÁNCHEZ-MARGALLO FM, GÓMEZ EJ (2015). “Controlling virtual scenarios for minimally invasive surgery training using the EVA Tracking System”. Libro de actas del XXXIII Congreso Anual de la Sociedad Española de Ingeniería Biomédica (CASEIB2015): 431-434.

OROPESA I, SÁNCHEZ-GONZÁLEZ P, SÁNCHEZ-MARGALLO JA, GARCÍA-NOVOA J, SÁNCHEZ-MARGALLO FM, GÓMEZ EJ (2014) “EVA: Endoscopic Video Analysis of the Surgical Scene for the Assessment of MIS Psychomotor Skills”. XIII Mediterranean Conference on Medical and Biological Engineering and Computing, IFMBE Proceedings, 41:52-56.

OROPESA I, SÁNCHEZ-GONZÁLEZ P, CHMARRA MK, LAMATA P, FERNÁNDEZ A, SÁNCHEZ-MARGALLO JA, JANSEN FW, DANKELMAN J, SÁNCHEZ-MARGALLO FM AND GÓMEZ EJ (2013). “EVA: Laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment”. Surgical Endoscopy; 27(3):1029-1039.

Main photo: Instrument tracking based on endoscopic video analysis. / UPM.