Building a DVRK Dataset of Surgical Tasks for Autonomous Auxiliary Assistance

Some great projects coming out from the TERRINET program:

Dr. Blanco is an Assistant Lecturer in the Department of Systems Engineering and Automation at the University of Malaga (Malaga, Spain). She applied to the TERRINet project to build a dataset of robotic surgical tasks using the Da Vinci Research Kit platform (dVRK), offered by the TERRINet partner The BioRobotics Institute – SSSA (Pontedera, Italy).

Da Vinci Research Kit (Figure 1) is the leading reference in the field of surgical robotics. The platform consists of a surgeon’s console to tele-operate the surgery and a patient side system where the surgery takes place. The platform enables the execution of complex motions due to additional degrees of freedom of the instruments. The users can perform the motions that are not possible on other platforms.

Designing collaborative strategies for robots to aid surgeons in performing certain actions with autonomy can lead to increased precision and accuracy of robot-assisted operations, improved consistency in treatments, and higher dexterity and access to tissues.

Building a Dataset of Surgical Tasks for Autonomous Auxiliary Assistance

Surgical datasets containing surgical videos of real interventions are no novelty to the public. So far, they were mostly used for the recognition of surgical actions.

The objective of this experiment was to build a dataset of surgical tasks with different levels of complexity combining camera image and robotic system sensory data, which would serve for further exploration of deep learning techniques for autonomous auxiliary tasks in laparoscopic procedures.

Surgical Tasks Addressing Motions and Skills Necessary for Laparoscopic Surgery

The experiment implementation was divided into two access periods – the preparation period (July 2019) and the execution period (February 2020).

The preparation period was devoted to getting trained for the platform usage, software preparation and definition of the surgical tasks performed by the users. The surgical tasks were selected based on the SAGES manual skills in laparoscopy proposition (Choy, 2012).

In the execution period, 16 users performed the surgical tasks to obtain the required data (Figure 2). 10 users were male and 6 females ranging between 21 and 42 years old. Each user performed 6 trials of each exercise – 3 trials using the right hand and 3 using the left hand. 

 

Figure 2: Experimental setup conducted in the execution period. On the left picture: the master console of dVRK, where the operator tele operatively controls the instruments while watching the operational site in a 3D viewer. On the right picture: the slave arms are reproducing the operator movements.

The surgical tasks were the following:

• Post and Sleeve: The goal was to move the coloured sleeves from side to side, whereas the user was allowed to take a sleeve with one hand, pass it to the other, and place it over a peg on the other side.
• Pea on a Peg: The goal was to take beads from the cup and place them on top of the pegs using only a single hand.
• Wire Chaser: The goal was to move a ring from one side to the other using only one hand.

The surgical tasks addressed motions and skills necessary for laparoscopic surgery, such as hand-eye coordination, bimanual dexterity, depth perception, and interaction between the dominant and non-dominant hand. The teleoperation of the tools was performed with haptic guidance, using the dVRK console. The control of the teleoperation was carried out with a ROS software provided by the host institution.

Dataset of Surgical Tasks Based On 282 Recordings

The final dataset contained 282 recordings in total – 92 of post and sleeve task, 95 of pea on a peg, and 95 of wire chaser. The parameters were recorded on the master tool manipulators (44 recordings), patient side manipulators (80), and the pedal (40).

The mean scores of the surgical tasks (Figure 4) were 138.72 for post and sleeve, 152.25 for pea on a peg, and 42.94 for wire chaser. The mean score of the latter was lower due to a shorter duration of the task.

Figure 4: The distribution of the total score of the 282 trials divided by exercise using boxplots.

The dataset included the following data:

• Images (each exercise was recorded using the mp4 format),
• Kinematic data (kinematic data of the two master tool manipulators and patient’s side manipulators),
• Skills evaluation (the user performance evaluation based on time and task efficiency),
• Questionnaire (about the personal data, experience with teleoperated systems and visuo-motor skills).

“The TERRINet project was a unique opportunity to work with da Vinci Research Kit that I can’t access at my home institution. It was an amazing experience to get to know the platform, work on it, and collect data, which enabled further research on the topic”, Irene described her TERRINet experience.

She suggests applying to students and researchers who would like to implement their research experiment on a platform not available at their home institution and would like to expand the knowledge from a different research group.

References

• Choy I, Okrainec A. Fundamentals of Laparoscopic Surgery-FLS. In: The SAGES Manual of Quality, Outcomes and Patient Safety. Boston, MA: Springer US; 2012. p. 461–71.