Medical robotics research at UMA

University of Malaga is the organizer of the upcoming European Robotics Forum (ERF), having a decent history in medical robotics research:

"The medical robotics research team from University of Malaga is comprised of professors of the Department of System and Automatic Engineering and the first Chair of Surgery. Since 1998, the members of this Engineering Group have been working in the fields of robotics and telemanipulation participating and conducting numerous research projects. At the same time, the researchers of the first Chair of Surgery have been pioneers in Spain in the field of laparoscopic surgery.

The Medical Robotics research team is aimed at developing robotic assistants for laparoscopic surgery. The purpose is defined as the creation of machines which are not intended to replace the doctor in surgical interventions, but to work side-by-side with him. The working method is focus on an early stage of new knowledge creation and a second stage leading to the construction of demonstrator systems. The developed prototypes exhibit a dual function: they are useful to experimentally verify the theoretical developments; and they facilitate the transmission of research data.

These lines of research lead to a series of abilities acquired by the Medical Robotics research team from University of Malaga. The classification of these abilities as well as the level acquired by the team for each project, have been built by taking into account the Robotics 2020 Multi-Annual Roadmap for Robotics in Europe document written by the SPARC Partnership for Robotics in Europe. Medical Robotics research team continues developing and improving the level of such abilities regarding to its activity on the research projects it is currently involved." Key projects involve:

Design of Surgical Robots

ISA (Intelligent Surgery Assistant) It is the first robotic assistant designed to be used in laparoscopic surgery; it is based on an industrial Stäubli RX60 robot. The robot must fulfil various requirements, such as accessing the entire working area by means of an endoscope without limiting the surgeon’s movements. The assistant is handled in response to the surgeon’s commands. In the left picture, it can be appreciated a clinical trial using animals for experimentation purposes. It was developed under the project framework FIS-00/0050-02.

ERM (Edoscope Robotics Manipulator) Robotic assistant developed under the project framework PI021708. It is designed to fulfil the kinematic requirements used for the ISA assistant, as well as the real-time needs found in a human operating room. In this sense, there was emphasis on the fact that the robot used here does not require any structural modification in the operating room, and that it does not obstruct the surgeons’ movements. In the left picture, it is shown the ERM, version 3.0.

RTUP (Trans-Uretral Resection Prostate) The left picture shows the ISA robot adaptation for the transurethral resection of the prostate. The intervention involves passing a narrow metal telescope, attached to the robot end effector, through the patient’s penis until it reaches the prostate. The robot is handled by a master manipulator.

CISOBOT It consists of a two-arm robot, one to control the laparoscopic camera and another to help the surgeon operate along the intervention in a semi-autonomous mode. The objective of the CISOBOT system is to help the surgeon operate alone. To that end, the system is provided with a multimodal HMI leading the robot to a higher reaction capacity and a high level intelligence. This robotic assistant has been developed under the project frameworks DPI2007-62257 y P07-TEP-02897.

Miniatured Robots We have designed a set of miniature robots which are introduced into the abdominal cavity through one of the incisions surgeons perform to operate. These devices are provided with a magnet for the attachment to the abdominal wall. Two kinds of robots have been designed: a miniature camera robot, provided with a high definition camera and an illumination system, and a lighting-robot, which only incorporates a lighting system and makes it possible to direct the light source in order to create shadows in the image.

 Movements Control of the Instruments

Positioning problems of surgical instruments The endoscope is inserted through the abdomen at the so called point of fulcrum. It consists of positioning the instrument with a steering angle α, a rising angle β and positioning the camera at a distance ρ of the fulcrum. These spherical coordinates are defined by a coordinates system expressed in the point of fulcrum. These aspects are detailed in the left picture.

Passive wrist mechanism It is a wrist mechanism used in the surgical assistant consisting of two joints perpendicular to each other (see figure on the left). Thanks to the joint encoders, α and β-positions can be calculated, however ρ (defined as the distance from the passive wrist to the fulcrum) is impossible to be calculated. This latter increases the accuracy in placing the endoscope.

Passive wrist control scheme The control schemes for surgical tools are based on the estimation of the distance ρ (from the camera to the point of fulcrum) through geometrical methods. Likewise, they do not avoid the orientation errors resulting from the endoscope due to a bad estimation of ρ. The suggested scheme (see figure on the left) proposes a solution to this. It is based on the control theory, which does not just estimate ρ, but it also avoids the orientation errors mentioned above. The controller has two nested loops with two different functionalities. The inner loop imposes the robot a specific dynamic behaviour when following a Cartesian trajectory. The outer loop carries out two actions: it avoids the orientation errors resulting from the endoscope by means of spherical control, and at the same time, the geometric estimator calculates the ρ value in order to reduce minimally the estimation errors. This coupling effect provides strength and accuracy to the control scheme. This strategy has been implemented in the ERM robo.

Three-Layer Control by Passive Wrist Emulation Robots with active wrists can apply undesired forces to the abdominal wall of the patient if any uncertainty concerning the fulcrum location is produced. In order to solve this issue, it has been proposed a methodology that allows the emulation of a passive wrist with an actuated one. To achieve this, the measure of the force applied to the abdomen permits the fulcrum estimation in order to control the endoscope’s orientation. The control scheme, which is divided into three layers to facilitate the system stability study, consists of an inner layer based on the Cartesian control for the robot, an intermediate layer devoted to control the estimated fulcrum position, and an outer layer devoted to control the spherical positioning of the endoscope.

Autonomous Planning

(back to top)

Automatic guidance system It consists of reaching a position defined by a surgeon’s tool, whereas the other instrument is considered as an obstacle the robot must avoid. To that end, the current velocity of the robot and the obstacle tool is processed through a Fuzzy Logic algorithm for planning the robot’s trajectory with the combination of three behaviors: the first one is devoted to find a target trajectory avoiding static obstacles by means of the artificial potential fields method; the second one is devoted to correct the robot’s velocity depending on the obstacle motion and the distance between the two tools; finally, a third behavior is needed to displace the robot-guided tool in the direction of the obstacle when the collision is forced by the surgeon.

Automated systems The automated systems are developed to assist certain movements distinctive of the operations. Specifically, it details the automation of one of the surgical tools to assist surgeons during a suture procedure. The total system detects the different movement phases in suture procedures through a maneouvre recognition system and a computer vision system. This latter detects the needle position and orientation so that the robot can collaboratively work with the surgeon.

  

Fault Tolerant Control Architectures

Functional architecture An essential requirement in medical applications is the guarantee of the system safety. To achieve this, the first step is to design a system based on a set of modules which have a specific functionality. A surveyor process will keep the modules in good working order (centre of the left figure). If any anomalous situation happens, it will activate the “failure mode” of the surgical assistant. This technique needs to study all possible errors appearing in every module, to analyze the way they may be detected, and to plan to act accordingly.

UML design The surveyor programming of the functional architecture can also be critical. In order to ensure that the program is free of errors, a codification is followed with some styles rules. Likewise, a series of unit tests, functional integration tests and hardware integration tests are conducted in every UML module. The left figure details the UML scheme programmed in the surveyor of the functional architecture.

 Human-Machine Interfaces

PDA-based interface The use of a PDA device makes possible a comfortable communication with the robot assistant. It does not only emit motion commands to the robot assistant but it can also carry out a series of supervision tasks. Specifically, it can be used to obtain information from the status of the robot, motion precision, or to keep a record of all performed activities. The left figure shows the images of the PDA-based interface.

Speech command recognition It is a system containing DSP-based speech recognition software. In such system, the coded motion commands can be digitally sent to the robot assistant. With that, the problem resulting from the interferences of the analog transmissions-based solutions is solved. Besides, users can add the voice profile to their cards so the only orders to be recognized are those coming from the selected profile. In the picture on the left, it can be appreciated the developed system.

Recognition of surgical gestures The recognition of the surgeon’s surgical gestures or manoeuvres along the intervention permits to know the current situation of the surgical protocol. This provides a more natural and comfortable communication between the surgeon and / or the robot, and so it is considered as an alternative way for sending orders or commands. A manoeuvres library and a manoeuvres recognition system have been constructed in order to recognize the current surgeon’s manoeuvre. It is achieved by means of a surgical task modeling in which the Hidden Markov Models have been used. The recognition system includes a data collection system that informs about the surgeon’s surgical tools positioning using 3D sensors, and in this way the tool interactions are achieved.

Multimodal interface The development of a robotic assistant for laparoscopic procedures, which can semi-autonomously operate working with the surgeon in the management of the optic and the additional support tool, has provided the interface design with new challenges. The required interface to control this kind of robot should not overload the surgeon with many commands and button bars, reason why it has been developed a system which can recognize the surgeon’s surgical gestures in order to know the current intervention phase or the specific surgeon’s commands. The multimodal interface combines both the gestural and voice commands in such a way that it is similar to the relationship between the surgeon and the human assistant.

 Telesurgery

Teleoperation of the ERM assistant In the pictures, it is firstly appreciated, on the upside, a group of surgeons in the operating room during an intervention with an animal for experimentation purposes, and on the downside, the remote surgeon with an interface monitor, via which the surgeon can see the image of the laparoscopic camera, and a space ball with which he can control the robot. The task of the remote surgeon is to supervise the operation performed in order to act in three ways: i) to communicate with the local surgeons via videoconference, ii) to make marks in its monitor so that local surgeons can see them on theirs, or iii) to take control of the robot in order to center the camera on a specific area of interest.

Remote teleoperation of an automated operating room On the left pictures, it is appreciated the system configuration. On the upside, it is shown the remote surgeon with an interface containing a monitor on which the laparoscopic image is displayed via IP, and two haptic devices. On the downside, it is appreciated the automated operating room containing the ERM robotic assistant devoted to handle the endoscope, and two manipulator robots using each one a tool. This way, the remote surgeon controls the ERM through the voice, and the manipulator arms, which are provided with a force sensor, through the haptic devices.

Force feedback Control Teleoperated surgical robots allow the surgeon a straight control over the actuation of the manipulators, which provide a higher precision on the movement of the surgical tools. However, this mechanism avoids the physical contact between the surgeon and the patient, which leads to a loose of both tactile and pressure sensations. In order to minimize the effect of this limitation the group of Medical Robotics at University of Malaga has developed teleoperated control systems with force feedback which aim to recover part of that contact sensations for the surgeons. This control methodology requires to solve some aspects which are inherent to this kind of systems. For example, the force signal delay sent from the force sensor attached to the manipulator end effector towards the haptic device managed by the surgeon, as well as the measure distinction between the force applied on the insertion point or fulcrum and the force exerted by the tool tip over the internal patient tissues.

 Clinical Essays

ISA assistant It is the first prototype designed to be used in laparoscopic surgery and based on an industrial manipulator. It was used to study the viability of the robot assistant for handling the endoscope in this kind of operations. In-vitro tests were firstly performed to evaluate the efficacy of robot in laparoscopic camera positioning. It was followed by the phase of clinical trials with animals for experimentation purposes in which interventions such as the cholecystectomy, the Nissen fundoplication, or the removal of part of the colon were evaluated. On the left, it is appreciated a picture of one of the interventions with animals.

Automated Cystoscopy It was an adaptation technique of ISA assistant for the examination of the urinary bladder. The essays were performed with animals for experimentation purposes, and this time, the cystoscope was inserted directly into the animal’s urethra. The success of these essays provided evidence that a same robot can perform more than one task, representing a cost recovery in a shorter space of time. On the left picture, it is appreciated one of the essays with animals for experimentation purposes.

ERM assistant The robot was used in human clinical medicine. This system has been tested with thirty two patients in clinical trials. To achieve this, the robot was adapted in order to fulfil the demanding requirements concerning the electro-medicine machines. A clinical protocol for the description of the study field was also designed. It was concluded that, through this robot’s use, the amount of time of the operation as well as the surgeon’s fatigue were significantly reduced. Similarly, another advantage was that the patient needed less time to recuperate after the operation. On the left, there is a picture showing one of the operation moments.

In-vitro essays for the analysis of surgical manoeuvres The in-vitro manoeuvres essays have been carried out taking into account the study of clinical protocols for acquiring the surgeon’s model, particularly the cholecystectomy procedure. These in-vitro essays have enabled the construction of a maneouvre recognition system contained in the multimodal interface, and the creation of the automatic movements so that the robot and the surgeon can collaborate in manoeuvres such as the suture technique.

Miniatured Robots We have performed a clinical trial with a porcine model to test the viability of using a miniature camera robot in a real environment. The main objective of this experiment was to analyze both the quality of the image provided by the camera, and the magnetic interaction through the abdominal wall.

Source: University of Malaga