Research intro - UC Berkeley's RAMIS calibration
It is always a good idea to check Ken Goldberg's group AUTOLAB website at UC Berkeley.
Hwang et al.'s ICRA paper was on "Efficiently Calibrating Cable-Driven Surgical Robots With RGBD Sensing, Temporal Windowing, and Linear and Recurrent Neural Network Compensation", which received a nice visualization on the website:
Abstract
"Automation of surgical subtasks using cable-driven robotic surgical assistants (RSAs) such as Intuitive Surgical's da Vinci Research Kit (dVRK) is challenging due to imprecision in control from cable-related effects such as backlash, stretch, and hysteresis. We propose a novel approach to efficiently calibrate a dVRK by placing a 3D printed fiducial coordinate frame on the arm and end-effector that is tracked using RGBD sensing. To measure the coupling effects between joints and history-dependent effects, we analyze data from sampled trajectories and consider 13 modeling approaches using LSTM recurrent neural networks and linear models with varying temporal window length to provide corrective feedback. With the proposed method, data collection takes 31 minutes to produce 1800 samples and model training takes less than a minute. Results suggest that the resulting model can reduce the mean tracking error of the physical robot from 2.96mm to 0.65mm on a test set of reference trajectories. We evaluate the model by executing open-loop trajectories of the FLS peg transfer surgeon training task. Results suggest that the best approach increases success rate from 39.4% to 96.7% comparable to the performance of an expert surgical resident.
Joint Estimation
Sphere fiducials to estimate the pose of the end effector. We place two spheres on the shaft to obtain wrist position and four on the cross-shaped reference frame to find the orientation of the jaw. Yellow circles and dots indicate the detected spheres and its center locations, which are estimated via least squares from the point cloud obtained by RGBD sensing. The green dotted lines show a skeleton of the estimated tool posture, which is estimated from the spheres by solving an inverse kinematics problem.
Error Analysis
We investigate how the commanded trajectories q_c compare to the physical trajectories q_p. Top Row: We observe that the tool joint angles (q_4, q_5, q_6) are not affected by the movement of the external robot arm (q_1, q_2, q_3). The RMSE values for (q_4, q_5, q_6) respectively are [0.187, 0.193, 0.360] and [0.175, 0.207, 0.347] before and after fixing the external arm. Bottom Row: Two wrist joint angles, q_5 and q_6, are closely coupled to each other. Joint q_6 moved in correspondence to q_5 despite its desired command was fixed as constant, and vice versa.
Fiducial-free State Estimation
We investigate fitting both forward and inverse dynamics models on trajectory data collected on the robot with the fiducial spheres. We investigate an array of modeling approaches and find that history is necessary in order to accurately predict the robot's errors. We hypothesize this is due to the history-dependent nature of cabling effects such as hysteresis, cable stretch, and backlash. The left figure below reports offline joint configuration MSE for different forward modeling approaches. The middle figure reports offline joint configuration MSE for the inverse model. The right figure and table below reports the error distribution of the end effector position for different modeling approaches for reference trajectories run on the physical robot. Please refer to the paper for additional details.
Linear Regression Analysis
We visualize sections of the matrices when the model is fit via LASSO to provide insights about the sources of temporal correlation below.
Peg Transfer Experiments
We evaluate the learned controllers on a peg transfer task, similar to a surgeon training task in the Fundamentals of Laparoscopic Surgery. This task involves transferring 6 blocks from one set of pegs to another, then transferring them back. We report transfer success rates for the method below. The baseline is the uncalibrated robot, and the two datasets are a set of random trajectories and pick and place style trajectories. We observe that even the controller trained on random trajectories is able to increase peg transfer success rate from 39.4% to 69.2%. When trained on pick and place data, which is similar to the distribution of data encountered in the task, the controller is able to successfully complete the task 96.7% of the time."
"Automation of surgical subtasks using cable-driven robotic surgical assistants (RSAs) such as Intuitive Surgical's da Vinci Research Kit (dVRK) is challenging due to imprecision in control from cable-related effects such as backlash, stretch, and hysteresis. We propose a novel approach to efficiently calibrate a dVRK by placing a 3D printed fiducial coordinate frame on the arm and end-effector that is tracked using RGBD sensing. To measure the coupling effects between joints and history-dependent effects, we analyze data from sampled trajectories and consider 13 modeling approaches using LSTM recurrent neural networks and linear models with varying temporal window length to provide corrective feedback. With the proposed method, data collection takes 31 minutes to produce 1800 samples and model training takes less than a minute. Results suggest that the resulting model can reduce the mean tracking error of the physical robot from 2.96mm to 0.65mm on a test set of reference trajectories. We evaluate the model by executing open-loop trajectories of the FLS peg transfer surgeon training task. Results suggest that the best approach increases success rate from 39.4% to 96.7% comparable to the performance of an expert surgical resident.
Joint Estimation
Sphere fiducials to estimate the pose of the end effector. We place two spheres on the shaft to obtain wrist position and four on the cross-shaped reference frame to find the orientation of the jaw. Yellow circles and dots indicate the detected spheres and its center locations, which are estimated via least squares from the point cloud obtained by RGBD sensing. The green dotted lines show a skeleton of the estimated tool posture, which is estimated from the spheres by solving an inverse kinematics problem.
Error Analysis
We investigate how the commanded trajectories q_c compare to the physical trajectories q_p. Top Row: We observe that the tool joint angles (q_4, q_5, q_6) are not affected by the movement of the external robot arm (q_1, q_2, q_3). The RMSE values for (q_4, q_5, q_6) respectively are [0.187, 0.193, 0.360] and [0.175, 0.207, 0.347] before and after fixing the external arm. Bottom Row: Two wrist joint angles, q_5 and q_6, are closely coupled to each other. Joint q_6 moved in correspondence to q_5 despite its desired command was fixed as constant, and vice versa.
Fiducial-free State Estimation
We investigate fitting both forward and inverse dynamics models on trajectory data collected on the robot with the fiducial spheres. We investigate an array of modeling approaches and find that history is necessary in order to accurately predict the robot's errors. We hypothesize this is due to the history-dependent nature of cabling effects such as hysteresis, cable stretch, and backlash. The left figure below reports offline joint configuration MSE for different forward modeling approaches. The middle figure reports offline joint configuration MSE for the inverse model. The right figure and table below reports the error distribution of the end effector position for different modeling approaches for reference trajectories run on the physical robot. Please refer to the paper for additional details.
Linear Regression Analysis
We visualize sections of the matrices when the model is fit via LASSO to provide insights about the sources of temporal correlation below.
Peg Transfer Experiments
We evaluate the learned controllers on a peg transfer task, similar to a surgeon training task in the Fundamentals of Laparoscopic Surgery. This task involves transferring 6 blocks from one set of pegs to another, then transferring them back. We report transfer success rates for the method below. The baseline is the uncalibrated robot, and the two datasets are a set of random trajectories and pick and place style trajectories. We observe that even the controller trained on random trajectories is able to increase peg transfer success rate from 39.4% to 69.2%. When trained on pick and place data, which is similar to the distribution of data encountered in the task, the controller is able to successfully complete the task 96.7% of the time."
Source: AUTOLAB
Comments