Gestonurse: A robotic assistant that understands verbal and non-verbal communication

Going more into RSN system: we have already cited the great work being done at Purdue University on developing a scrub nurse robot. You can get a lot of information about the system from the recent publication: "Robot, Pass Me the Scissors! How Robots Can Assist Us in the Operating Room" written by Juan P. Wachs.

 The system architecture  of the system contains a FANUC robot, a Kinect, a CCD camera and adjont control. "The depth   maps  obtained  from  the  Kinect  sensor  were  processed  by  the  gesture  recognition   module and in parallel the  speech commands were captured by a microphone. These   commands  were interpreted using the speech recognition  module. Once recognized,   the command was sent through the network to the FANUC robotic arm using a Telnet interface. The robot, in turn, delivers the required surgical instrument to the surgeon   and stays ‘on hold’ until the next command is received."   

Update: a brand new article in the Communications of ACM magazine.

 
Credit: JP. Wachs@Springer