Surgeons: Operate With Kinect

Surgeons use leap motion to interact with imaging data during procedure

Check out the movie on YouTube

Robotic Surgery Assistant

A group of Purdue University, in West Lafayette, Ind. developed a robotic system that acts as a surgery assistant providing the surgeon with the correct tools using gesture and voice commands. Starting as a 'joke' during a brainstorm to build a robotic nurse, but now an international research project. 
Check it out on youtube:

http://youtu.be/s8eTLT1QexM

Also check out the coverage on IEEE Spectrum: Profile: Juan Wachs

Nomination for Innovation Award of the Hanze Institute of Technology for Kinect project

One of the students working on an assignment in my group to evaluate the use of the MS Kinect for use in the operating theatre has been nominated for the innovation award of the Hanze Institute of Technology based on this thesis work. The official press release can be found here.

Just a short demonstration video:

The student used the Kinect to allow a surgeon to walk through the data available for a patient using minimal gestures using an easy and intuitive user interface. We are currently trying to arrange the continuation of his project.

New and improved gesture based interaction for medical data using MS Kinect

The guys at iHealthLab in Italy further improved their gesture based interaction system for medical data using the Kinect. They have included interaction with 3D visualizations and all kinds of other cool features. Check it out at their website.

LEAP Motion sensor - DEMO by ENGADGET

Now there are also other movies with a more 'live' demonstration of the amazing leap motion sensor. Here is one from engadget. This really seems to be true!!

Kinect imaging lets surgeons keep their focus - tech - 17 May 2012 - New Scientist

THE surgeon enters the operating theatre, covered in sterile blue scrubs. Machines beep and hiss. Nurses wait, tools at the ready: scalpel, forceps, bandage, Xbox... Xbox?

On Tuesday last week, a surgeon at Guy's and St Thomas' hospital in London began trials of a new device that uses an Xbox Kinect camera to sense body position. Just by waving his arms the surgeon can consult and sift through medical images, such as CT scans or real-time X-rays, while in the middle of an operation.

Read the full article at:

Kinect imaging lets surgeons keep their focus - tech - 17 May 2012 - New Scientist

LEAP Motion sensor - Novel gesture-based interaction at high accuracy

Leap represents an entirely new way to interact with your computers. It’s more accurate than a mouse, as reliable as a keyboard and more sensitive than a touchscreen.  For the first time, you can control a computer in three dimensions with your natural hand and finger movements.

This isn’t a game system that roughly maps your hand movements.  The Leap technology is 200 times more accurate than anything else on the market — at any price point. Just about the size of a flash drive, the Leap can distinguish your individual fingers and track your movements down to a 1/100th of a millimeter.

This is like day one of the mouse.  Except, no one needs an instruction manual for their hands.

If this is for real they really have a breakthrough technology at their hands. This would really enable the introduction of gesture-based interaction into medicine with high accuracy and flexibility with minimal movements.
Check it out at their website.

Interaction Beyond the Keyboard

IEEE Computing Now features an article on interaction beyond the keyboard. In this article Albrecht Schmidt from the University of Stuttgart, Germany and Elizabeth Churchill from Yahoo Research explore possibilities to replace the conventional mouse and keyboard to achieve a more intuitive and versatile way of interacting with computers to explore the world of online communications and information access.

Gesture based control of AquariusNET using Kinect

Using the Kinect with the TeraRecon AquariusNET software. I'm only wondering how they change between the different operations since they do not show that. Probably they change the function using the mouse and then interact again with gestures.

Hand Tracking

Little bit older already, but still a very good handtracking that might be beneficial in manipulation of medical data since they capture a lot of different gestures with only a simple webcam.

KineMed - Gestural interface for Philips iSite

Developing a gestural user interface for examination of medical images using the Kinect. This project was done in association with residents from UPMC, Pittsburgh and was demonstrated at RSNA, 2011

Kinect and Medical Imaging

This is the first time that I really see some kind of workflow using gesture based interfaces. Most only interact with already selected data, here the selection of the data is also included.


Josh Wall from the InfoStrat Adavanced Technology Group demos a medical imaging application that uses the InfoStrat Motion Framework

Gesture Interface using Kinect for Medical Imaging Visualization in Surgeries

This work was developed in the Renato Archer Center for Information Technogy - CTI (www.cti.gov.br) in Campinas - SP, Brazil (3D Technologies Division) published at the 5th International Conference on Advanced Research in Virtual and Rapid Prototyping (VRAP2001).  The system allows the surgeon to control the medical imaging software InVesalius (also developed by CTI) without any physical contact, just by hand gestures. The system uses the OpenNI/NITE framework and is cross-platform (Windows/Linux).  Invesalius website: http://svn.softwarepublico.gov.br/trac/invesalius