Hacked Kinect Understands Sign Language

This article is over 13 years old and may contain outdated information

Recommended Videos

French researchers have done what many thought impossible: created a truly useful hack for the Kinect. Using the body tracking capabilities of the Kinect, coupled with some clever coding, they have developed software to interpret sign language. Interestingly, this fulfills the original patent claim of the Kinect, and promises made at its introduction by Microsoft.

Key to the team’s work was using the Fast Artificial Neural Network, or FANN, to allow the computer to understand users’ signs. From the  YouTube comments made by the team:

For simples moves , like a punch or a kick, it easy to recognize : high vélocity of the hand or the foot. But for more complex moves (like the ones in this video), the problem is mathematicaly harder…

Neural network is a way to bypass the problem[.] It builds a mathematical model of the moves using move samples.

The system can only recognize two words at the moment — “hello” and “sorry” — but the team says that with all the software in place adding more words is a relatively easy task. While the XBox has supported text-based inputs for some time, sign language recognition could provide a fast and natural way for signers to communicate. It could also be used as a valuable teaching tool for those unfamiliar with the language.

Read on after the break to get a look at the Kinect hack in action.

(Image and story Ãœbergizmo via Kotaku)


The Mary Sue is supported by our audience. When you purchase through links on our site, we may earn a small affiliate commission. Learn more about our Affiliate Policy
Author
Image of Max Eddy