Skip to main content

Sound Gesture Intelligence

Dr. Greg Beller

Hand Tracking

Depth-sensing cameras

To the fast capture of the dynamics of the gesture by the accelerometer gloves, one can add the relatively slow capture of the absolute position of the hands in space, by the use of depth cameras. The body skeleton can be tracked using a Microsoft Kinect [18], while the hand skeleton is tracked using a Leap Motion, for example. This makes it possible to obtain, in addition to the fine temporal precision of the percussive type triggering, the continuous control of sound processes according to the posture and the spatial position of the hands/fingers.

img_Microsoft Kinect.png
Microsoft Kinect

img_Leap Motion.png
Leap Motion


On the other hand, the depth sensor brings other constraints such as a reduction in the playing area, a single performer possible, the need for a phase of calibration and detection of the skeleton, the risk of infrared disturbance by lights.