Sound Gesture Intelligence

To the fast capture of the dynamics of the gesture by the accelerometer gloves, one can add the relatively slow capture of the absolute position of the hands in space, by the use of depth cameras. The body skeleton can be tracked using a Microsoft Kinect [18], while the hand skeleton is tracked using a Leap Motion, for example. This makes it possible to obtain, in addition to the fine temporal precision of the percussive type triggering, the continuous control of sound processes according to the posture and the spatial position of the hands/fingers.

Microsoft Kinect

Leap Motion

On the other hand, the depth sensor brings other constraints such as a reduction in the playing area, a single performer possible, the need for a phase of calibration and detection of the skeleton, the risk of infrared disturbance by lights.