Zum Hauptinhalt

Sound Gesture Intelligence

Dr. Greg Beller

Direct mapping between voice and gesture

Air Theremin

While gyroscope and magnetometer data are static and can therefore be used to continuously control sound processes (a value can be mapped directly to a slider), this is not the case for accelerometer data, which only have values when they are moved. However, with the sensor at rest, a constant, non-zero value appears on the accelerometer's Z axis, due to terrestrial gravitation. In the absence of a magnetometer, this offset can be projected onto the XYZ axes to estimate static rotation angles, as with a magnetometer. In the beginning of [to do] Babil-on, this offset of acceleration due to gravity, projected onto the vertical axis of the hand, provides a continuous controller of a sound effect (in this case, the transposition and gain of a voice sample).

Babil-on, with Richard Dubelski, CMMR2013, Marseille France, October 2013.