Sound Gesture Intelligence

In Luna Park, a musical theater work by Georges Aperghis, I integrated some dynamic sensors into gloves and developed a first instrument called SpokHands, which literally made it possible to speak with the hands (Beller 2011a, b, c, d). SpokHands allows the triggering and modulation of voice samples by aerial percussion and hand elevation. Like a vocal Theremin, SpokHands offers the performer the option of three-voice polyphony (her.his own and both hands) or control of text-to-speech parameters. In this case, the natural division of a conductor's brain is used, the left brain (right hand) for the segmental part, and the right brain (left hand) for expressivity. The percussive gestures of the right-hand trigger pre-selected syllables whose pitch and intensity are modulated by continuous gestures of the left hand.

Rehearsal video of Luna Park, 2010, Georges Aperghis, showing Richard Dubelski discovering SpokHands (Beller 2011a, b ,c, d) @IRCAM, Paris, France.