Movement sonification tools and methods 33:45
- Legos project: Sensori-motor learning in gesture-sound interactive systems : International Workshop on Movement Sonification and Learning
- March 2, 2015
- Ircam, Paris
- Program note: Legos
Participants
- Jules Françoise (conférencier)
We present several tools we developed for designing interactive sonification strategies. In particular, we developed an ecological approach of continuous movement sonification we call “mapping by demonstration”. The relationship between motion and sound is defined by providing movement examples performed while listening to sound examples. The system automatically learns the motion-sound mapping by jointly modeling movement and sound data. The system integrates specific probabilistic models with hybrid sound synthesis models.
We will describe and demonstrate two specific implementations of the mapping by demonstration method.
The first one makes use of the Leap motion to sonify movement with sound textures.
The second system makes use of inertial measurement units and vocalizations as sound material.