Équipe Analyse/Synthèse

Axel Mulder, Infusion Systems/ATR - Japan, Invited Lecturer.

Axel Mulder presented a talk on his research at the ATR Center in Kyoto, Japan on Empty-Handed Gesture Analysis in MAX/FTS.

The main applications of his research are Sound Sculpting (see below). He proposes an interesting strategy for mapping hand movements to sound: through shapes of a virtual input device controlled by both hands. He uses two Virtual Technologies Cybergloves and a Polhemus Fastrak sensor controlling an SGI running MAX/FTS.

During his talk many questions were raised about his work and also broadly about gestural capture and gestural control. One particularly interesting discussion was about the way to perform movement acquisition: the acquisition of position or the velocity/acceleration and their virtues/drawbacks.

Next, we present an abstract of his talk and some links to his paper and other references on these subjects.

Abstract from A. Mulder's talk:

"Sound Sculpting" - Virtual Object Interactive Sound Editing (VOISE)

Sound sculpting is sound editing. In typical sound editing environments the sound editor changes sound parameters using a keyboard and a mouse as input devices. To facilitate the coordination of the hands while designing a new sound, in sound sculpting the input device is presented to the sound designer as a virtual object (i.e. there is no touch or force feedback, but it is visible on a computer display) for each sound being edited. This virtual object is computed from the data of four sensor devices - two instrumented gloves that register the hand articulations and two sensors that measure the hand position and orientation. The sound designer can modify attributes of the virtual object like shape, orientation etc. These attributes are mapped to sound parameters so that the sound designer in fact changes the sound when changing attributes of the virtual object.

The aim of this approach is to reduce the cognitive load in simultaneous multidimensional control tasks such as sound design. During the first phase of this project we have developed objects for the computation of human movement primitives within Max/FTS, a real-time visual programming language for the SGI with built-in sound synthesis and algorithmic music objects. We have programmed Max/FTS objects for interfacing serial peripheral devices such as sensors, objects for geometric computation and objects specifically for the computation of object attributes, such as shape.