Sonification

Equipe design sonore -> Activités -> Dossier sonification - 2002

Sound design team -> Activities -> Computer sonification study - 2002

I. Introduction
II. Software examples
III. Sonification categories
IV. Conclusion 1 : category summary
V. Conclusion 2 : making sonification more adequate and useful
VI. A systematic approach : MPC service study
VII. Category summary - modified in light of MPC study
VIII. Sonification scheme example - proposed in light of MPC study

Appendices : links - tools

I. Introduction

What is described and studied here can be called "computer audio signaletics".
It embraces three fields :
- software audio behaviour - software audio reactions to user actions - software audio adorning -

We'll start with numerous examples taken from common existing softwares including OS and games (II)
Then we'll compare samples belonging to common action categories (III)
Using simple audio classification category, we'll be able to define, for each category, an average morphological portrait (IV)
In regards to all the examples that have been studied, we'll propose simple rules concerning the reasonable use of sonification (V)
An experiment to objectivize sonification's impact on users will be described, and its results analysed (VI)
Conclusions from this experiment will then be put in balance with previous conclusions, thus modifying the morphological portraits obtained in IV - (VII)

II. Software examples

The first thing to do is to have a close look at sonificated programs.
In this section, different programs are studied - instant messengers, mail clients, operating systems, and games.
A few sonificated web pages are also included.

Each program is described using the following method :
* short program description
* which actions are sonificated
* the audio files : location on the computer, format, how to modify them (in case of archive files)
* official audio samples (designed by/for the software society)
* some unofficial audio samples if possible (designed by anonymous users)

- open page - back to top -

III. Sonification categories

Similar sonificated actions / alerts / functionalities are put together in order to create categories.

A given category can include samples and/or functionalities from very different programs.
For instance, samples that are designed to welcome users can be found either in operating systems or in instant messengers - they will both belong to the "welcome" category.

Some categories : "welcome", "goodbye", "open window", "close window" ; "alarms", "alerts", "negative feedbacks", "positive feedbacks".
Also broader categories : "menu sonification", "user interface sonification"...

Categorization problems that arise when dealing with operating system sonification are explained and studied.

Samples that belong to a given category are compared in terms of morphology : melodic & dynamic profile, length, complexity, source...

- open page - back to top -

IV. Conclusions 1 : category summary

Typical distinctive aspects of samples belonging to the same category are given : what is the average length ? the most common melodic profile ? etc.

Thus, when possible, it is tried to define a "typical sample" associated to each category.

- open page - back to top -

V. Conclusions 2 : making sonification more adequate and useful

When sonification is useful, or isn't at all.
Cases that are particularly interesting, significant or paradoxal (ex log off Win robotz)
Cases that don't work at all, and why.

- open page - back to top -

VI. A systematic approach : MPC service study

In the field of audio signaletics, most sound designers have their own recipes to make samples that convey a certain meaning.
It could be interesting to check whether this meaning is indeed understood by the actual user, and if not, what the user exactly understand from the samples he hears.

This chapter proposes and checks a systematic method that can help us getting more precise views about this issue.

The Musical Perception & Cognition service at Ircam has proposed the following protocol : 48 samples that were supposed to convey certain meanings were made by the Sound Design service, and then proposed to a pool of 18 people. These persons has been asked specific questions about these samples, and the results were carefully studied and sorted by the MPC service.

Questions asked were for example : "put together the samples that convey the same meaning" and "describe this meaning", "chose amongst the samples the one that corresponds best to a given meaning" etc.

- open page - back to top -

VII. Category summary - modified in light of MPC study

One of the goals of the previous chapter's study was to validate the morphological portraits described in chapter IV, portraits that we obtained from chapter II & III.

In light of the conclusions that we obtained at the end of this study, we can modify and add new specifities to samples associated to a certain meaning, thus getting to more precise sonfication category definitions.

- open page - back to top -

VIII Sonification scheme example - proposed in light of MPC study

Chapter VII proposed a classification based on the conclusions we were able to draw from all the previous chapters.

It is now interesting to propose several sonification schemes using samples that comply with the specifications we can find in this classification.
These samples have then been proposed for the RadioThem project, which was described in chapter VI.