[demo video]
[sound examples]

 

[how it works]
[orchidée matlab prototype]
[orchidée standalone osc server]

 

Orchidée is a innovative tool for computer-aided musical orchestration purposes. The word Orchidée comes from the contraction of orch (for "orchestration") and idée (French word for "idea"). Alternatively, it is a beautiful and very delicate flower, with an extremely complex genetic code. And guess what? Orchidée uses a genetic-based evolutionary search algorithm.

The Orchidée prototype was originally developed in Matlab. It included a set of graphical user interfaces (GUIs) but had a limited and non-extendible musical instrument knowledge. Today, Orchidee is a compiled standalone application server that communicates with client interfaces through OSC messages. Thus, Orchidée allows an easy and flexible control from traditional computer music environments such as OpenMusic or Max/MSP. Another key feature is that the musical knowledge provided with Orchidée may be easily generalized and extended by the user. However, the GUIs have now moved to the client side and are no more delivered with the application.

The Orchidée prototype was designed by researchers Grégoire Carpentier and Damien Tardieu during their PhDs at IRCAM, with the help and supervision of composer Yan Maresz. Their work was supported by the French ANR Sample Orchestrator project. Orchidée is currently developed as a standalone OSC server by Gregoire Carpentier in the Music Representations Group at IRCAM.

Orchidée has already been used by many composers: Yan Maresz, Jonathan Harvey, Oscar Bianchi, Gérard Buquet, Mauro Lanza, Marco Suarez, Fernando Villanueva, Kenji Sakai, Raphael Cendo, Marc Victoria and many more...

 

Recent musical works using Orchidée

- Jonathan Harvey, Speakings, August 2008, Royal Albert Hall, London.
- Marco Suarez Cifuentes, Poetry for //dark-/ dolls, January 2009, IRCAM, Paris.
- Fernando Villanueva Carretero, Bukowski Madrigals, June 2009, IRCAM, Paris.
- Kenji Sakai, Astral/Chromoprojection, October 2009, IRCAM, Paris.
- Gérard Buquet, L'Astre Echevelée, November 2009, IRCAM, Paris.
- Marc Garcia Vitoria, The P Extensions, May 2010, Geneva.
- Christopher Trapani, Cognitive Consonance, Le 104, June 2010, Paris.
- Christopher Trapani, Westering, Carneghie Hall, New York, December 2010.
- Alec Hall, Striped Noise, New York, 2011.
- Nicolas Tzortzis, Ient, 2011.
- Javier Torres Maldonado, Un Possible Dia, 2011.

 

[how it works]
[how it sounds]
[how it looks]
[orchidée matlab prototype]
[orchidée standalone osc server]

 

How it works

Orchidée is a computer-aided orchestration system. It aims at helping composer to write instrumental mixtures that achieve a given idea of timbre, i.e. a given orchestral color. The target timbre is provided by the user as a simple soundfile, and Orchidée searches within a large instrument sample database combinations of sounds that perceptually match the target.

Compared to former automatic orchestration systems Orchidée offers a set of innovative and powerful features: A heterogeneous, multi-level knowledge of instrument capabilities (allowing, among other facilities, the definition of symbolic constraints), a multiobjective approach of timbre similarity, and an interactive search process allowing the discovery of implicit listening preferences.

Multi-level musical knowledge

Rather than working at the signal level (or, equivalently, at the spectrum level), a sound description approach is prefered. Each sound in the instrument database is linked to a set of audio features and symbolic attributes. On one hand, audio features describe perceptive characteristics of the sound, such as brightness, roughness, loudness, harmonic color, etc. They are automatically extracted from the signal raw representation. On the other hand, symbolic attributes are the traditional music notation parameters: Instrument, note, dynamics, playing style, mutes, etc. They are known by manual indexation (usually done during the recording sessions).

Audio features are used to compute perceptual dissimilarites in the sonic space, whereas symbolic attributes are involved in the specification of musical constraints.

Multiobjective approach of timre similarity

The orchestration problem may be seen as combinatorial optimization problem. Within the signal description framework, the goal is to find the set of musical variables that best "approach" the target set of audio features. In this problem, the variables are symbolic attributes and the objective functions to be minimized are the dissimilarities between the orchestration audio features and the target features. Of course, these distances are not linear.

Rather than aggregating the different distances into a single value to minimize we choosed to keep timbre dimensions separated and optimize jointly each criterium through a multiobjective process. This approach has many advantages:

- The output is not a single solution but a set of efficient solutions, also called the Pareto set. A solution is said efficient if it cannot be improved on one objective without decrasing the performances on the other objectives.

- The relative weight of each objective (or each timbre dimension) does not need to be known in advance. Automatic objective scaling and weighting is performed during the search through interaction with the composer.

Here is a nice illustration multiobjective timbre similarity in the bi-objective case. The objectives here are distances to the target noise and to the target spectral peaks. The smaller value of the objective, the closer the solution to the target on the given dimension. The target sound (at the origin of the axes) is a A4 tremolo flute sound. Solution (1) a very good on spectral peaks but totally miss the noisy part of the tremolo. Solution (3) is pure noise without any harmonic components, and solution (2) is a kind of compromise.

Target sound: Solution 1: Solution 2: Solution 3:

Listening preferences

Guessing the user's implicit preferences is possible through interaction. Assume that that after a first optimization step the system output the three above solutions and ask user to choose his/her "prefered" one. Choosing solution (1) would mean that the spectral peaks distance is more important than noise to the composer's ears. Choosing solution (3) implies that noisiness come first, and solution (2) that noise and harmonic components share the same weight.

As shown on the figure below, the choice of a prefered solution provides the search algorithm with a direction of interest, allowing the intensification of the exploration in a specific region of the search space.

Overall orchestration process

The global orhcestration process in Orchidee is shown on the figure below. On the left side the instrumental knowledge is represented by a sound sample database from which a description database was preliminarily extracted. The composer may define three different types of inputs: (i) The target timbre, (ii) the instrument set (orchestra) used in the piece, and (iii) a set of symbolic constraints used to explicitly discard regions of the search space. The search for optimal orchestrations is then handled by a genetic-based evolutionary algorithm, aiming at the discovery of regions that sound close to the target while fulfilling the requirements imposed by the symbolic constraints.

The search process is supervised by the composer through the best-solution-choice mechanism described upper. At each interaction point, the implicit listening preferences are automatically inferred and the search is redrawn toward a specific direction. This interactive search process iterates until a relevant orchestration is found.

When the target cannot be specified by a concrete pre-recorded sound, it can yet be "constructed" through a compositional process (e.g. an OpenMusic patch, [more here]). Assuming that there is no best langage than sound to express a timbre idea, the purpose is to use symbolic data as input of a feature-controlled synthesis module to write a conrete target sound. Once the user is satisfied with this intermediary synthetized target (it way sound very poor but who cares, it's just there to reflect the target timber essentials, no more!) the problem is no different than before and can be solved in the same way, leading to solutions that combine the target timber characteristics with richness of real instruments.

Search algorithm

The search algorithm relies on evolutionary metaheuristics. This means that the search process uses a population of solutions and a set of neighborhood operators to converge towards Pareto frontier. A solution is represented by a tuple whose elements corresponds to sounds of the database.

Staying away from technical details, the search algorithm has two main tasks to performs simultaneously:

- Audio feature opitimization: Find a good approximation (in terms of convergence and diversity) of the theoretic Pareto set.

- Symbolic constraints consistency: Make sure that solutions are consistent with the symbolic constraint network. The lowest constraint level is the feasability of the solution regarding the instrumental ressource of the orchestra.

The solution evalutation flowchart is shown on the following figure:

Audio evaluation first call a set of features estimators to compute the solution features from its individual components. Then, distance functions are invoked to output audio criteria (vector relative distances to target features). On the symbolic side, error functions are used to compute the constaint violation level of the solution. Audio optimization is driven by the evolutionary process, whereas constraint consistency is garantied by the joint use of a powerful repair procedure and an adapted solution ranking scheme.

 

[how it works]
[how it sounds]
[how it looks]
[orchidée matlab prototype]
[orchidée standalone osc server]

 

Matlab prototype

Orchidée was originally developed as a Matlab prototype, with audio modules in Max/MSP. It was intended to prove the validity of our approach and to quickly collect feedbacks from the composers. The Orchidée prototype has the following features:

- A fixed database of 7000 instrument sound samples, covering 11 instruments with various playing styles, including more than 2500 quarter-tone samples.

- 3 spectral timbre features: Main resolved partials (harmonic color), spectral centroid (brightness) and spectral spread (volume).

- An explicit and implicit constraint handling langage and solver.

- A target analysis and definition interface, including the possibility to define filters on the pitch attribute.

- A multiple views interface for the exploration of the orchestration solutions.

 

Below, five screenshots of the Orchidée prototype. From left to right and up to bottom: (1) target definition, (2) multiple views solution exploration interface, (3) solution transformation with constraints, (4) manual edition and playback, (5) score export.

(1) (2) (3) (4) (5)

 

The video below is a video sreen capture of a typical use-case scenario: Target definition, constraint specification, exploration and edition of the solution space, dynamic timbral motion with additional constraints, score export.

 

[how it works]
[how it sounds]
[how it looks]
[orchidée matlab prototype]
[orchidée standalone osc server]

 

The Orchidée OSC server

After the success of the first Matlab prototype, Orchidée is currently being developed as a standalone application. A client/server framework is put forward in order to ease the integration of automatic orchestration tasks into traditional computer music environments, e.g. OpenMusic or Max/MSP. Orchidée has now become a simple, transparent background application, with which client programs communicate through OSC messages.

Below is the class diagram of the current implementation of Orchidée. Like in the former prototype, the instrument knowledge is derived from the analysis of large sound sample databases. However, this database is now easily extendible by users. The metadata associated to the sound files are stored in XML files that should be seen as a textual version of the internal knowledge. Within this framework, users can easily share extend/share their knowledge by the exchange of simple XML files. Extraction methods to create XML metadata from soundfiles are also provided.

The Orchidée server has the following features:

- A database of 20000 sound samples covering 22 instruments, extendible by users.

- A microtonic interpolation scheme allowing a pitch resolution up to 1/16 tone.

- 6 timbre features: Main resolved partial (harmonic color), spectral centroid (brightness), spectral spread (volume), attack time (hardness/smoothness of attack), amplitude modulation (tremolo and roughness), and Mel spectrum envelope (overall perceputal spectral balance).

- Explicit constraint handling through filter design.

 

Below, a few screenshots of OpenMusic patches that call various functions of the Orchidée server. As one can see, orchestration tasks may now be fully integrated into large-scale compositional processes.

 

[how it works]
[how it sounds]
[how it looks]
[orchidée matlab prototype]
[orchidée standalone osc server]