The OMax Project Page

Check the new OMax Page
Check the OMax performances videos on DailyMotion

OMax is a software environment which  learns in real-time typical features of a musician's style and  plays along with him interactively, giving the flavor of a machine co-improvisation.
OMax uses OpenMusic and Max. It is based on researches on stylistic modeling carried out by Gerard Assayag and Shlomo Dubnov and on researches on improvisation with the computer by G. Assayag, M. Chemillier and G. Bloch (Aka the OMax Brothers) in the Ircam Music Representations group.

OMax reinjects in several different ways the musician's material that has gone through a machine-learning stage, allowing a semantics-level representation of the session and a smart recombination and transformation of this material in real-time. We use to employ the notion of Stylistic Reinjection.

OMax uses original variants of the Factor Oracle algorithm (Crochemore & al) specially adapted for music processing. Arshia Cont has recently joined the family with researches on Music Anticipation,  Reinforcement Learning and Geometry of Information that pave the way to future improvements. Emilie Rossez helped on the video tools for OMax.

OMax handles Midi, audio and video data.

OMax in a Nutshell

Here are a few audio / video examples of selected performances using OMax that will give you a flavor of it.  The rest of this web site provides lots of other examples, software and documentation resources.

Cecile Daroux & Evdokija Danajloska  (Flute + voice, Avril 2009 / Ircam)
Cecile Daroux & Mederic Collignon (Flute, voice, cornet, avril 2009 / Ircam)
Vincent Courtois (Cello, Avril 2009 Ircam)
A piano duo (Mike garson and Omax)  using OMaxMidi
A piano duo (Hélène Schwarz and Omax) using OMaxAudio
A piano (Bernard Lubat) and virtual orchestra (OMax) improvisation at Uzeste Spring Festival
A Clarinette duo (J.B. Godet) and OMax using audio 
An OMax solo bop performance (Omax trained by B. Lubat and performing by itself on a virtual jazz orchestra)
Reconstructing Jaco,  a Jaco Pastorius pastiche learned directly from the CD.
A collective impro (The bernard Lubat's Lubathyscaphe-K band featuring OMax at the Uzeste Summer Festival)


Contents

NEW : The Cecile Daroux / Mederic Collignon / Vincent Courtois sessions (Avril 2009 / Ircam) -- The EIC sessions (Ircam 2009) --  Lubat/Omax at 2008 Uzeste Spring Festival in May -- Reorchestrations of Lubat/OMax Impros --  Garson/OMax impros 


 

Audio / video samples


The Bernard Lubat Sessions
The Hélène Schwartz sessions
The Philippe Leclerc sessions
Agora 2007, Jean-Brice Godet (clarinet)
Revisiting Jaco
The Mike Garson sessions
The Lubathyscaphe-K
The Dennis Thurmond sessions

Resources

Download OMax
OMax Documentation
OMax related publications
A Multimedia presentation of OMax/Lubat research

Related projects

MIMI home page
MIMI's groove


Jean-Brice Godet, Ircam Agora Festival, June 2007
Jean-Brice Godet graduated from the ATIAM MSc program. He is now pursuing a musical carreer, as a soloist clarinet player as well as a performer with Fröhn
This performance took place during the Agora festival, at a seminar on sound technologies organised for the music teachers of Paris city.

Credits : Daniel Raguin, shooting and editing.


Click on picture


Helene Schwartz Sessions, IRCAM, Studio 5, May 2 2007
Presented here is a sizzling improvisation by Hélène Schwartz recorded at IRCAM. Georges Bloch plays OMax. We use the "piano bar" version of OMax. A Moog piano bar is used to detect key motions and send Midi to OMax. The Steinway piano is also recorded using a couple of microphone and OMax uses this recombined audio material to play its improvisations instead of playing through a Midi expander, just as in the plain audio version.

Click on picture

The OMax Garson sessions took place in november 2006 in Mike Garson's recording studio in North Los Angeles. Mike Garson is an astounding musician who studied with Bill Evans and Herbie Hancock. He has played with Stanley Clarke, Elvin Jones, Lee Konitz and Freddie Hubbard, and he is also the regular pianist of David Bowie since the early seventies. Mike had heard about OMax through Dennis Thurmond and contacted us during a tour in Paris. After a first try he insisted that we experiment in good recording conditions in his studio equipped with 2 grand yamah disklavier.
During these sessions, Mike gave a great feed back that led to several interesting improvements of OMax. Especially the "follow" mode that can be heard in the following examples, where the memory zone in which OMax improvizes actually follows the musician with a variable memory depth. Specially for a musician that is able to react at the milliseconds to whatever variation the software produces, this option is both very challenging and very reinsuring since the musician has a better and a faster control over the directions taken by the software. Mike also suggested "algebraic" transformation such as retrograde, inversions, etc. that can't be heard there, but that have been implemented since and will be available in the soon to come release of OMax.

In these 2 examples, Mike improvizes with OMax, while just discovering the new "follow" mode. OMax controls the second disklavier and is panned on a different channel.

Impro 1 (mp3) Impro 1 (aif)
Impro 2 (mp3) Impro 2 (aif)

Another  brillant Mike/Omax impro sent by Mike :  listen

Reorchestration of some of Mike+OMax impro using Applied Acoustics Systems'  Tassman)


Lubathyscaphe-K at the 28th Uzeste Festival (2005), featuring Submersible Jazzcognitif de la Compagnie Lubat de Gasconha, Bernard Lubat , Fabrice Vieira, Fawzi Berger , Nathalie Boitaud, Christophe Rocher , OMax (Marc Chemiller, Gerard Assayag) , Yves Chaudouet (visuals)

Lubathyscaphe-K is an improvisation group that performs in the dark with some visuals. In this excerpt OMax is elaborating on musial patterns previsously captured, until it is joined by the group in a trance-like groove.

Lubathyscaphe-K selection by Y. Chaudouet : Lubsthyscaphe-K exerpt
Click on pictures

The Deliberation Orchestra at the 29th Uzeste Festival (2006), featuring Francois Corneloup (saxophones), Fabrice Vieira (guitar, voice), Yves Carbonne (bass guitar ), Fawzi Berger (percussions), Stokley Williams (drums), Paco Charlery (percussions), Tony Hymas (piano), Jef Lee Johnson (guitar), Christophe Monniot (saxophones), Denis Charolles (drums), Bernard Lubat (voice, kbd).

OMax (G. Assayag & Marc Chemillier) was invited as guest. It can be heard, for instance, at the beginning (Cello sample, processed reinjection of Bernard Lubat voicings). A giant storm exploded while we were setting up. The energetic character of the set and its motion toward a fast climax is explained because everybody was worried that the fabric roof was threatening to collapse under the weight of the rain water. We were prepared to run for our lives as soon as the last chord would resonate.

Click on picture

Real time audio/video experiments

In this audio example, OMax plays with saxophonist Philippe Leclerc. On one channel (the first to play in this recording) is the live source. On the other channel is the OMax performance. OMax plays as soon as it has learned a few notes from the live musician, so it begins with a trill. Soon enough, the model gets enriched and converges towards a duo.

In this video example, the same kind of interaction is achieved, but in addition, the musician (Philippe Leclerc) is filmed. The virtual improvisation by OMax is not only a sound improvisation, but one may actually see the virtual performer playing -- things he did not really play in the real world ! By a feed back process, the human performer is solicited to improvize with up to three clones coming from the recent musical pas -- an imaginary past. This sample is an excerpt from Georges Bloch dvd "Memories of Monk", with a collaboration by Emilie Rossez for the video tools.

Click on picture
In this video example with pianist Hélène Schwartz, a special version of OMax is used. A moog Piano Bar is used to get the Midi information from the ordinary piano. But it is the real sound of the piano that is captured and processed just as in the saxophone example. This way, we skip the pitch tracking that wouldn't work with a piano. Unfortunately a bug, solved since then, generates a distortion in the rendering of OMax impros. The video on the right is a testimony of Georges Bloch's true despair about this. The sound quality is not good.

Click on pictures


Revisiting Jaco Another audio example, where a famous impro by Jaco Pastorius (Donna Lee) is "listened to" and reimprovized in real time by OMax. Here we give only the OMax Impro. At the beginning of the recording, the recombination rate is very high, so the impro is very different from the original. As time passes, this rate is lowered, so one may notice the theme coming back and trying to rule the game. At the very end the theme manages to be exposed litterally (as a repetition from the original).


A concert with OMax

Recordings of a concert at UCS in Los Angeles on april 4, 2006, featuring Dennis Thurmond, pianist, and G. Assayag playing OMax.

In the following examples, all that is not acoustic piano is played by OMax. Except for the first piece, the model is empty at the start of each piece and learns all its material from what the pianist plays. Sometimes, a model is stored for later use, and waken up at some point in order to recall a past situation. The concert is totally improvised by both the performers.

1- Contemporary groove listen (mp3)
2- Tone row listen (mp3)
3- Messiaen at midnight listen (mp3)
4- Ballade at midnight listen (mp3)
5- Walking the bass listen (mp3)

Videos of rehearsal : click on pictures


The Bernard lubat sessions

Bernard Lubat is one of the greatest french improvizer. Poly-instrumentalist, he  played with many  top Jazz musicians and was involved in contemporary music experimentation as well.  In 1977 Lubat  founded the more original french music festival, the Uzeste Music Festival (or Hestejada de las arts in Occitan language). Bernard Lubat collaborates with the OMax Brothers since 2004.

OMax plays Bop from Lubat's 2004 improvisation (2008 reorchestration using Native Instrument's Kontakt)
OMax plays Bop from Lubat's 2004 improvisation (2008 reorchestration using Applied Acoustics Systems' Lounge Lizard)

In these two improvisations, OMax plays alone by reimprovizing the bop session learned from Bernard Lubat in the 2004 work sessions (see below). It is in a way a second order impro since the walking bass was already an Omax impro and Lubat was improvizing on top of it. Here Omax improvizes freely on the whole material and the Midi data is orchestrated using different virtual samplers/synths.

New Uzeste Spring Festival,  Concert OMax / Lubat at L'Estaminet, Uzeste, November 10, 2008 
Bernard Lubat, Piano. Gérard Assayag,  OMax.

Concert at Strasbourg Modern Art Museum, Jan 2007, Bernard Lubat & Gerard Assayag. Reimprovisation / Reorchestration of the Lubat OMax Model Captured during the concert.

In this concert the OMax Video module was used for the first time with bernard Lubat. Unfortunately the film and recording has been lost. Fortunately the OMax model file  captured from Lubat was saved, so we have generated an OMax solo impro from it.

Uzeste Spring Festival, OMax Demo at L'Estaminet, Uzeste, March 16, 2007. 

The demo starts on a single note, then develops several aspects of OMax. Bernard Lubat plays the piano on the left channel. OMax plays the Rhodes Mark1 on the right. Features of special interests are the parts where a counterpoint like interaction settles down between Lubat and OMax, and a large section in the middle where OMax, playing alone, variates continuously a groove it has captured from Lubat.

Listen to demo :

Complete (mp3) Complete (aif)
Final section (mp3) Final section (aif)

Work sessions at l'Estaminet, Uzeste, 2004

In this setup, Omax learns and play in real-time. Examples recorded with Bernard Lubat, french jazz pianist at Uzeste.

Lubat discovers OMax I , II & III

Click on pictures

Walking bass

OMax plays a walking bass in the style of Bernard Lubat who improvises on top of it. The learning has occured just before by having OMax listen to Lubat playing a few bars.


Clik on picture

Percussions

OMax plays percussion rhythms in the style of Bernard Lubat who improvises on top of it. The learning has occured just before by having OMax listen to Lubat playing a few bars of percussion on a synthe. Then Omax goes polyphonic by superposing several layers.


Click on pictures


Other OMax off-line examples (experiments in style modeling)

In this setup, Omax works off-line with midifile. The martino example uses the oracle algorithm. The 2 others use the IP/Lz algorithm.

Machine improvization generated after learning "Donna Lee" by Charlie Parker

Listen to Impro (mp3)

Comment :

From a midifile containing an arrangement of this standard (theme exposition plus chorus). Took only the sax and bass channels. The strange bass rhythm behaviour is due to excessive quantization , we kept it because the somewhat free style that results is an interesting remainder of some jazz tendancies in the sixties.

The machine impro begins with a recombinant variant of the theme, then dives into a bop style chorus.

Machine improvization generated after learning J.S. bach Art of Fugue Ricercar

Listen to Impro (mp3)

Comment :

Bach's ricercar is a six voice fugue. The information is extremely constrained, so the analysis/generation algorithm has very few choices for continuations. It tends to reproduce the original. But if you listen carefully, you'll hear that there are discrete bifurcations where it recombines differently from the original.

A study in the style of Jazz guitarist Pat Martino. Here's an idea of the original style (Blue Bossa) :

Listen to Pat Martino (mp3)

The learning process was based on a Midifile containing a transcription of Martino chorusing on Blue Bossa. After generating a few machine choruses, and choosing carefully a one that would fit, we mixed it back into Martino's audio recording, in a place where only the rhythmic section was playing (plus some piano). The machine impro is played with an (ugly) synthetic Midi Sax sound.

Listen to Mix (mp3)

Comment :

That experience was done in order to evaluate if the techniques used could make sense in a performance situation, with a musician playing with his clone. The result is encourageing, but in a real-time experiment, we would have to extract the beat and the harmony in order to control what's happening. In this case, we just inserted the machine impro by hand, tuning the tempo so it would fit with the audio.


A multimedia presentation of researches carried by  Ircam and Cie Lubat 

This research was funded by a grant from the French Ministry of Research in 2004-2005 :  "Creation of Scientific Mediation Products on the Internet"

Conception  : 2/3 of The OMax Brothers  (Gérard Assayag, Marc Chemillier),  IRCAM
Music  : Bernard Lubat, Cie Lubat de Gasconha
Realisation  : Elise Racinais, IKOTAME
Tests / Maquette  : Pascal Joube, Lune Bleue


click on picture




OMax related publications

Assayag, G. , Bloch, G. « Navigating the Oracle: a Heuristic Approach », Proc. ICMC'07, The In. Comp. Music Association, Copenhagen 2007.

Cont, S. Dubnov, G. Assayag « Anticipatory Model of Musical Style Imitation using Collaborative and Competitive Reinforcement Learning », Anticipatory Behavior in Adaptive Learning Systems, (Martin Butz and Olivier Sigaud and Gianluca Baldassarre, Berlin), 2007

E. Amiot, T. Noll, M. Andreatta, C. Agon, « Fourier Oracles for Computer-Aided Improvisation », ICMC 2006, New Orleans, 2006

G. Assayag, G. Bloch, M. Chemillier « OMax-Ofon », Sound and Music Computing (SMC) 2006, Marseille, 2006

G. Assayag, G. Bloch, M. Chemillier « Improvisation et réinjection stylistiques », Le feed-back dans la création musicale contemporaine - Rencontres musicales pluri-disciplinaires, Lyon, 2006

G. Assayag, G. Bloch, M. Chemillier, A. Cont, S. Dubnov « Omax Brothers : a Dynamic Topology of Agents for Improvization Learning », Workshop on Audio and Music Computing for Multimedia, ACM Multimedia 2006, Santa Barbara, 2006

Cont, S. Dubnov, G. Assayag « A framework for Anticipatory Machine Improvisation and Style Imitation », Anticipatory Behavior in Adaptive Learning Systems (ABiALS), Rome, 2006

Rueda, G. Assayag, S. Dubnov « A Concurrent Constraints Factor Oracle Model for Music Improvisation », XXXII Conferencia Latinoamericana de Informática CLEI 2006, Santiago, 2006

G. Assayag, S. Dubnov « Improvisation Planning and Jam Session Design using concepts of Sequence Variation and Flow Experience », Sound and Music Computing 2005, Salerno, 2005

Mondher, A. Gérard, M. Stephen, L. Olivier, C. Jean-Marc, R. Francis « De la théorie musicale à l’art de l’improvisation : Analyse des performances et modélisation musicale », ed. Mondher AYARI (DELATOUR-France, Paris), 2005

G. Assayag, S. Dubnov « Using Factor Oracles for machine Improvisation », Soft Computing, vol. 8, n° 9, Septembre, 2004

S. Dubnov, G. Assayag, O. Lartillot, G. Bejerano « Using Machine-Learning Methods for Musical Style Modeling », IEEE Computer, vol. 10, n° 38, Octobre, 2003

S. Dubnov, G. Assayag « Universal Prediction Applied to Stylistic Music Generation », Mathematics and Music. A Diderot Mathematical Forum, ed. Assayag, G., Feichtinger, H.G., Rodrigues, J.F. (Springer, Berlin), 2002

G. Assayag, G. Bejerano, S. Dubnov, O. Lartillot « Automatic modeling of musical style », 8èmes Journées d'Informatique Musicale, Bourges, 2001

O. Lartillot, S. Dubnov, G. Assayag, G. Bejerano « Automatic Modeling of Musical Style », International Computer Music Conference, La Havane, 2001

G. Assayag, S. Dubnov, O. Delerue « Guessing the Composer's Mind : Applying Universal Prediction to Musical Style », ICMC: International Computer Music Conference, Beijing, 1999

S. Dubnov G. Assayag « Universal Classification Applied to Musical Sequences », ICMC: International Computer Music Conference, Ann Arbor Michigan, 1998

Student works

B. Levy « Visualising OMax », Paris 6 [DEA ATIAM], 2009

J. Godet « Grammaires de substitution harmonique dans un improvisateur automatique », Paris 6 [DEA ATIAM], 2004

Laurier « Attributs multiples dans un improvisateur automatique », UPMC Paris 6 [DEA ATIAM], 2004

Seleborg « Interaction temps-réel/temps différé », DEA ATIAM [Mémoire de stage], 2004

N. Durand « Apprentissage du style musical et interaction sur deux échelles temporelles », Paris 6 [DEA ATIAM], 2003

Poirson « Simulations d'improvisations à l'aide d'un automate de facteurs et validation experimentale », UPMC [DEA ATIAM], 2002

O. Lartillot « Modélisation du style musical par apprentissage statique : Une application de la théorie de l'information à la musique », Paris 6/Ircam [DEA Atiam], 2000