is a software environment which learns in real-time typical features
of a musician's style and plays along with him interactively,
giving the flavor of a machine co-improvisation. |
OMax uses OpenMusic and Max. It is based on researches on stylistic modeling carried out by Gerard Assayag and Shlomo Dubnov and on researches on improvisation with the computer by G. Assayag, M. Chemillier and G. Bloch (Aka the OMax Brothers) in the Ircam Music Representations group.
OMax reinjects in several different ways the musician's material that has gone through a machine-learning stage, allowing a semantics-level representation of the session and a smart recombination and transformation of this material in real-time. We use to employ the notion of Stylistic Reinjection.
OMax uses original variants of the Factor Oracle algorithm (Crochemore & al) specially adapted for music processing. Arshia Cont has recently joined the family with researches on Music Anticipation, Reinforcement Learning and Geometry of Information that pave the way to future improvements. Emilie Rossez helped on the video tools for OMax.
OMax handles Midi, audio and video data.
Here are a few audio / video examples of selected performances using OMax that will give you a flavor of it. The rest of this web site provides lots of other examples, software and documentation resources.
Cecile Daroux & Evdokija Danajloska (Flute + voice, Avril 2009 / Ircam)
Cecile Daroux & Mederic Collignon (Flute, voice, cornet, avril 2009 / Ircam)
Vincent Courtois (Cello, Avril 2009 Ircam)
A piano duo (Mike garson and Omax) using OMaxMidi
A piano duo (Hélène Schwarz and Omax) using OMaxAudio
A piano (Bernard Lubat) and virtual orchestra (OMax) improvisation at Uzeste Spring Festival
A Clarinette duo (J.B. Godet) and OMax using audio
An OMax solo bop performance (Omax trained by B. Lubat and performing by itself on a virtual jazz orchestra)
Reconstructing Jaco, a Jaco Pastorius pastiche learned directly from the CD.
A collective impro (The bernard Lubat's Lubathyscaphe-K band featuring OMax at the Uzeste Summer Festival)
Audio / video samples
The Bernard Lubat Sessions
The Hélène Schwartz sessions
The Philippe Leclerc sessions
Agora 2007, Jean-Brice Godet (clarinet)
The Mike Garson sessions
The Dennis Thurmond sessions
OMax related publications
A Multimedia presentation of OMax/Lubat research
MIMI home page
Jean-Brice Godet graduated from the ATIAM MSc program. He is now pursuing a musical carreer, as a soloist clarinet player as well as a performer with Fröhn
This performance took place during the Agora festival, at a seminar on sound technologies organised for the music teachers of Paris city.
Credits : Daniel Raguin, shooting and editing.
Presented here is a sizzling improvisation by Hélène Schwartz recorded at IRCAM. Georges Bloch plays OMax. We use the "piano bar" version of OMax. A Moog piano bar is used to detect key motions and send Midi to OMax. The Steinway piano is also recorded using a couple of microphone and OMax uses this recombined audio material to play its improvisations instead of playing through a Midi expander, just as in the plain audio version.
OMax Garson sessions took
place in november 2006 in Mike Garson's recording studio in North Los
Angeles. Mike Garson is an astounding musician who studied with Bill
Evans and Herbie Hancock. He has played with Stanley Clarke, Elvin
Jones, Lee Konitz and Freddie Hubbard, and he is also the regular
pianist of David Bowie since the early seventies. Mike had heard about
OMax through Dennis Thurmond and contacted us during a tour in Paris.
After a first try he insisted that we experiment in good recording
conditions in his studio equipped with 2 grand yamah disklavier.
During these sessions, Mike gave a great feed back that led to several interesting improvements of OMax. Especially the "follow" mode that can be heard in the following examples, where the memory zone in which OMax improvizes actually follows the musician with a variable memory depth. Specially for a musician that is able to react at the milliseconds to whatever variation the software produces, this option is both very challenging and very reinsuring since the musician has a better and a faster control over the directions taken by the software. Mike also suggested "algebraic" transformation such as retrograde, inversions, etc. that can't be heard there, but that have been implemented since and will be available in the soon to come release of OMax.
In these 2 examples, Mike improvizes with OMax, while just discovering the new "follow" mode. OMax controls the second disklavier and is panned on a different channel.
|Impro 1 (mp3)||Impro 1 (aif)|
|Impro 2 (mp3)||Impro 2 (aif)|
Another brillant Mike/Omax impro sent by Mike : listen
Reorchestration of some of Mike+OMax impro using Applied Acoustics Systems' Tassman)
Lubathyscaphe-K at the 28th Uzeste Festival (2005), featuring Submersible Jazzcognitif de la Compagnie Lubat de Gasconha, Bernard Lubat , Fabrice Vieira, Fawzi Berger , Nathalie Boitaud, Christophe Rocher , OMax (Marc Chemiller, Gerard Assayag) , Yves Chaudouet (visuals)
|Lubathyscaphe-K is an improvisation group that performs in the dark with some visuals. In this excerpt OMax is elaborating on musial patterns previsously captured, until it is joined by the group in a trance-like groove.|
The Deliberation Orchestra
at the 29th Uzeste Festival (2006), featuring Francois Corneloup
(saxophones), Fabrice Vieira (guitar, voice), Yves Carbonne (bass
guitar ), Fawzi Berger (percussions), Stokley Williams (drums), Paco
Charlery (percussions), Tony Hymas (piano), Jef Lee Johnson (guitar),
Christophe Monniot (saxophones), Denis Charolles (drums), Bernard Lubat
|OMax (G. Assayag & Marc Chemillier) was invited as guest. It can be heard, for instance, at the beginning (Cello sample, processed reinjection of Bernard Lubat voicings). A giant storm exploded while we were setting up. The energetic character of the set and its motion toward a fast climax is explained because everybody was worried that the fabric roof was threatening to collapse under the weight of the rain water. We were prepared to run for our lives as soon as the last chord would resonate.|
In this audio example, OMax plays with saxophonist Philippe Leclerc. On one channel (the first to play in this recording) is the live source. On the other channel is the OMax performance. OMax plays as soon as it has learned a few notes from the live musician, so it begins with a trill. Soon enough, the model gets enriched and converges towards a duo.
|In this video example, the same kind of interaction is achieved, but in addition, the musician (Philippe Leclerc) is filmed. The virtual improvisation by OMax is not only a sound improvisation, but one may actually see the virtual performer playing -- things he did not really play in the real world ! By a feed back process, the human performer is solicited to improvize with up to three clones coming from the recent musical pas -- an imaginary past. This sample is an excerpt from Georges Bloch dvd "Memories of Monk", with a collaboration by Emilie Rossez for the video tools.|
|In this video example with pianist Hélène Schwartz, a special version of OMax is used. A moog Piano Bar is used to get the Midi information from the ordinary piano. But it is the real sound of the piano that is captured and processed just as in the saxophone example. This way, we skip the pitch tracking that wouldn't work with a piano. Unfortunately a bug, solved since then, generates a distortion in the rendering of OMax impros. The video on the right is a testimony of Georges Bloch's true despair about this. The sound quality is not good.|
|Revisiting Jaco Another audio example, where a famous impro by Jaco Pastorius (Donna Lee) is "listened to" and reimprovized in real time by OMax. Here we give only the OMax Impro. At the beginning of the recording, the recombination rate is very high, so the impro is very different from the original. As time passes, this rate is lowered, so one may notice the theme coming back and trying to rule the game. At the very end the theme manages to be exposed litterally (as a repetition from the original).|
Recordings of a concert at UCS in Los Angeles on april 4, 2006, featuring Dennis Thurmond, pianist, and G. Assayag playing OMax.
In the following examples, all that is not acoustic piano is played by OMax. Except for the first piece, the model is empty at the start of each piece and learns all its material from what the pianist plays. Sometimes, a model is stored for later use, and waken up at some point in order to recall a past situation. The concert is totally improvised by both the performers.
|1- Contemporary groove listen (mp3)
2- Tone row listen (mp3)
3- Messiaen at midnight listen (mp3)
4- Ballade at midnight listen (mp3)
5- Walking the bass listen (mp3)
Bernard Lubat is one of the greatest french improvizer. Poly-instrumentalist, he played with many top Jazz musicians and was involved in contemporary music experimentation as well. In 1977 Lubat founded the more original french music festival, the Uzeste Music Festival (or Hestejada de las arts in Occitan language). Bernard Lubat collaborates with the OMax Brothers since 2004.
OMax plays Bop from
Lubat's 2004 improvisation (2008 reorchestration
using Native Instrument's Kontakt)
OMax plays Bop from Lubat's 2004 improvisation (2008 reorchestration using Applied Acoustics Systems' Lounge Lizard)
In these two improvisations, OMax plays alone by reimprovizing the bop session learned from Bernard Lubat in the 2004 work sessions (see below). It is in a way a second order impro since the walking bass was already an Omax impro and Lubat was improvizing on top of it. Here Omax improvizes freely on the whole material and the Midi data is orchestrated using different virtual samplers/synths.
New Uzeste Spring Festival, Concert OMax / Lubat at
L'Estaminet, Uzeste, November 10, 2008
Bernard Lubat, Piano. Gérard Assayag, OMax.
Concert at Strasbourg Modern Art Museum, Jan 2007, Bernard Lubat & Gerard Assayag. Reimprovisation / Reorchestration of the Lubat OMax Model Captured during the concert.In this concert the OMax Video module was used for the first time with bernard Lubat. Unfortunately the film and recording has been lost. Fortunately the OMax model file captured from Lubat was saved, so we have generated an OMax solo impro from it.
Uzeste Spring Festival, OMax Demo at L'Estaminet, Uzeste, March 16, 2007.
The demo starts on a single note, then develops several aspects of OMax. Bernard Lubat plays the piano on the left channel. OMax plays the Rhodes Mark1 on the right. Features of special interests are the parts where a counterpoint like interaction settles down between Lubat and OMax, and a large section in the middle where OMax, playing alone, variates continuously a groove it has captured from Lubat.
Listen to demo :
Work sessions at l'Estaminet, Uzeste, 2004
In this setup, Omax learns and play in real-time. Examples recorded with Bernard Lubat, french jazz pianist at Uzeste.
|Lubat discovers OMax I , II & III|
OMax plays percussion rhythms in the style of Bernard Lubat who improvises on top of it. The learning has occured just before by having OMax listen to Lubat playing a few bars of percussion on a synthe. Then Omax goes polyphonic by superposing several layers.
In this setup, Omax works off-line with midifile. The martino example uses the oracle algorithm. The 2 others use the IP/Lz algorithm.
Machine improvization generated after learning "Donna Lee" by Charlie Parker
Listen to Impro (mp3)
From a midifile containing an arrangement of this standard (theme exposition plus chorus). Took only the sax and bass channels. The strange bass rhythm behaviour is due to excessive quantization , we kept it because the somewhat free style that results is an interesting remainder of some jazz tendancies in the sixties.
The machine impro begins with a recombinant variant of the theme, then dives into a bop style chorus.
Machine improvization generated after learning J.S. bach Art of Fugue Ricercar
Listen to Impro (mp3)
Bach's ricercar is a six voice fugue. The information is extremely constrained, so the analysis/generation algorithm has very few choices for continuations. It tends to reproduce the original. But if you listen carefully, you'll hear that there are discrete bifurcations where it recombines differently from the original.
A study in the style of Jazz guitarist Pat Martino. Here's an idea of the original style (Blue Bossa) :
Listen to Pat Martino (mp3)
The learning process was based on a Midifile containing a transcription of Martino chorusing on Blue Bossa. After generating a few machine choruses, and choosing carefully a one that would fit, we mixed it back into Martino's audio recording, in a place where only the rhythmic section was playing (plus some piano). The machine impro is played with an (ugly) synthetic Midi Sax sound.
Listen to Mix (mp3)
That experience was done in order to evaluate if the techniques used could make sense in a performance situation, with a musician playing with his clone. The result is encourageing, but in a real-time experiment, we would have to extract the beat and the harmony in order to control what's happening. In this case, we just inserted the machine impro by hand, tuning the tempo so it would fit with the audio.
multimedia presentation of researches carried by Ircam and
research was funded by a grant from the French
Ministry of Research in 2004-2005 : "Creation of Scientific
Products on the Internet"
Conception : 2/3
of The OMax Brothers (Gérard Assayag, Marc Chemillier),
click on picture
Assayag, G. , Bloch, G. « Navigating the Oracle: a Heuristic Approach », Proc. ICMC'07, The In. Comp. Music Association, Copenhagen 2007.
Cont, S. Dubnov, G. Assayag « Anticipatory Model of Musical Style Imitation using Collaborative and Competitive Reinforcement Learning », Anticipatory Behavior in Adaptive Learning Systems, (Martin Butz and Olivier Sigaud and Gianluca Baldassarre, Berlin), 2007
E. Amiot, T. Noll, M. Andreatta, C. Agon, « Fourier Oracles for Computer-Aided Improvisation », ICMC 2006, New Orleans, 2006
G. Assayag, G. Bloch, M. Chemillier « OMax-Ofon », Sound and Music Computing (SMC) 2006, Marseille, 2006
G. Assayag, G. Bloch, M. Chemillier « Improvisation et réinjection stylistiques », Le feed-back dans la création musicale contemporaine - Rencontres musicales pluri-disciplinaires, Lyon, 2006
G. Assayag, G. Bloch, M. Chemillier, A. Cont, S. Dubnov « Omax Brothers : a Dynamic Topology of Agents for Improvization Learning », Workshop on Audio and Music Computing for Multimedia, ACM Multimedia 2006, Santa Barbara, 2006
Cont, S. Dubnov, G. Assayag « A framework for Anticipatory Machine Improvisation and Style Imitation », Anticipatory Behavior in Adaptive Learning Systems (ABiALS), Rome, 2006
Rueda, G. Assayag, S. Dubnov « A Concurrent Constraints Factor Oracle Model for Music Improvisation », XXXII Conferencia Latinoamericana de Informática CLEI 2006, Santiago, 2006
G. Assayag, S. Dubnov « Improvisation Planning and Jam Session Design using concepts of Sequence Variation and Flow Experience », Sound and Music Computing 2005, Salerno, 2005
Mondher, A. Gérard, M. Stephen, L. Olivier, C. Jean-Marc, R. Francis « De la théorie musicale à l’art de l’improvisation : Analyse des performances et modélisation musicale », ed. Mondher AYARI (DELATOUR-France, Paris), 2005
G. Assayag, S. Dubnov « Using Factor Oracles for machine Improvisation », Soft Computing, vol. 8, n° 9, Septembre, 2004
S. Dubnov, G. Assayag, O. Lartillot, G. Bejerano « Using Machine-Learning Methods for Musical Style Modeling », IEEE Computer, vol. 10, n° 38, Octobre, 2003
S. Dubnov, G. Assayag « Universal Prediction Applied to Stylistic Music Generation », Mathematics and Music. A Diderot Mathematical Forum, ed. Assayag, G., Feichtinger, H.G., Rodrigues, J.F. (Springer, Berlin), 2002
G. Assayag, G. Bejerano, S. Dubnov, O. Lartillot « Automatic modeling of musical style », 8èmes Journées d'Informatique Musicale, Bourges, 2001
O. Lartillot, S. Dubnov, G. Assayag, G. Bejerano « Automatic Modeling of Musical Style », International Computer Music Conference, La Havane, 2001
G. Assayag, S. Dubnov, O. Delerue « Guessing the Composer's Mind : Applying Universal Prediction to Musical Style », ICMC: International Computer Music Conference, Beijing, 1999
S. Dubnov G. Assayag « Universal Classification Applied to Musical Sequences », ICMC: International Computer Music Conference, Ann Arbor Michigan, 1998
B. Levy « Visualising OMax », Paris 6 [DEA ATIAM], 2009
J. Godet « Grammaires de substitution harmonique dans un improvisateur automatique », Paris 6 [DEA ATIAM], 2004
Laurier « Attributs multiples dans un improvisateur automatique », UPMC Paris 6 [DEA ATIAM], 2004
Seleborg « Interaction temps-réel/temps différé », DEA ATIAM [Mémoire de stage], 2004
N. Durand « Apprentissage du style musical et interaction sur deux échelles temporelles », Paris 6 [DEA ATIAM], 2003
Poirson « Simulations d'improvisations à l'aide d'un automate de facteurs et validation experimentale », UPMC [DEA ATIAM], 2002
O. Lartillot « Modélisation du style musical par apprentissage statique : Une application de la théorie de l'information à la musique », Paris 6/Ircam [DEA Atiam], 2000