| OMax
is a software environment which learns in real-time typical features
of a musician's style and plays along with him interactively,
giving the flavor of a machine co-improvisation.
OMax uses OpenMusic
and Max. It is based on researches on stylistic modeling carried out
by Gerard Assayag
and Shlomo Dubnov and on researches on improvisation
with the computer by G. Assayag, M. Chemillier
and G. Bloch
(Aka the
OMax Brothers) in the Ircam Music Representations group.
OMax reinjects in several different ways the
musician's material that has gone through a machine-learning stage,
allowing a semantics-level representation of the session and a smart
recombination and transformation of this material in real-time. We use to employ the notion of Stylistic Reinjection.
OMax
uses original variants of the Factor Oracle algorithm (Crochemore
&
al) specially adapted for music processing. Arshia Cont has recently
joined the family with researches on Music Anticipation,
Reinforcement Learning and Geometry of Information that pave the way to future improvements. Emilie
Rossez helped on the video tools
for OMax.
OMax handles Midi, audio and video data. |
OMax in a Nutshell
Here
are a few audio / video examples of selected performances using OMax
that will give you a flavor of it. The rest of this web site
provides lots of other examples, software and documentation resources.
Cecile Daroux & Evdokija Danajloska (Flute + voice, Avril 2009 / Ircam)
Cecile Daroux & Mederic Collignon (Flute, voice, cornet, avril 2009 / Ircam)
Vincent Courtois (Cello, Avril 2009 Ircam)
A piano duo (Mike garson and Omax) using OMaxMidi
A piano duo (Hélène Schwarz and Omax) using OMaxAudio
A piano (Bernard Lubat) and virtual orchestra (OMax) improvisation at Uzeste Spring Festival
A Clarinette duo (J.B. Godet) and OMax using audio
An OMax solo bop performance (Omax trained by B. Lubat and performing by itself on a virtual jazz orchestra)
Reconstructing Jaco, a Jaco Pastorius pastiche learned directly from the CD.
A collective impro (The bernard Lubat's Lubathyscaphe-K band featuring OMax at the Uzeste Summer Festival)
Contents
Jean-Brice
Godet, Ircam Agora Festival, June 2007
|
|
Click on picture
|
Helene
Schwartz Sessions, IRCAM, Studio 5, May 2 2007
|
Presented here is a sizzling improvisation by Hélène
Schwartz
recorded at IRCAM. Georges Bloch plays OMax. We use the "piano bar"
version of OMax. A Moog piano bar is used to detect key motions and
send Midi to OMax. The Steinway piano is also recorded using a couple
of microphone and OMax uses this recombined audio material to play its
improvisations instead of playing through a Midi expander, just as in
the plain audio version.
|
Click on picture
|
The
OMax Garson sessions took
place in november 2006 in Mike Garson's recording studio in North Los
Angeles. Mike Garson is an astounding musician who studied with Bill
Evans and Herbie Hancock. He has played with Stanley Clarke, Elvin
Jones, Lee Konitz and Freddie Hubbard, and he is also the regular
pianist of David Bowie since the early seventies. Mike had heard about
OMax through Dennis Thurmond and contacted us during a tour in Paris.
After a first try he insisted that we experiment in good recording
conditions in his studio equipped with 2 grand yamah disklavier.
During these sessions, Mike gave a great feed back that led to several
interesting improvements of OMax. Especially the "follow" mode that can
be heard in the following examples, where the memory zone in which OMax
improvizes actually follows the musician with a variable memory depth.
Specially for a musician that is able to react at the milliseconds to
whatever variation the software produces, this option is both very
challenging and very reinsuring since the musician has a better and a
faster control over the directions taken by the software. Mike also
suggested "algebraic" transformation such as retrograde, inversions,
etc. that can't be heard there, but that have been implemented since
and will be available in the soon to come release of OMax.
In
these 2 examples, Mike improvizes with OMax, while just discovering the
new "follow" mode. OMax controls the second disklavier and is panned on
a different channel.
Lubathyscaphe-K
at
the 28th Uzeste Festival (2005), featuring Submersible Jazzcognitif de
la Compagnie Lubat de Gasconha, Bernard Lubat , Fabrice Vieira, Fawzi
Berger , Nathalie Boitaud, Christophe Rocher , OMax (Marc Chemiller,
Gerard Assayag) , Yves Chaudouet (visuals)
| Lubathyscaphe-K
is an improvisation group that performs in the dark with some visuals.
In this excerpt OMax is elaborating on musial patterns previsously
captured, until it is joined by the group in a trance-like groove. |
Lubathyscaphe-K selection
by Y. Chaudouet : Lubsthyscaphe-K exerpt
Click on pictures
|
The Deliberation Orchestra
at the 29th Uzeste Festival (2006), featuring Francois Corneloup
(saxophones), Fabrice Vieira (guitar, voice), Yves Carbonne (bass
guitar ), Fawzi Berger (percussions), Stokley Williams (drums), Paco
Charlery (percussions), Tony Hymas (piano), Jef Lee Johnson (guitar),
Christophe Monniot (saxophones), Denis Charolles (drums), Bernard Lubat
(voice, kbd).
| OMax
(G. Assayag & Marc Chemillier) was invited as guest. It can be
heard, for instance, at the beginning (Cello sample, processed
reinjection of Bernard Lubat voicings). A giant storm exploded while we
were setting up. The energetic character of the set and its motion
toward a fast climax is explained because everybody was worried that
the fabric roof was threatening to collapse under the weight of the
rain water. We were prepared to run for our lives as soon as the last
chord would resonate. |
Click on picture
|
Real
time audio/video experiments
In this audio example,
OMax plays with saxophonist Philippe Leclerc. On one channel (the first
to play in this recording) is the live source. On the other channel is
the OMax performance. OMax plays as soon as it has learned a few notes
from the live musician, so it begins with a trill. Soon enough, the
model gets enriched and converges towards a duo.
| In
this video example, the same kind of interaction is achieved, but in
addition, the musician (Philippe Leclerc) is filmed. The virtual
improvisation by OMax is not only a sound improvisation, but one may
actually see the virtual performer playing -- things he did not really
play in the real world ! By a feed back process, the human performer is
solicited to improvize with up to three clones coming from the recent
musical pas -- an imaginary past. This sample is an excerpt from
Georges Bloch dvd "Memories of Monk", with a collaboration by Emilie
Rossez for the video tools. |
Click on picture
|
| In
this video example with pianist Hélène Schwartz, a
special version of OMax is used. A moog Piano Bar is used to get the
Midi information from the ordinary piano. But it is the real sound of
the piano that is captured and processed just as in the saxophone
example. This way, we skip the pitch tracking that wouldn't work with a
piano. Unfortunately a bug, solved since then, generates a distortion
in the rendering of OMax impros. The video on the right is a testimony
of Georges Bloch's true despair about this. The sound quality is not
good. |
 
Click on pictures
|
 |
Revisiting Jaco Another
audio example,
where a famous impro by Jaco Pastorius (Donna Lee) is "listened to" and
reimprovized in real time by OMax. Here we give only the OMax Impro. At
the beginning of the recording, the recombination rate is very high, so
the impro is very different from the original. As time passes, this
rate is lowered, so one may notice the theme coming back and trying to
rule the game. At the very end the theme manages to be exposed
litterally (as a repetition from the original). |
A
concert with OMax
Recordings of
a concert at UCS in Los Angeles on april 4, 2006, featuring
Dennis Thurmond, pianist, and G. Assayag playing OMax.
In
the following examples, all that is not acoustic piano is played by
OMax. Except for the first piece, the model is empty at the start of
each piece and learns all its material from what the pianist plays.
Sometimes, a model is stored for later use, and waken up at some point
in order to recall a past situation. The concert is totally improvised
by both the performers.
The
Bernard lubat sessions
Bernard Lubat
is one of the greatest french improvizer. Poly-instrumentalist,
he played with many top Jazz musicians and was involved in
contemporary music experimentation as well. In 1977 Lubat
founded the more original french music festival, the Uzeste Music Festival (or Hestejada de las arts in Occitan language). Bernard Lubat collaborates with the OMax Brothers since 2004.
OMax plays Bop from
Lubat's 2004 improvisation (2008 reorchestration
using Native Instrument's Kontakt)
OMax plays Bop from
Lubat's 2004 improvisation (2008 reorchestration
using Applied Acoustics Systems' Lounge Lizard)
In
these two improvisations, OMax plays alone by reimprovizing the bop
session learned from Bernard Lubat in the 2004 work sessions (see below).
It is in a way a second order impro since the walking bass was already
an Omax impro and Lubat was improvizing on top of it. Here Omax
improvizes freely on the whole material and the Midi data is
orchestrated using different virtual samplers/synths.
New Uzeste Spring Festival, Concert OMax / Lubat at
L'Estaminet, Uzeste, November 10, 2008
Bernard Lubat, Piano. Gérard Assayag, OMax.
Concert at Strasbourg
Modern Art Museum, Jan 2007, Bernard Lubat & Gerard Assayag. Reimprovisation / Reorchestration of the
Lubat OMax Model Captured during the concert.
In
this concert the OMax Video module was used for the first time with
bernard Lubat. Unfortunately the film and recording has been lost.
Fortunately the OMax model file captured from Lubat was saved, so
we have generated an OMax solo impro from it.Uzeste Spring Festival, OMax Demo at
L'Estaminet, Uzeste, March 16, 2007.
The
demo starts on a single note, then develops several aspects of OMax.
Bernard Lubat plays the piano on the left channel. OMax plays the
Rhodes Mark1 on the right. Features of special interests are the parts
where a counterpoint like interaction settles down between Lubat and
OMax, and a large section in the middle where OMax, playing alone,
variates continuously a groove it has captured from Lubat.
Listen to demo :
Work sessions at
l'Estaminet, Uzeste, 2004
In this setup, Omax learns and
play in real-time. Examples recorded with Bernard Lubat, french jazz
pianist at Uzeste.
| Lubat
discovers OMax I , II & III |
|
Walking bass
OMax
plays a walking bass in the style of Bernard Lubat who improvises on
top of it. The learning has occured just before by having OMax listen
to Lubat playing a few bars.
|
Clik on picture
|
Percussions
OMax
plays percussion rhythms in the style of Bernard Lubat who improvises
on top of it. The learning has occured just before by having OMax
listen to Lubat playing a few bars of percussion on a synthe. Then Omax
goes polyphonic by superposing several layers.
|
|
Other
OMax off-line examples (experiments in style modeling)
In
this setup, Omax works off-line with midifile. The martino example uses
the oracle algorithm. The 2 others use the IP/Lz algorithm.
Machine improvization
generated after learning "Donna Lee" by Charlie Parker
Listen
to Impro (mp3)
Comment :
From
a midifile containing an arrangement of this standard (theme exposition
plus chorus). Took only the sax and bass channels. The strange bass
rhythm behaviour is due to excessive quantization , we kept it because
the somewhat free style that results is an interesting remainder of
some jazz tendancies in the sixties.
The machine impro begins with a
recombinant variant of the theme, then dives into a bop style chorus.
Machine improvization
generated after learning J.S. bach Art of Fugue Ricercar
Listen
to Impro (mp3)
Comment :
Bach's
ricercar is a six voice fugue. The information is extremely
constrained, so the analysis/generation algorithm has very few choices
for continuations. It tends to reproduce the original. But if you
listen carefully, you'll hear that there are discrete bifurcations
where it recombines differently from the original.
A study in the style of Jazz
guitarist Pat Martino. Here's an idea of the original style (Blue
Bossa) :
Listen
to Pat Martino (mp3)
The
learning process was based on a Midifile containing a transcription of
Martino chorusing on Blue Bossa. After generating a few machine
choruses, and choosing carefully a one that would fit, we mixed it back
into Martino's audio recording, in a place where only the rhythmic
section was playing (plus some piano). The machine impro is played with
an (ugly) synthetic Midi Sax sound.
Listen
to Mix (mp3)
Comment :
That
experience was done in order to evaluate if the techniques used could
make sense in a performance situation, with a musician playing with his
clone. The result is encourageing, but in a real-time experiment, we
would have to extract the beat and the harmony in order to control
what's happening. In this case, we just inserted the machine impro by
hand, tuning the tempo so it would fit with the audio.
A
multimedia presentation of researches carried by Ircam and
Cie Lubat
This
research was funded by a grant from the French
Ministry of Research in 2004-2005 : "Creation of Scientific
Mediation
Products on the Internet"
Conception : 2/3
of The OMax Brothers (Gérard Assayag, Marc Chemillier),
IRCAM
Music : Bernard Lubat, Cie Lubat de Gasconha
Realisation : Elise Racinais, IKOTAME
Tests / Maquette : Pascal Joube, Lune Bleue
|
click on picture
|
OMax related publications
Assayag, G. , Bloch, G. « Navigating the
Oracle: a Heuristic Approach », Proc. ICMC'07, The In. Comp. Music
Association, Copenhagen 2007.
Cont, S. Dubnov, G.
Assayag
« Anticipatory Model of Musical Style Imitation using
Collaborative and Competitive Reinforcement Learning », Anticipatory
Behavior in Adaptive Learning Systems, (Martin Butz and
Olivier Sigaud and Gianluca Baldassarre, Berlin), 2007
E. Amiot, T. Noll, M. Andreatta,
C. Agon, « Fourier
Oracles for Computer-Aided Improvisation »,
ICMC 2006, New Orleans, 2006
G. Assayag, G. Bloch, M.
Chemillier « OMax-Ofon »,
Sound and Music Computing (SMC) 2006, Marseille, 2006
G. Assayag, G. Bloch, M.
Chemillier « Improvisation
et réinjection stylistiques », Le
feed-back dans la création musicale contemporaine - Rencontres
musicales pluri-disciplinaires, Lyon, 2006
G. Assayag, G. Bloch, M.
Chemillier, A. Cont, S. Dubnov « Omax
Brothers : a Dynamic Topology of Agents for Improvization Learning »,
Workshop on Audio and Music Computing for Multimedia, ACM
Multimedia 2006, Santa Barbara, 2006
Cont, S. Dubnov, G.
Assayag « A
framework for Anticipatory Machine Improvisation and Style Imitation »,
Anticipatory Behavior in Adaptive Learning Systems (ABiALS),
Rome, 2006
Rueda, G. Assayag, S.
Dubnov « A
Concurrent Constraints Factor Oracle Model for Music Improvisation »,
XXXII Conferencia Latinoamericana de Informática CLEI 2006,
Santiago, 2006
G. Assayag, S. Dubnov
« Improvisation
Planning and Jam Session Design using concepts of Sequence Variation
and Flow Experience », Sound and
Music Computing 2005, Salerno, 2005
Mondher, A. Gérard, M.
Stephen, L. Olivier, C. Jean-Marc, R. Francis « De
la théorie musicale à l’art de
l’improvisation : Analyse des performances et
modélisation musicale », ed. Mondher AYARI
(DELATOUR-France, Paris), 2005
G. Assayag, S. Dubnov
« Using
Factor Oracles for machine Improvisation »,
Soft Computing, vol. 8, n° 9, Septembre, 2004
S. Dubnov, G. Assayag,
O. Lartillot, G. Bejerano « Using
Machine-Learning Methods for Musical Style Modeling »,
IEEE Computer, vol. 10, n° 38, Octobre, 2003
S. Dubnov, G. Assayag
« Universal
Prediction Applied to Stylistic Music Generation »,
Mathematics and Music. A Diderot Mathematical Forum,
ed. Assayag, G., Feichtinger, H.G., Rodrigues, J.F. (Springer, Berlin),
2002
G. Assayag, G. Bejerano,
S. Dubnov, O. Lartillot « Automatic
modeling of musical style », 8èmes
Journées d'Informatique Musicale, Bourges, 2001
O. Lartillot, S. Dubnov,
G. Assayag, G. Bejerano « Automatic
Modeling of Musical Style », International
Computer Music Conference, La Havane, 2001
G. Assayag, S. Dubnov,
O. Delerue « Guessing
the Composer's Mind : Applying Universal Prediction to Musical
Style », ICMC: International
Computer Music Conference, Beijing, 1999
S. Dubnov G. Assayag
« Universal
Classification Applied to Musical Sequences »,
ICMC: International Computer Music Conference, Ann
Arbor Michigan, 1998
Student works
B. Levy
« Visualising OMax », Paris 6 [DEA ATIAM], 2009
J. Godet
« Grammaires de substitution harmonique dans un improvisateur
automatique », Paris 6 [DEA ATIAM], 2004
Laurier
« Attributs
multiples dans un improvisateur automatique »,
UPMC Paris 6 [DEA ATIAM], 2004
Seleborg
« Interaction
temps-réel/temps différé », DEA ATIAM
[Mémoire de stage], 2004
N. Durand
« Apprentissage
du style musical et interaction sur deux échelles temporelles »,
Paris 6 [DEA ATIAM], 2003
Poirson
« Simulations d'improvisations à l'aide d'un automate
de facteurs et validation experimentale », UPMC [DEA ATIAM],
2002
O. Lartillot
« Modélisation
du style musical par apprentissage statique : Une application
de
la théorie de l'information à la musique »,
Paris 6/Ircam [DEA Atiam], 2000
