|
|
Music Information Retrieval (MIR) has been for a long time associated to the extraction of information from already-created and already-recorded music in order to facilitate search, navigation and access over music collections. In this workshop, we study how MIR by becoming Music Information Research has extended its scope and is now used for the creation process itself. We invite key-actors to give their point-of-view on the present and future of MIR for creation, being at the composition, interaction, performance or research level, at the audio, symbolic and database level.
This workshop is supported by the MIReS project (7th Framework European Project) "Roadmap for Music Information ReSearch"
| 09.45am | Introduction | Geoffroy Peeters |
| 10.00am | Gérard Assayag | "Information retrieval and deployment in interactive improvisation systems" |
| 11.00am | Norbert Schnell | "Gestural Re-Embodiment of Digitized Sound and Music" |
| 11.30am | Diemo Schwarz | "Interactive Exploration of Sound Corpora for Music Performance and Composition" |
| 12.00am | Philippe Manoury | |
| 01.00pm | Break | |
| 02.00pm | Tristan Jehan (EchoNest) | "Playing with Music" |
| 03.00pm | François Pachet (SonyCSL) | "VirtualBand, a MIR-approach to interactive improvisation" |
| 04.00pm | Sergi Jorda (IUA/UPF) | "MIR beyond retrieval: Music Performance, Multimodality and Education" |
MIR and Creation Workshop. Introduction par Ircam-CGP
Abstract: Interactive Improvisation Systems involve at least three cooperating and concurrent expert agents: machine listening, machine learning, model based generation. Machine listening may occur during the initial learning stage (off-line or real-time in live situations) and during the generation stage as well in order to align the computer production with current live input. Machine learning can be based on any statistical model capturing significant signal or symbolic stream of features that can be exploited in the generation stage. In particular, the OMax interactive computational improvisation environment will be presented.
Bio: Gerard Assayag is head of the Music Representation Research Group at IRCAM (Institut de Recherche et de Coordination Acoustique/Musique) in Paris, and head of the STMS (Sciences and Technologies of Music and Sound) Ircam/CNRS Lab. Born in 1960, he studied computer science, music and linguistics. In 1980, while still a student, he won research awards in "Art and the Computer", a national software contest launched in 1980 by the French Ministry of Research, and another one in the "Concours Micro", a contest in computing in the arts using early micro-computers. In the mid-eighties, he wrote the first IRCAM environment for score-oriented Computer Assisted Composition. In the mid-nineties he created, with Carlos Agon, the OpenMusic environment which is currently the standard for computational composition and musicology. . The concept behind OpenMusic is to provide a visual counterpart of major programming paradigms (such as functional, object and logical programming) along with an extensive set of musical classes and methods, plus an original metaphor for representing musical time in its logical, as well as chronological, aspects. Recently Gerard Assayag has created with other colleagues the OMax computaitonal improvisation system based on machine listening and machine learning and has become a widely recognized reference in the field. Gerard Assayag's research interests center on music representation issues, and include computer language paradigms, machine learning, constraint and visual programming, computational musicology, music modeling, and computer-assisted composition. His research results are regularly published in proceedings, books and journals.
Information retrieval and deployment in... par Ircam-CGP
Abstract: Over the past years, music information research has elaborated powerful tools for creating a new generation of applications that redefine the boundaries of music listening and music making. The recent availability of affordable motion capture technology has not just allowed for creating novel musical instruments, but also for integrating the study of bodily motion and gesture into the mainstream of music information research. We will present a variety of playful real-time interactive applications based on analysis techniques and models combining digitized sounds, movements, and symbolic representations.
Bio: Norbert Schnell is researcher and developer at the Real-Time Musical Interactions team at IRCAM focussing on real-time digital audio processing techniques for interactive music applications. He studied Telecommunications and Music in Graz/Austria and worked as studio assistant at the IEM. At IRCAM he initiated and participated in numerous international research and development projects as well as artistic works in the field of interactive audio-visual installations, music pedagogy, and sound simulation. He chaired the 6th International Conference on New Interfaces for Musical Expression (NIME) in 2006 and held the DAAD Edgard Varèse Guest Professorship for Electronic Music at the Technische Universität Berlin in 2007. Currently he focuses on his PhD on the animation of digitized sounds and their re-embodiment by bodily movements and gestures.
Gestural Re-Embodiment of Digitized Sound and... par Ircam-CGP
Abstract: The wealth of tools developed in music information retrieval (MIR) for the description, indexation, and retrieval of music and sound can be easily (ab)used for the creation of new musical material and sound design. Based on automated audio description and selection, corpus-based concatenative synthesis allows to exploit large collections of sound to compose novel timbral and harmonic structures. The metaphor for musical creation is here an explorative navigation through the sonic landscape of the corpus. We will present examples and applications of real-time interactive corpus-based concatenative synthesis for music composition, sound design, installations, and interactive performance.
Bio: Diemo Schwarz is researcher–developer at the Real-Time Music Interaction (IMTR) team at Ircam, working on sound analysis and interactive corpus-based concatenative synthesis in multiple research and musical projects at the intersection between computer science, music technology, and audio-visual creation. He holds a PhD in computer science applied to music from the University of Paris, awarded in 2004 for the development of a new method of concatenative musical sound synthesis by unit selection from a large database.
Interactive Exploration of Sound Corpora for... par Ircam-CGP
Abstract: Un des aspects les plus prospectifs de la relation instrument/machine réside dans le développement des moyens d’analyse acoustique des sons instrumentaux et vocaux en temps réel que l’on appelle descripteurs audio. Le nombre de ces descripteurs ne cesse de s’accroître au fil du temps : centroid, spread, skewness, kurtosis, roll-off,fundamental frequency, noisiness, inharmonicity, odd-to-even energy ratio, deviation, loudness, roughness, … Ces termes désignent des attributs du son que l’on n’a découverts que récemment. Notre connaissance du son se complexifie. L’extraction de paramètres constitutifs du son en vue de leur utilisation en tant qu’éléments compositionnels permet de créer une cohérence musicale entre les sons acoustiques et les sons électroniques. Cependant, seul un petit nombre de ces descripteurs sont aujourd’hui réellement adaptés aux impératifs musicaux réels. Le perfectionnement de ces descripteurs est un axe de recherche indispensable si l’on souhaite réduire encore l’écart qui sépare le monde instrumental de celui des sons électroniques.
Bio: Allergic to academic studies and a sworn autodidact, Philippe Manoury presented his first compostions to Gérard Condé who introduced him to Max Deutsch, a former student of Arnold Schoenberg. He initially studied composition at the École normale de musique de Paris where he also worked on harmony and counterpoint before going on to study at the Cnsmdp under Ivo Malec, Michel Philippot (composition), and Claude Ballif (analysis). It is the premiere of Cryptophonos performed by the pianist Claude Helffer at the Metz Festival in 1974 that introduced the public to Manoury’s works. In 1978, he moved to Brazil where he taught classes and conferences on contemporary music in a number of universities (Sao Paulo, Brasilia, Rio de Janeiro, Salvador). In 1981, Manoury returned to France where he was a guest researcher at IRCAM. Since this era, he has continued to be a part of the activities at the institute, as a composer or as a professor. At IRCAM Manoury developed a range of research in the domain of real-time interaction between acoustic instruments and new technologies in connection with computer-music in collaboration with the mathematician Miller Puckette. From this work, his series of interactive works for a range of instruments was born: Sonus ex machina, comprenant Jupiter, Pluton, La Partition du Ciel et de l’Enfer, and Neptune. From 1983 to 1987, Philippe Manoury was the head of Education with the Ensemble intercontemporain. He was a composition and electronic music professor at the CNSMD de Lyon from 1987 to 1997. After numerous residencies in diverse institutions in France and abroad, Philippe Manoury decided in 2004 to divide his time between Europe and the United States where he teaches composition at the University of California San Diego.
Descripteurs audio : un enjeux majeur pour la... par Ircam-CGP
Abstract: For the past 60 years, machines have been involved in all aspects of music: playing, recording, processing, editing, mixing, composing, analyzing, and synthesizing. However, in software terms, music is nothing but a sequence of numbers and functions describing waveforms (what to play) and scores (when to play). It doesn't have a notion of what music sounds like, and how it is perceived and received by listeners, in its context, time and space. The Echo Nest is a music intelligence company that provides a deep and granular level of musical information at scale, on both content and context. By listening to every song (tempo, rhythm, timbre, harmony), and reading every piece of music text online (blog posts, news, reviews), the "musical brain" constantly learns to reverse engineer music. Its knowledge on 35 million unique songs and 2 million artists was generated automatically and dynamically over the past 6 years. Through many examples and live demos, we demonstrate the power of big-data driven software in the context of personalized listening experiences and music creation.
Bio: Tristan earned a doctorate in Media Arts and Sciences from MIT in 2005. His academic work combined machine listening and machine learning technologies in teaching computers how to hear and make music. He first earned an MS in Electrical Engineering and Computer Science from the University of Rennes in France, later working on music signal parameter extraction at the Center for New Music and Audio Technologies at U.C. Berkeley. He has worked with leading research and development labs in the U.S. and France as a software and hardware engineer in areas of machine listening and audio analysis. He is a co-founder and the Chief Science Officer of Music Intelligence company The Echo Nest, which powers smarter music applications for a wide range of customers including MTV, Spotify, The BBC, MOG, eMusic, Clear Channel, Rdio, EMI, and a community of more than 12,000 independent application developers.
Playing with Music par Ircam-CGP
Abstract: This talk introduces a new music interaction system based on style modeling, in a MIR-oriented perspective called VirtualBand. VirtualBand aims at combining musical realism and quality with real-time interaction, by capturing essential elements of the style of a musician and by reusing these elements during the musical improvisation of the user so that an interactive, real-time musical engagement takes place just as it happens with a real band of responsive musicians. To enable this to take place, we address style modeling from the new perspective of combinatorial statistical modeling. Markov chains provide a definition of style, though rudimentary, as the set of local patterns of a given fixed length. However, Markov chain approaches suffer from a latent “control problem”: control constraints are not compatible with Markov models, as they induce long-range dependencies that violate the Markov hypothesis of limited memory. To overcome this problem, we have reformulated Markov generation in the framework of constraint satisfaction, and have demonstrated that this approach solves the control problem, and opens the door to fully malleable representations of style. VirtualBand uses this technology to provide interactive jazz accompaniment. VirtualBand proceeds in two steps: a recording and a playing phase. First, recordings of professional musicians are analyzed to extract musical metadata (such as harmony, energy, or rhythm) to build a style database. When the musician plays, VirtualBand explores the style database, for each virtual musician, to produce music that matches the players’ own performance features (e.g., volume, density of notes, pitch). Thanks to this adaptive behavior, the playing experience is unique: every time the user plays with the system the rhythm section adapts to the performance and generates a new accompaniment.
Bio: François Pachet received his Ph.D. and Habilitation degrees from Paris 6 University (UPMC). He is a Civil Engineer (Ecole des Ponts and Chaussées) and was Assistant Professor in Artificial Intelligence and Computer Science, at Paris 6 University, until 1997. He then set up the music research team at SONY Computer Science Laboratory Paris, where he developed the vision that metadata can greatly enhance the musical experience in all its dimensions, from listening to performance. His team conducts research in interactive music listening and performance and musical metadata and developed several innovative technologies (constraint-based spatialization, intelligent music scheduling using metadata) and award winning systems (MusicSpace, PathBuilder, The Continuator for Interactive Music Improvisation, etc.). He is the author of over 80 scientific publications in the fields of musical metadata and interactive instruments. His current research focuses on creativity and content generation, as he was recently awarded an ERC Advanced Grant to develop the concepts and technologies of "flow machines": a new generation of content generation tools that help users find and develop their own "style".
VirtualBand, a MIR-approach to interactive... par Ircam-CGP
Abstract: Although MIR did arguably not start as a research discipline for promoting creativity and music performance, this trend has begun to gain importance in recent years. The possibilities of MIR for supporting musical creation and musical education are indeed many-folded. While the use of MIR techniques for real-time music creation may indeed help both experts and complete novices to explore new creative musical universes laying somewhere in between micro-level synthesis control and macro-level remixing, the application of MIR tools oriented to music education and to children's musical performance seems another unexplored area with radical new possibilities. In this talk we will describe some recent applications of MIR techniques to music and multimodal creation recently developed at the MTG and at Reactable Systems, and we will explore the potential of MIR for children music education and performance.
Bio: Sergi Jordà (1961) holds a B.S. in Fundamental Physics and a Ph.D. in Computer Science and Digital Communication. He is a researcher in the Music Technology Group of Universitat Pompeu Fabra in Barcelona, where he specializes in real-time interaction and tabletop interfaces, and an Associate Professor in the same university, where he teaches computer music, HCI, and interactive media arts. He has written many articles, books, given workshops and lectured though Europe, Asia and America, always trying to bridge HCI, music performance and interactive media arts. He has received several international awards, including the prestigious Ars Electronica’s Golden Nica in 2008. He is currently best known as one of the inventors of the Reactable, a tabletop musical instrument that in 2007 accomplished mass popularity after being integrated in Icelandic artist Bjork’s Volta world Tour. He is also one of the founding partners of Reactable Systems, a spin-off company created in 2009 (www.reactable.com).
MIR beyond retrieval: Music Performance... par Ircam-CGP