Ressources.ircam

Abstract: This talk introduces a new music interaction system based on style modeling, in a MIR-oriented perspective called VirtualBand. VirtualBand aims at combining musical realism and quality with real-time interaction, by capturing essential elements of the style of a musician and by reusing these elements during the musical improvisation of the user so that an interactive, real-time musical engagement takes place just as it happens with a real band of responsive musicians. To enable this to take place, we address style modeling from the new perspective of combinatorial statistical modeling. Markov chains provide a definition of style, though rudimentary, as the set of local patterns of a given fixed length. However, Markov chain approaches suffer from a latent “control problem”: control constraints are not compatible with Markov models, as they induce long-range dependencies that violate the Markov hypothesis of limited memory. To overcome this problem, we have reformulated Markov generation in the framework of constraint satisfaction, and have demonstrated that this approach solves the control problem, and opens the door to fully malleable representations of style. VirtualBand uses this technology to provide interactive jazz accompaniment. VirtualBand proceeds in two steps: a recording and a playing phase. First, recordings of professional musicians are analyzed to extract musical metadata (such as harmony, energy, or rhythm) to build a style database. When the musician plays, VirtualBand explores the style database, for each virtual musician, to produce music that matches the players’ own performance features (e.g., volume, density of notes, pitch). Thanks to this adaptive behavior, the playing experience is unique: every time the user plays with the system the rhythm section adapts to the performance and generates a new accompaniment.

Bio: François Pachet received his Ph.D. and Habilitation degrees from Paris 6 University (UPMC). He is a Civil Engineer (Ecole des Ponts and Chaussées) and was Assistant Professor in Artificial Intelligence and Computer Science, at Paris 6 University, until 1997. He then set up the music research team at SONY Computer Science Laboratory Paris, where he developed the vision that metadata can greatly enhance the musical experience in all its dimensions, from listening to performance. His team conducts research in interactive music listening and performance and musical metadata and developed several innovative technologies (constraint-based spatialization, intelligent music scheduling using metadata) and award winning systems (MusicSpace, PathBuilder, The Continuator for Interactive Music Improvisation, etc.). He is the author of over 80 scientific publications in the fields of musical metadata and interactive instruments. His current research focuses on creativity and content generation, as he was recently awarded an ERC Advanced Grant to develop the concepts and technologies of "flow machines": a new generation of content generation tools that help users find and develop their own "style".