Geoffroy Peeters

Project leader - Audio and Music Indexing Research Projects - IRCAM R&D

Main Technologies Publications Supervision Commitees Projects Teaching Test-Sets

Automatic Generation of Visual Summary/Audio Summary from Signal Analysis

1. Description general

Developpement d'algorithmes permettant d'extraire automatiquement la structure d'un morceau de musique a partir de son signal audio.

Cette structure est ensuite utilisee pour la generation d'un resume sonore (court segment musical resumant les differents contenus d'un morceau).

2. Methode proposee

2.1. Features extraction

Des descripteurs sont d'abord extrait a chaque instant du morceau de musique (description du timbre, de l'evolution du timbre, de la hauteur).

2.2. Structure estimation

L'estimation de la structure du morceau est basee sur une mesure de repetition:

a) Sequence representation

The music audio signal is considered as repetitions of sequences of events.

  1. Lines are derived from the so-called similarity matrix using 2D structuring filter.
  2. Detected lines are then analyzed in order to detecte sequences repetitions in the audio track.

Illustrations of the results on the title "Love me do" from the artist "The Beatles" are indicated in Figure 1.
b) State representation

The music audio signal is considered as a succession of states so that each state represents a (somehow) similar information found in the different parts of the music. The states are found using a two-pass algorithm based

  1. on segmentation and
  2. unsupervised learning algorithm (k-means + hidden Markov Model).

Illustrations of the results on the title "Oh so quiet" from the artist "Bjork" are indicated in Figure 2.

2.4. Audio summary generation

Signal represented by a successions of sequences/states AABABCAAB Which sequences/states use for the summary ?

Short fragment of audio signal corresponding to chosen states Information continuity: Overlap-add + Tempo/beat

2.5. Examples


Figure 2. State approach on the title "Oh so quiet" from the artist "Bjork"


Figure 1. Sequence approach on the title "Love me do" from the artist "The Beatles"

3. Related publications