Melody Detection in Polyphonic Musical Signals: Exploiting Perceptual Rules, Note Salience and Melodic Smoothness
Authors
Abstract
Research on melody extraction from polyphonic audio is a subject of increasinginterest. In fact, it has a wide range of applications, for example in Music
Information Retrieval (MIR) - namely regarding query-by-humming in audio
databases, i.e., searching musical databases through hummed queries -, automatic
melody transcription, performance and expressiveness analysis, extraction of
melodic descriptors for music content metadata, plagiarism detection, to name but a
few. This is all the more relevant nowadays, as digital music archives are
continuously expanding. The present state of affairs places new challenges to music
librarians and service providers, regarding the organization of large-scale music
databases and the development of meaningful ways of interaction and retrieval.
In this article, we tackle the problem of melody detection in polyphonic audio,
following a multi-stage approach, inspired on principles from perceptual theory and
musical practice. Our system comprises three main modules: pitch detection,
determination of musical notes (with precise temporal boundaries, pitches and
intensity levels) and identification of melodic notes. The main contribution of this
article is in the last module, where a number of rule-based systems are proposed
with the objective of extracting the notes that convey the main melodic line among
the whole set of detected notes. The system performs satisfactorily in a small
database collected by us and in the database created for the ISMIR'2004 melody
extraction contest. However, the performance of the algorithm decreased in
the MIREX'2005 database.
Keywords
naSubject
Music Information RetrievalJournal
Computer Music Journal, Vol. 30, #4, na, December 2006PDF File
Cited by
Year 2015 : 2 citations
Laaksonen, Antti. "Algorithms for melody search and transcription." (2015).
DOMINGO BALLESTER, M. I. G. U. E. L. Reconocedor automático de melodías de música clásica. Diss. 2015.
Year 2014 : 7 citations
Salamon, Justin, et al. "Melody extraction from polyphonic music signals: Approaches, applications, and challenges." Signal Processing Magazine, IEEE 31.2 (2014): 118-134.
Ishwar, Vignesh. Pitch Estimation of the Predominant Vocal Melody from Heterophonic Music Audio Recordings. Diss. Master Thesis, Universitat Pompeu Fabra, Barcelona, 2014.
Giraldo, Sergio, and Rafael Ramirez. "Optimizing Melodic Extraction Algorithm for Jazz Guitar Recordings Using Genetic Algorithms." Joint Conference ICMC-SMC. 2014.
BRISUDA, Rudolf. "Transcription of Piano Music."
Nieto, Belén. "Addressing user satisfaction in melody extraction."
Laaksonen, Antti. "Automatic Melody Transcription based on Chord Transcription." ISMIR. 2014.
Brisuda, Rudolf. "Tones Recognition for Displaying Interactive Music Sheets on Mobile Devices." (2014).
Year 2013 : 5 citations
Salamon, J., Gómez E., Ellis D. P. W., & Richard G. (2013). “Melody Extraction from Polyphonic Music Signals: Approaches, Applications and Challenges”, IEEE Signal Processing Magazine (in press).
Rajan, R., Murthy, H.A.,"Group delay based melody monopitch extraction from music",2013,"ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",,, 6637634,186,190,,,"http://www.scopus.com/inward/record.url?eid=2-s2.0-84890505347&partnerID=40&md5=4d9974173b4d0539f70b90891999983c",Conference Paper,Scopus
Yao, G., Zheng, Y., Xiao, L., Ruan, L., Li, Y.,"Efficient vocal melody extraction from polyphonic music signals",2013,"Elektronika ir Elektrotechnika",19,6,,103,108,,,"http://www.scopus.com/inward/record.url?eid=2-s2.0-84879285786&partnerID=40&md5=d3282a096d6ecf7671c58cf7f750665e",Article,Scopus
Rajan, Rajeev, and Hema A. Murthy. "Melodic pitch extraction from music signals using modified group delay functions." Communications (NCC), 2013 National Conference on. IEEE, 2013.
Yao, G., et al. "Efficient Vocal Melody Extraction from Polyphonic Music Signals." Electronics and Electrical Engineering 19.6 (2013): 103-108.
Year 2012 : 3 citations
Salamon, J.; Gomez, E.; , "Melody Extraction From Polyphonic Music Signals Using Pitch Contour Characteristics," Audio, Speech, and Language Processing, IEEE Transactions on , vol.20, no.6, pp.1759-1770, Aug. 2012
Salamon J. and Urbano J. (2012), “Current Challenges in the Evaluation of Predominant Melody Extraction Algorithms”, 13th International Society for Music Information Retrieval Conference (ISMIR 2012).
J Salamon, G Peeters, A Röbel (2012). “Statistical characterisation of melodic pitch contours and its application for melody extraction”, ISMIR 2012
Year 2011 : 5 citations
Seokhwan Jo, Chang D. Yoo, and Arnaud Doucet (2011). “Melody Tracking based on Sequential Bayesian Model”, to appear for publication in IEEE Journal of Selected Topics in Signal Processing, November 2011.
S Jo, CD Yoo, A Doucet (2011). “Melody Tracking Based on Sequential Bayesian Model”, Selected Topics in Signal Processing, Vol. 5, No. 6, pp. 1216-27
Martens, Ga. "Extraction and representation of semantic information in digital media." (2011): HASH-0x6fd2dd8.
LF Thomaz (2011). “Um arcabouço para construçao de sistemas multiagente musicais”, PhD Thesis, Universidade de São Paulo, Brazil
Liao et al. (2011). “On the use of perceptual properties for melody estimation”, DAFx 2011.
Year 2010 : 4 citations
Vishweshwara Rao and Preeti Rao (2010). “Vocal melody extraction in the presence of pitched accompaniment in polyphonic music”, IEEE Transactions on Audio, Speech and Language Processing, Vol. 18(8), pp. 2145-2154.
Jo S. and Yoo C. D. (2010). “Melody extraction from polyphonic audio based on particle filter”, ISMIR’2010.
H Kwok (2010). “Using generative techniques to visualize music in a meaningful way”, BA Thesis, Using generative techniques to visualize music in a meaningful way
S Joo, S Jo, CD Yoo (2010). “Melody extraction from polyphonic audio signal”, mirex2010
Year 2009 : 6 citations
Dias R. (2009). “MuVis: Localização e Visualização de Músicas”, MSc Thesis, Technical University of Lisbon.
Vishweshwara Rao and Preeti Rao (2010). “Improving Polyphonic Melody Extraction By Dynamic Programming Based Dual F0 Tracking”, Proc. of the 12th Int. Conference on Digital Audio Effects (DAFx-09), Como, Italy, September 1-4, 2009.
Masahiro Niitsuma Tsutomu Fujinami‡ Yo Tomita (2009). “THE INTERSECTION OF COMPUTATIONAL ANALYSIS AND MUSIC MANUSCRIPTS: A NEWMODEL FOR BACH SOURCE STUDIES OF THE 21ST CENTURY”. International Society for Music Information Retrieval Conference (ISMIR 2009).
Niitsuma M. et al. (2009). “The Intersection of Computational Analysis and Music Manuscripts: A New Model for Bach Source Studies of the 21st Century”, ISMIR’2009.
Reyes N. R., Candeas P. V., Quesada F., Carabias J. J (2009). “New algorithm based on spectral distance maximization to deal with the overlapping partial problem in note-event detection”. Signal Processing, Vol. 89, No. 8, pp. 1653-1660.
V Rao, P Rao, Improving polyphonic melody extraction by dynamic programming based dual f0 tracking. Proc. of the 12th Int. Conference on Digital Audio Effects (DAFx-09), Como, Italy, September 1-4, 2009
Year 2008 : 3 citations
Canadas-Quesada, Francisco Jesus; Carabias-Orti, Julio Jose; Mata-Campos, Raul; Ruiz-Reyes, Nicolas; Vera-Candeas, Pedro (2008). “Polyphonic Piano Transcription Based on Spectral Separation”. 124th Audio Engineering Convention
Martins L. G. (2008). A Computational Framework for Sound Segregation in Music Signals. PhD Thesis, University of Porto, Portugal.
Oudtshoorn B. (2008). “Investigating the Feasibility of Near Real-Time Music Transcription on Mobile Devices”. Internal Report, University of Western Australia.