FASTLab Tools for Music Information Retrieval
Audio Analysis and Feature Extraction Frameworks
Over the past 12 years, FASTLab has developed and deployed several generations of digital audio analysis and feature extraction tools in C++, python, Java, Smalltalk and MATLAB, leading to a variety of concrete applications. The four versions of this C++ framework went under the name of the FASTLab Music Analysis Kernel, or FMAK. FMAK version 1 (2000-02) was sold to Parasoft/Predixis who later became MusicIP then AmpliFind, and are now owned by GraceNote. FMAK version 2 (2003-4) was licensed to Matsushita/Panasonic for the EMA application (See below); FMAK version 3 (2005-6) was licensed to Catalyst Mobile for a search engine; and FMAK version 4 (2007-11, along with the FMAK name) was acquired by Imagine Research, Inc. in 2010.
The underlying audio feature extraction model allows the population of large data sets of analyzed material, be it songs, loops, or sound effects. For each item, we extract a rich feature vector (see the list on the right of the Figure below) and support search, match, sorting, etc. on diverse data sets and feature weightings.
The low-level signal processing analyzes a set of simple time- and frequency-domain features, from which higher-level features such as tempo and key can be learned.
A wide variety of applications has been developed using the FMAK framework in C++; these range from song recommender systems to music transcription tools, as illustrated below and in the other pages on this site.
Music Recommender Systems
We have developed several content-based music recommender systems. Given a database of musical songs (such as a large iTunes collection), we analyze the songs and "listen" to them, recording a set of over 100 features in a database for later use. The recommender system then takes one or more target songs and creates a playlist of similar songs. The figure below shows the user interface of the SndsLike recommender system.
Expert Mastering Assistant (EMA)
As another example of our analysis and processing applications is the Expert
Mastering Assistant (EMA) program we developed under contract from Panasonic. The goal of EMA is to assist in the remastering of stereo CD-based music to higher-resolution surround-sound formats. The EMA system analyzes a musical selection and classifies it using more than 100 parameters within a detailed genre database. In the next stage, a rule-based expert system compares the recording and production of the current selection to that of its genre, and proposes high-level mapping parameters (dark/bright,
loose/tight,
narrow/wide, small/large, and focused/diffuse) for the remastering. Finally, mastering signal processing (e.g., gain control, equalization, reverb, and dynamic-range processing) can be done in real-time with high-level and low-level interactive control. A screen shot of the EMA application's main display is shown below.
Other applications of this kind of analysis-classification-processing-display software include music finger-printing, summarization/thumb-nailing, content-based search engines, smart players, speaker identification, and post-production for streaming or broadcast. The white papers available below describe FASTLab, Inc. music information retrieval technologies.
Please feel free to contact us with questions on any of these systems.
Downloads (PDF files)
Contacting FASTLab, Inc.
FASTLab, Inc.
Email: info - at - fastlabinc.com
