JSAP units are HTML5 audio processors for Web Audio API enabled sites. These allow the designing and deployment of advanced audio units with similar functionality to desktop VST and AU processors.Read more ->
The Open Multitrack Testbed is an online repository of multitrack audio, mixes or processed versions thereof, and corresponding mix settings or process parameters such as DAW files.Read more ->
The SAFE Project is motivated by the lack of statistically-defined transferable semantic terms in music production and the requirement for more intuitive control of low-level parameters.Read more ->
Perceptual listening tests are commonplace in audio research and a vital form of evaluation.Read more ->
The Semantic Audio Feature Extraction Dataset (SAFE-DB) is a continually updating database of semantically annotated music production metadata, taken from an international user group of sound engineers.Read more ->
To make the SAFE dataset of music production metadata confluent with semantic web technologies, we have produced a dataset of RDF triples representing the SAFE database.Read more ->
For software integration, we have released an open source web API to access the SAFE dataset.Read more ->
The project defines both the host and process frameworks to ease deployment. On the host side, we define the PluginFactory to hold the JSAP prototypes. The Factory can generate the plugins and manage them from one central resource. The host also defines SubFactory units which define a chain of plugins in the traditional form. These SubFactory units handle:
The standard also specifies a new method for cross-adaptive processing by sharing features rather than routing audio. If several plugins require the same feature from the same audio stream, traditional systems would require the audio to be routed between all the nodes and then process the audio multiple times to generate the same result. The JSAP system attaches a JS-Xtract node to each plugin output allowing for advanced querying of the features. These features are then sent to the requesting plugins, saving on processing power as well as improved memory management by sharing objects
The documentation is currently under development, as the standard itself is being processed. Documentation can be found here
The code can be accessed on GitHub
This work was presented at the AES 141st Convention in Los Angeles, USA and at the 2nd Workshop on Intelligent Music Production. Please cite this paper in any academic works: Jillings et al. “JSAP: A Plugin Standard for the Web Audio API with Intelligent Functionality”. Audio Engineering Society Convention 141, Los Angeles, CA. September, 2016. Available at http://www.aes.org/e-lib/browse.cfm?elib=18397. bibtex
The Open Multitrack Testbed is an online repository of multitrack audio, mixes or processed versions thereof, and corresponding mix settings or process parameters such as DAW files.
Multitrack audio is a much sought after resource for audio researchers, students, and content producers, and while some online resources exist, few are large and reusable and none allow querying audio fulfilling specific criteria.
This Testbed, however, present contains a semantic database of metadata corresponding with the songs and individual tracks, enabling users to retrieve “all recordings of an accordion from Canada”, or “all songs with at least 10 different mixes”.
Many entries have a Creative Commons license, allowing us to host the material and allowing you to use, edit and/or redistribute it for your own purposes. Some audio is hosted on other websites, but can be found using our multitrack browser or search engine because of the rich metadata we attach to it.
When using the Open Multitrack Testbed for academic purposes, please cite Brecht De Man, Mariano Mora-Mcginity, György Fazekas and Joshua D. Reiss, “The Open Multitrack Testbed,” 137th Convention of the Audio Engineering Society, October 2014. [pdf | BibTeX | poster | website]
The SAFE Project is motivated by the lack of statistically-defined transferable semantic terms in music production and the requirement for more intuitive control of low-level parameters. One of the main outcomes of the project is a suite of DAW plug-ins that allow you to both save and load semantic terms from within the audio production workflow. This means you can now control your audio effect plug-ins using terms that normal human beings can understand! Just type a term (such as warm, bright, etc…) into the text-box, and if we have it on our server, your parameters will be moved around to represent this. Similarly, if you achieve an effect that can be explained in semantic terms, input it into the box and hit save. As the plug-ins are based on the music production community, the more users that input terms, the more representative the effects become for everyone else!
The song used in the video is The Alchemist by Little Tybee.
All the stems and other things can be found here:
R. Stables, S. Enderby, B. De Man, G. Fazekas, and J. D. Reiss, “SAFE: A system for the extraction and retrieval of semantic audio descriptors,”, The International Society for Music Information Retrieval (ISMIR), 2014.
You can access the source code to the plug-ins: here. Alternatively, you can clone the repo using the following link:
git clone https://github.com/semanticaudio/SAFE.git
To demonstrate this, the following demonstration uses your microphone and processes the data in real time on your device. No data is transmitted or captured in any way as all the processing is completely done in your browser. We give you an option of several functions to calculate with already and the results are shown in the graph.
Full documentation on the library is available here. This covers the features available, using the object-oriented or procedural functions and guides on using it with a Web Audio environment.
Source code for the library can be found on Github: https://github.com/nickjillings/js-xtract
N. Jillings, J. Bullock, R. Stables “JS-Xtract: A Realtime Audio Feature Extraction Library for the Web”, International Society for Music Information Retrieval Conference (ISMIR 2016), August 2016. [PDF] [bibtex]
Perceptual listening tests are commonplace in audio research and a vital form of evaluation. Many tools exist to run such tests, however many operate one test type and are therefore limited whilst most require proprietary software. Using Web Audio, the Web Audio Evaluation Tool (WAET) addresses these concerns by having one toolbox which can be configured to run many different tests, perform it through a web browser and without needing proprietary software or computer programming knowledge. In this paper the role of the Web Audio API in giving WAET key functionalities are shown. The paper also highlights less common features, available to web based tools, such as easy remote testing environment and in-browser analytics.
MUSHRA test on the WAET platform
APE on the WAET platform
The Web Audio Evaluation Toolbox supports building tests complying to the following known standards:
Source code and examples are available at: https://github.com/BrechtDeMan/WebAudioEvaluationTool
Jillings, N., Moffat, D., De Man, B., Reiss, JD., and Stables, R. “Web Audio Evaluation Tool: A framework for subjective assessment of audio”. In Proceedings of the 2nd Web Audio Conference (WAC-2016), Atlanta, 2016. [PDF]
Jillings, N., De Man, B., Moffat, D., & Reiss, JD. “Web Audio Evaluation Tool: A Browser-Based Listening Test Environment”, in Proceedings of the 12th Sound and Music Conference, Maynooth, 2015. [PDF]
The Semantic Audio Feature Extraction Dataset (SAFE-DB) is a continually updating database of semantically annotated music production metadata, taken from an international user group of sound engineers. The data is taken from 4 audio effects: a dynamic range compressor, an overdrive distortion, a parametric equaliser and an algorithmic reverb. Each entry into the dataset contains the following items:
A string representing a user’s description of the audio transformation (e.g. warm, bright, fluffly).
An array of audio features extracted from the audio file, taken before and after processing is applied to the signal. Feature list: Mean, Variance, StdDev, RMS_Amplitude, Zero_Crossing_Rate, Spectral_Centroid, Spectral_Variance, Spectral_Standard_Deviation, Spectral_Skewness, Spectral_Kurtosis, Irregularity_J, Irregularity_K, F0 (autocorrelation), Smoothness, Spectral_Roll_Off, Spectral_Flatness, Spectral_Crest, Spectral_Slope, Peak_Spectral_Centroid, Peak_Spectral_Variance, Peak_Spectral_Standard_Deviation, Peak_Spectral_Skewness, Peak_Spectral_Kurtosis, Peak_Irregularity_J, Peak_Irregularity_K, Peak_Tristimulus (x3), Inharmonicity, Harmonic_Spectral_Centroid, Harmonic_Spectral_Variance, Harmonic_Spectral_Standard_Deviation, Harmonic_Spectral_Skewness, Harmonic_Spectral_Kurtosis, Harmonic_Irregularity_J, Harmonic_Irregularity_K, Harmonic_Tristimulus (x3) Noisiness, Parity_Ratio, Bark_Coefficients (x25), MFCCs (x13)
A variable sized array of plug-in parameters used to apply processing to the signal.
A series of strings representing the user’s age, location, language and experience, along with the genre of the music and the instrument being processed. These fields can be left blank.
Note: To limit the file-size of the download, feature-sets are averaged. For data from a specific date, or for multi-channel time series features please get in touch
Last Updated: Monday 11th July 2016
R. Stables, S. Enderby, B. De Man, G. Fazekas, and J. D. Reiss, “SAFE: A system for the extraction and retrieval of semantic audio descriptors”, The International Society for Music Information Retrieval (ISMIR), 2014. [pdf | bib]
Using multidimensional scaling, we can map the descriptive terms from the dataset into a more intuitive space. The interactive visualisation below shows the feature space of the SAFE-DB in 3-dimensions.
To make the SAFE dataset of music production metadata confluent with semantic web technologies, we have produced a dataset of RDF triples representing the SAFE database. This can be access here through an endpoint, or via a semantic web API.
For software integration, we have released an open source web API to access the SAFE dataset. Here, you can learn how to use the API and download the framework that was developed to host and distribute the dataset…
You can access the web API developed around the SAFE data using the following link: https://github.com/semanticaudio/safe-api