Introduction
This page gives you the vocabulary and a map of what libsonare covers. If you would rather run something first, go to Getting Started and come back; if you already know what you are building, the Learning Path picks the shortest route. The Glossary and MIR Overview go deeper on each term.
What You Will Learn
By the end of this page you should be able to:
- explain what libsonare is and which kinds of music/audio tools it supports;
- understand the basic MIR (Music Information Retrieval) pipeline from waveform to spectrogram, features, and higher-level music analysis;
- distinguish between analysis, editing, mixing, mastering, realtime streaming, room acoustics, built-in instruments, the headless-DAW runtime, and inverse features;
- choose the next page from the Learning Path without needing DSP background.
What is libsonare?
libsonare is a dependency-free audio engine for browser-native and native music tools. It covers the whole chain from analyzing existing audio (tempo, key, chords) to producing new audio (mixing, mastering, and sequencing MIDI into finished tracks).
libsonare covers several related jobs:
| Area | What libsonare can do | Read next |
|---|---|---|
| Music Information Retrieval (MIR) | Extract tempo, key, chords, beats, downbeats, time signature, structure, melody, timbre, and dynamics | Music Analysis |
| Source separation | Split a recording into harmonic and percussive parts (HPSS) or into unlabeled spectral components (NMF). Signal processing, not a trained separator | Source Separation |
| Audio to MIDI | Transcribe a mono recording into note events, or straight into a project's MIDI clip | Audio to MIDI |
| Editing and creative FX | Change pitch, timing, voice character, reverb, and modulation-style effects | Editing DSP, Effects Inserts |
| Repair | Measure and reduce noise, hum, clipping, clicks, crackle, and diffuse reverb | Audio Repair Workflow |
| Mixing | Build a real-time-safe mixer with routing, buses, sends, and meters | Mixing Engine |
| Mastering | Run broadcast-grade mastering processors and loudness/true-peak handling | Mastering Processors, Mastering Assistant |
| Built-in instruments and MIDI | Turn MIDI notes into sound with a built-in synthesizer (seventeen engines, eleven of them physical models, with a General MIDI and GS fallback bank that needs no sample files), or with a SoundFont 2 player (a SoundFont is a file of sampled instruments you bring yourself) | Sound Sources for why no samples ship, then Built-in Synthesizer |
| Headless-DAW runtime | Author projects with audio/MIDI tracks, sequence MIDI, and bounce (render the project to an audio file) or play back in realtime | Project Editing, Bouncing Projects |
| Streaming and realtime | Analyze audio block by block as it arrives, and run a transport (the play/stop/position clock) with live MIDI | Realtime and Streaming |
| Room acoustics | Measure, estimate, synthesize, or morph room character | Room Acoustics |
libsonare is written in C++17 for performance and can be compiled to WebAssembly, making it possible to run the same processors directly in web browsers — no server required.
What is WebAssembly?
WebAssembly (WASM) is a binary instruction format that runs in web browsers at near-native speed. It lets code written in languages like C++ be compiled and run in the browser, with no server and nothing to install. libsonare uses this to bring C++-level audio analysis performance directly to web applications.
What is Music Information Retrieval?
Music Information Retrieval (MIR) is a field of research that deals with extracting meaningful information from music. When you listen to a song, your brain effortlessly recognizes the tempo, identifies chord changes, and feels the song structure. MIR aims to do the same computationally.
Common MIR tasks include:
| Task | Question it answers |
|---|---|
| Tempo estimation | How fast is this song? (e.g., 120 BPM) |
| Key detection | What key is this song in? (e.g., C major) |
| Beat tracking | Where exactly are the beats? |
| Chord recognition | What chords are being played and when? |
| Structural analysis | Where are the verse, chorus, and bridge? |
| Pitch tracking | What notes are being sung or played? |
What is BPM?
BPM stands for beats per minute — the speed of a song's pulse. 60 BPM is one beat per second; most pop songs sit around 90–130 BPM.
libsonare provides all of these capabilities in a single library.
Who is it for?
libsonare is designed for developers building music-related applications:
Web app developers — Add audio analysis to your app without a backend. Detect BPM for a DJ tool, visualize chords for a practice app, or build an automatic song structure viewer, all running client-side in the browser via WebAssembly.
Music tool creators — Build DAW plugins, chord detectors, auto-transcription tools, browser mixers, mastering workflows, or music education software. libsonare provides the analysis, mixing, and mastering engine so you can focus on the user experience.
What is a DAW?
A DAW (Digital Audio Workstation) is software for recording, editing, and producing audio. DAW plugins extend it with effects, instruments, or analysis tools.
Audio researchers — If you use librosa in Python, libsonare provides a familiar API with compatible parameters and algorithms, but runs the heavy pipeline in native C++ and also works in the browser.
Game / interactive media developers — Analyze music in real time for rhythm games, music visualizers, or adaptive audio systems using the streaming API.
How Audio Analysis Works
Analysis is a pipeline: each stage reads what the stage before it produced, which is why some features depend on others. This is the short version; MIR Overview draws the whole map and explains each feature with its own demo.
Stage 1: spectral analysis. Raw audio is a waveform, a list of amplitude values called samples, with the sample rate telling you how many make up one second (Audio Basics covers samples, decibels, and frequency from zero). The Short-Time Fourier Transform (STFT) cuts the waveform into short overlapping windows and measures which frequencies are present in each one, producing a spectrogram: a map of frequency against time. Three numbers set it up. The sample rate (44,100 Hz for CD audio) caps the highest frequency at half its value, 22,050 Hz, the Nyquist limit. The FFT size (n_fft, commonly 2048) is the samples per window: larger windows separate pitches more finely but blur timing. The hop length (hop_length, commonly 512) is how far the window moves each step: smaller hops give more time detail at more computation. Spectrogram and STFT explains the window function.
Stage 2: feature extraction. From the spectrogram, libsonare computes features, each capturing one aspect of the sound. Chroma says how much of each of the 12 notes (C, C#, ... B) is sounding, ignoring octave; it is the basis of key and chord analysis. The onset envelope peaks where new notes or hits begin and drives tempo and beat tracking. The mel spectrogram spaces the frequency axis the way hearing does, and MFCCs squeeze it into a compact summary of timbre. CQT / VQT give one frequency bin per semitone instead of equal-width bins. These are useful on their own (a mel spectrogram is the usual input to an ML model) and they feed the next stage.
Stage 3: musical analysis. The results most people want are built on top of the features: key detection and chord recognition from chroma, BPM and beat tracking from the onset envelope, section detection (intro, verse, chorus) from timbre and harmony, and pitch tracking (the melody's F0) from the waveform itself. Because every stage is exposed, you can take a low-level feature for your own analysis or plug it into an ML pipeline instead of stopping at the answer.
Room Acoustics and Inverse Features
Some analysis is about the recording space rather than the song. An impulse response (IR) is the recording of a room's answer to a single click, so it contains the room and nothing else. Reverberation falls off at a measurable rate, so from an IR, or less precisely from ordinary audio that has decaying tails, libsonare can describe the room.
| You have | You get | Read |
|---|---|---|
| A clean impulse response | Direct measurements of how long the room rings (RT60, EDT) and how clear it sounds (clarity, definition) | Room Acoustics |
| Ordinary audio | Estimated room cues with a confidence value, or an equivalent room: a practical model (volume, dimensions, absorption) inferred from the sound | Room Acoustics |
| A room described as a box (length, width, height) | A synthesized room impulse response, or a room morph: a target room's character applied to your audio as an effect (not dereverberation), in mono or stereo | Room Acoustics |
Room-acoustic abbreviations
RT60 is the time for reverberation to fall by 60 dB. EDT is early decay time, often closer to perceived reverberance. C50 / C80 are speech/music clarity metrics, and D50 is the fraction of energy arriving in the first 50 ms. The equivalent-room estimate also reports DRR, the direct-to-reverberant ratio: how loud the direct sound is against the reverb (Source Distance). For now, read these as numbers that describe how a room rings.
libsonare also exposes inverse helpers for debugging and ML workflows: once you have a mel spectrogram or MFCC, these helpers run the transform in reverse to recover an approximate spectrum or a rough preview of what that feature "sounds like" — useful for sanity-checking an ML pipeline. They are not restoration tools; they are lossy previews. See Inverse Features.
Audio Effects
libsonare also provides audio processing capabilities that operate on the spectral representation:
- HPSS (Harmonic-Percussive Source Separation) — Separates audio into harmonic (tonal) and percussive (rhythmic) components using median filtering on the spectrogram. Often used as a pre-processing step to improve analysis accuracy; Source Separation covers it with the other separation methods.
- Time Stretch / Pitch Shift — Changes tempo or transposes pitch by combining phase-vocoder processing (rebuilding the signal from short overlapping spectral frames) and resampling.
- Editing DSP — Pitch correction to a target MIDI note, note-region stretch, and voice-change pitch/formant controls (Editing DSP).
- Creative FX / inserts — Sound-design DSP such as reverb, chorus, flanger, phaser, and stereo delay, available when the build has creative FX enabled; Effects Inserts lists every insert and its parameters.
Real-Time Streaming
The StreamAnalyzer runs the same pipeline on audio chunk by chunk with low latency.
It produces per-frame features for real-time visualization or live analysis, along with an updating estimate of BPM, key, current chord, and chord progression (the chord heard most often in each bar).
In the browser, it integrates with the Web Audio API's AudioWorklet for real-time processing. Realtime and Streaming covers both.
Mixing and Routing
The libsonare mixing engine provides channel strips, pan modes, stereo width controls, sends, FX buses, VCA groups, scene presets, automation lanes, goniometer/true-peak metering, and offline stereo rendering. Use the Mixing Engine guide when you need a persistent mixer scene rather than a one-shot analysis or mastering render.
Mixer terms in one place
A send taps a copy of a channel to a shared effect; a bus is a sub-mix several channels feed into; a VCA group is one fader that controls the level of many channels at once. A goniometer visualizes the stereo image, and true-peak metering catches peaks that appear only between samples.
Platform Support
| Platform | Interface | Use case |
|---|---|---|
| Browser | JavaScript/TypeScript (WebAssembly) | Web apps, client-side analysis |
| Node.js | JavaScript/TypeScript (WebAssembly) | Server-side processing |
| Node.js | JavaScript/TypeScript (N-API native addon) | High-performance server-side processing |
| Python | Python (ctypes over the native C API) | Data science, scripting, librosa migration |
| Linux / macOS | C++ | Native applications, CLI tools |
| Any (via C API) | C | FFI integration with other languages |
The full WebAssembly and JavaScript bundle is about 8,500 KB, or 2,792 KB gzipped. A page that only analyzes can load the much smaller analysis-only entry instead; the WebAssembly Guide has the breakdown.
Relationship with librosa
librosa is the de facto standard Python library for audio analysis in the MIR community. libsonare provides similar MIR building blocks for environments where Python is not available, where native integration is useful, or where a single C++ core is preferred:
- Common DSP parameters and APIs are intentionally close to librosa where practical
- Core features such as STFT, Mel spectrogram, MFCC, and chroma are covered by reference tests against librosa
- Function names and API patterns are intentionally similar for easy migration
libsonare also goes beyond librosa's scope with features like chord recognition, section detection, timbre analysis, and real-time streaming — capabilities that are typically separate tools in the Python ecosystem.
See librosa Compatibility for detailed comparison and migration guides.
Project Status & Support
libsonare is an actively developed open-source project (Apache-2.0) maintained by a single developer. It is past 1.0 and releases carry semantic-version tags, but treat it the way you would any young library: read the release notes before upgrading, and pin versions in production.
- Releases & changes — every release is tagged on GitHub Releases with notes on what changed. The npm and PyPI packages track the same tags.
- Bug reports & questions — open an issue on the libsonare issue tracker. For problems with this documentation site or the demos, use the homepage repository.
- What "tested" means here — features covered by reference tests are listed in Implementation Validation, and known behavior differences between bindings are recorded in Binding Parity. Anything not listed there should be validated against your own material before you depend on it.
Next Steps
Take the path that matches what you want out of the next hour.
If you want to try it in the browser
Every demo runs the engine in your browser, and nothing is uploaded.
| You want to... | Try it | Then read |
|---|---|---|
| Read BPM, key, chords, and sections from a song | Music Analysis Studio | Getting Started |
| Master or mix a track | Mastering Studio, Mixing Studio | Mastering Assistant, Mixing Engine |
| Turn MIDI into sound | Synth Playground | Sound Sources |
| Turn a recording into notes, or edit and repair one | Piano Transcription | Audio to MIDI, Audio Repair Workflow |
Loudness, LUFS, and true peak
The mastering demos report these. Digital levels are measured in decibels below full scale (dBFS): 0 dBFS is the largest value a sample can hold, so ordinary levels are negative. Audio Basics covers the decibel from zero.
LUFS (Loudness Units relative to Full Scale) is a standardized estimate of program loudness used in streaming and broadcasting. Matching two tracks by integrated LUFS is a useful starting point for comparison, but differences in spectrum and dynamics can still make them sound unequally loud. True peak estimates the highest level of the reconstructed signal, including peaks between samples that a sample-peak meter misses. Those peaks can cause distortion during playback or encoding.
If you want to build something
- Getting Started — a first result in the runtime you already use, in one short program
- Learning Path — a route by goal through the ten subject areas, so you open the one guide written for what you are building
- Examples — small working patterns per runtime, to copy from rather than read
- Feature Map — which feature family lives where, and which binding exposes it
If you want to understand the engine
- Audio Basics — samples, sample rate, decibels, frequency and harmonics, from zero; the vocabulary every other page assumes
- Sound Sources — why an audio engine ships no samples, what each synthesis method costs, and why GS is its control vocabulary
- Music Analysis — what each analysis result means, which call produces it, and where its confidence comes from
- MIR Overview — the map of how every feature relates, with the Glossary behind each term
- Architecture — how the C++ core, the feature modules, and the bindings are layered, with Implementation Validation for what is checked against references