C++ Analysis API
Feature extraction functions and analysis result types for the libsonare C++ interface; see C++ API Reference for the rest of the C++ surface.
Feature Extraction
MelSpectrogram Medium
MelConfig config;
config.n_mels = 128;
config.n_fft = 2048;
config.hop_length = 512;
auto mel = MelSpectrogram::compute(audio, config);
// Power spectrum [n_mels x n_frames]
auto power = mel.power();
// Convert to dB
auto db = mel.to_db();
// MFCC
auto mfcc = mel.mfcc(13); // 13 coefficientsChroma Medium
ChromaConfig config;
config.n_chroma = 12;
auto chroma = Chroma::compute(audio, config);
// Features [12 x n_frames]
auto features = chroma.features();
// Mean energy per pitch class
auto energy = chroma.mean_energy();
// Move the feature matrix out instead of copying it; the Chroma is left without features
auto matrix = chroma.release_features();Spectral Features
// Per-frame spectral centroid (Hz)
std::vector<float> spectral_centroid(const Spectrogram& spec, int sr);
// Per-frame spectral bandwidth (Hz)
std::vector<float> spectral_bandwidth(const Spectrogram& spec, int sr);
// Per-frame spectral rolloff (Hz)
std::vector<float> spectral_rolloff(const Spectrogram& spec, int sr, float roll_percent = 0.85f);
// Per-frame spectral flatness
std::vector<float> spectral_flatness(const Spectrogram& spec);
// Zero crossing rate
std::vector<float> zero_crossing_rate(const Audio& audio, int frame_length, int hop_length);
// RMS energy
std::vector<float> rms_energy(const Audio& audio, int frame_length, int hop_length);
// Spectral contrast (difference between peaks and valleys in frequency bands)
std::vector<float> spectral_contrast(const Spectrogram& spec, int sr, int n_bands = 6,
float fmin = 200.0f, float quantile = 0.02f);Pitch Tracking Medium
PitchConfig config;
config.frame_length = 2048;
config.hop_length = 512;
config.fmin = 65.0f; // C2
config.fmax = 2093.0f; // C7
config.threshold = 0.1f;
// YIN algorithm
PitchResult yin = yin_track(audio, config);
// pYIN algorithm (probabilistic YIN with HMM smoothing)
PitchResult pyin_result = pyin(audio, config);
// Access results
float median = pyin_result.median_f0();
float mean = pyin_result.mean_f0();
const std::vector<float>& f0 = pyin_result.f0;
const std::vector<bool>& voiced = pyin_result.voiced_flag;CQT / VQT Medium
Constant-Q Transform and Variable-Q Transform for music analysis.
CqtConfig config;
config.fmin = 32.7f; // C1
config.n_bins = 84; // 7 octaves
config.bins_per_octave = 12; // Semitone resolution
auto cqt_result = cqt(audio, config);
// Access magnitude [n_bins x n_frames]
auto mag = cqt_result.magnitude();
auto power = cqt_result.power();
// Variable-Q Transform (with variable Q factor)
VqtConfig vqt_config;
vqt_config.gamma = 0.0f; // 0 = CQT behavior
auto vqt_result = vqt(audio, vqt_config);Thread Safety
CqtResult and VqtResult objects use lazy initialization for cached results. They are not thread-safe for concurrent access. Create separate copies for multi-threaded use.
Griffin-Lim
Griffin-Lim is an iterative phase-reconstruction algorithm: it recovers a plausible waveform from a magnitude-only spectrum (which stores no phase) by transforming to the time domain and back over successive passes, refining the phase estimate each time. A phase vocoder instead tracks and manipulates the STFT phase directly.
Deprecated Functions
The inverse transform functions icqt() and ivqt() are deprecated in the current C++ headers. Prefer Griffin-Lim or phase-vocoder based reconstruction paths for new code.
// Deprecated - do not use in new code
[[deprecated("Use Griffin-Lim or phase vocoder for better reconstruction quality")]]
Audio icqt(const CqtResult& cqt_result, int length = 0);
[[deprecated("Use griffinlim_vqt or phase vocoder for better reconstruction quality")]]
Audio ivqt(const VqtResult& vqt_result, int length = 0);Migration: griffinlim_cqt and griffinlim_vqt are declared in the same <feature/cqt.h> / <feature/vqt.h> headers as cqt() and vqt(), so no extra include is needed. For preview audio reconstruction, use these Griffin-Lim paths, or keep phase information in your own STFT-domain pipeline when quality matters.
const auto& cqt_magnitude = cqt_result.magnitude();
auto reconstructed = griffinlim_cqt(cqt_magnitude.data(), cqt_result.n_bins(),
cqt_result.n_frames(), config,
cqt_result.sample_rate());
auto reconstructed_vqt = griffinlim_vqt(vqt_result, vqt_result.sample_rate());NNLS Chroma
NNLS chroma uses a configurable CQT hop length. The default is 512 samples; set it on NnlsChromaConfig::cqt when matching another frame grid.
NnlsChromaConfig nnls_config;
nnls_config.cqt.hop_length = 512;
nnls_config.enable_stft_blend = true;
auto nnls_result = nnls_chroma(audio, nnls_config);Each CQT bin is whitened against its neighbors within half an octave, so a held chord keeps its chord tones; there is no whitening-window setting on NnlsChromaConfig.
C ABI analysis additions
These are declared in sonare_c_features.h, sonare_c_types_functions.h and sonare_c.h; the struct versions and defaults they depend on are under C ABI versions.
| Function | Purpose |
|---|---|
sonare_detect_chords_ex(samples, length, sample_rate, &options, &result) | Chord detection with a SonareChordDetectionOptions (no _default(); set every field, see Versioned structs); the result carries struct_version and tuning, and is released with sonare_free_chord_analysis_result |
sonare_chord_functional_analysis(samples, length, sample_rate, &options, key_root, key_mode, &chords, &roman, &functions) | Detects chords, then labels each one relative to the key. chords is a SonareChordAnalysisResult; roman and functions are parallel SonareStringArrays with one entry per chord, released with sonare_free_string_array |
sonare_detect_key_with_tuning(...) | Key detection with a recording tuning (a fraction of a semitone in [-0.5, 0.5)) or tuning_auto to measure it; out_tuning receives the value used |
sonare_chord_functions(chords, count, key_root, key_mode, &roman, &functions) | Roman numeral and harmonic function of chords already in hand, as two SonareStringArrays of count entries released with sonare_free_string_array; no audio is analyzed |
sonare_chord_symbol_from_analysis(...), sonare_key_mode_from_analysis(mode, &out_mode) | Convert a detected chord or key into the fields of an arrangement annotation (SonareProjectChordSymbol, SonareProjectKeySegment) through the library's own reduction table; declared in sonare_c_project_annotate.h |
sonare_tuning_to_reference_hz, sonare_reference_hz_to_tuning | Convert between the analysis tuning unit and a reference in Hz |
sonare_scale_mask_for_mode(root, mode, &mask) | The 12-bit scale mask of a SonareMode, in the layout sonare_scale_quantize_midi and SonarePitchCorrectionConfig read |
sonare_mfcc_to_mel_checked_ex(mfcc, input_length, n_mfcc, n_frames, n_mels, lifter, &out) | Length-checked, lifter-aware MFCC-to-Mel inversion; lifter is the one the forward transform used (0 = none) |
sonare_mfcc_to_audio_checked_ex2(...) | Length-checked MFCC-to-audio inversion with htk and lifter |
SonareMelResult and SonareMfccResult carry the parameters of their forward transform (n_fft, fmin, fmax as applied, htk, is_db; MFCC also sample_rate, hop_length, n_mels, lifter), so a result is enough to invert it. sonare_analyze_json_ex takes a SonareMusicAnalyzeOptions whose tuning and tuning_auto fields select the tuning, and its JSON reports the value used under tuning.
Structural boundaries
BoundaryDetector finds section transitions from a checkerboard-kernel novelty curve over MFCC and chroma features. BoundaryConfig exposes the detector's parameters; the two thresholds do different jobs.
BoundaryConfig config;
config.threshold = 0.3f; // relative: fraction of the local novelty maximum
config.absolute_threshold = 0.005f; // floor on the raw response: did anything change at all?
config.reference_window = 60.0f; // one-sided seconds the relative threshold is measured over
BoundaryDetector detector(audio, config);
for (const Boundary& b : detector.boundaries()) {
// b.time (seconds), b.strength (novelty scaled by the whole-track maximum)
}threshold is measured against the largest raw novelty event within reference_window seconds of the peak, with a lone dominant event set aside (one no other event within reference_window comes within 30% of), so a dominant change further away than the window cannot gate a weaker one out. 0 makes each frame its own reference and disables threshold; a value at least the track length makes the reference whole-track. Because the reference is local, Boundary::strength (scaled by the whole-track maximum, a rank within the track) can be below threshold.
The C ABI is sonare_detect_boundaries(samples, length, sample_rate, &options, &out) with a SonareBoundaryOptions that must start from sonare_boundary_options_default(): zeroing it sets absolute_threshold to 0, which segments a stationary input anyway. Release the result with sonare_free_boundary_result.
Types
Key
struct Key {
PitchClass root; // C=0, Cs=1, ..., B=11
Mode mode; // Major, Minor, Dorian, Phrygian, Lydian, Mixolydian, Locrian
float confidence; // 0.0 - 1.0
std::string to_string() const; // "C major"
std::string to_short_string() const; // "C", "Am"
};
// Spell a key from raw ordinals, as to_string / to_short_string do ("" when root is not 0-11)
std::string key_name(int root, bool minor, bool short_form);Key tuning is covered in Key detection. KeyConfig::tuning, ChordConfig::tuning and MusicAnalyzerConfig::tuning are a recording's offset from concert A440 in fractions of a semitone, in [-0.5, 0.5). Set auto_tuning = true on any of the three to measure it from the audio (measure_tuning(audio) in feature/pitch.h) instead; KeyAnalyzer::tuning(), ChordAnalyzer::tuning() and MusicAnalyzer::tuning() report the value used, and AnalysisResult::tuning carries it. tuning_to_reference_hz(tuning, a4 = 440) and reference_hz_to_tuning(hz, a4 = 440) in feature/chroma.h convert to and from the Hz a reference pitch is stated in.
Chord
Background: Chord recognition.
struct Chord {
PitchClass root;
ChordQuality quality; // Major, Minor, Dim, Aug, 7th, etc.
float start; // seconds
float end; // seconds
float confidence;
PitchClass bass; // Bass pitch class for inversion notation
std::string to_string() const; // "C", "Am", "G7"
};
// Spell a chord from raw root / quality ordinals ("N.C." for no chord; never throws)
std::string chord_symbol(int root, int quality);ChordAnalyzer::chord_to_roman_numeral(chord, key_root, mode) returns the Roman numeral of a chord in a key ("N.C." for an unknown chord), and ChordAnalyzer::chord_function(chord, key_root, mode) its harmonic function: "tonic", "subdominant", "dominant", "chromatic" for a root outside the key's scale, or "none" for an unknown chord. Every mode other than minor reads the major scale. Both are static and analyze no audio.
Section
struct Section {
SectionType type; // Intro, Verse, Chorus, etc.
float start;
float end;
float energy_level; // median frame RMS, normalised by the loudest section
float confidence;
std::string type_string() const;
float duration() const;
};AnalysisResult
struct AnalysisResult {
float bpm;
float bpm_confidence;
std::vector<BpmCandidateHypothesis> bpm_candidates;
Key key;
TimeSignature time_signature;
std::vector<TimeSignature> time_signature_candidates;
std::vector<Beat> beats;
std::vector<int> downbeat_indices; // Indices into beats, not one flag per beat
int downbeat_phase;
BeatObservations beat_observations; // Parallel onset, low-frequency and chord-change evidence
std::vector<float> beat_local_bpm; // Parallel to beats; empty unless compute_tempo_curve
std::vector<Chord> chords;
std::vector<std::string> chord_roman_numerals; // Parallel to chords, relative to key; empty for N.C.
std::vector<Section> sections;
Timbre timbre;
Dynamics dynamics;
RhythmFeatures rhythm;
MelodyContour melody;
std::string form; // "IABABCO"
float tuning; // fractions of a semitone the chromagrams were built with (given or measured)
};The two perceptual sub-structs are expanded below; RhythmFeatures and MelodyContour follow the same by-value pattern.
Timbre
Perceptual sound-color descriptors, each normalized to [0, 1].
struct Timbre {
float brightness; // high = bright / harsh
float warmth; // high = warm / full
float density; // high = rich / complex
float roughness; // high = rough / harsh
float complexity; // high = harmonically complex
};Dynamics
Loudness and dynamic-range descriptors (levels in dB).
struct Dynamics {
float dynamic_range_db; // dynamic range (dB)
float peak_db; // peak level (dB)
float rms_db; // RMS level (dB)
float crest_factor; // peak-to-RMS ratio
float loudness_range_db; // loudness range / LRA (dB)
bool is_compressed; // true if the audio appears heavily compressed
};