Voice and Formant
Changing a voice well means separating two things that naive pitch shifting smears together: the note being sung and the character of the voice.
That character lives in the formants.
This page grounds voice editing in libsonare's VoiceChanger and FormantWarp; for the vocabulary, read Editing Basics.
What a formant is
A formant is a resonance peak in the spectrum of a voice. The peaks come from resonances of the vocal tract (mouth, throat, and nasal cavity), so their frequencies change with the vowel being shaped and with the size of the speaker's vocal tract, but not with the note being sung.
Formants define two things:
| Formants affect | Example |
|---|---|
| Vowel identity | The difference between "ee" and "oh" |
| Perceived voice character | Whether a voice feels smaller, larger, brighter, or darker |
The important point is that formants stay in roughly the same frequency regions regardless of the note. A soprano and a bass singing the same vowel have very different pitches, but their formants sit in similar regions; a shorter vocal tract places them only somewhat higher, typically 15 to 20 percent for adult female against adult male voices.
Put differently: pitch carries which note, formants carry whose voice and which vowel. An instrument keeps its timbre as you play up and down the keyboard, and in the same way a singer keeps the character their vocal tract gives them across different notes. Because the two are independent, it is worth being able to move them separately.
The figure below draws this as a source and a filter. The vocal folds produce a comb of harmonics spaced F0 apart, the vocal tract acts as a filter whose envelope peaks are the formants, and what comes out is the same comb with each harmonic's height read off that envelope.
- harmonics (the comb)
- spectral envelope
Why pitch shifting alone sounds wrong
A plain pitch shift moves everything up, formants included. Shift a voice up several semitones and the formants rise too, shrinking the apparent size of the singer — the "chipmunk" effect. Shift down and you get an unnatural, oversized voice. The problem is that pitch and formants were never meant to move together.
Pitch and formant on separate controls
Good voice editing exposes the two independently:
- Change pitch alone → the same person sounds like they are singing a higher or lower note.
- Change formant alone → the voice sounds smaller/brighter or larger/darker at the same pitch.
- Change both → from subtle character tweaks to dramatic voice design.
To make a male voice read as female, for instance, raise the formants a little as well as the pitch. Leave the pitch where it is and lower the formants instead, and the same performance sounds fuller and calmer.
Having the two on separate controls is what makes natural transposition (transpose, then put the formants back) and creative voice design (warp formants freely) both possible from one tool. As always, small moves stay natural; large moves are an effect.
formantFactor stacks on top of the pitch shift
In voiceChange the pitch shift runs first, and a pitch shift moves the whole spectrum — formants included. FormantWarp then works on that already-shifted signal, so formantFactor multiplies the formant displacement the transpose has already caused. formantFactor: 1.0 means "no extra formant move", not "formants preserved": { pitchSemitones: 5, formantFactor: 1.0 } is a plain chipmunk shift, with the formants sitting 25/12 ≈ 1.33× too high.
To transpose while holding the formants where they were, pass the reciprocal of the pitch ratio:
const pitchSemitones = 5;
const formantFactor = 2 ** (-pitchSemitones / 12); // 0.749 for +5 semitones
const natural = voiceChange(vocal, sampleRate, { pitchSemitones, formantFactor });Anything above that reciprocal brightens and shrinks the voice; anything below it darkens and enlarges it.
Pass formantMode: 'absolute' to skip the arithmetic: formantFactor is then the formant shift relative to the input whatever the pitch shift, so 1 holds the formants still. The same mode exists on the realtime voice changer, where it is fixed at creation; see Realtime Voice Changer.
Compare this with the pitch shift above: there the harmonic comb — the evenly spaced stack of overtones above the sung note — slides and the formants ride along; here the comb and the fundamental (the lowest of those, the one that sets the note) hold still while only the envelope moves, meaning the formant shape riding on top of that comb. Those are the two independent axes voiceChange exposes. The figure below puts the cases side by side, with the input's F1 and F2 marked in every row.
- harmonics (the comb)
- spectral envelope
- input formant position
How libsonare changes a voice
voiceChange takes pitchSemitones and formantFactor as separate arguments and applies them as two stages in that order. The pitch stage reuses the phase-vocoder/resampling backend from Time Stretch and Pitch Shift, and its resampling step scales the whole spectrum, envelope and all. The formant stage, FormantWarp (FormantWarpConfig), then does LPC (linear prediction, a model that captures the spectral envelope) analysis of the shifted signal and resamples that envelope along the frequency axis by formantFactor, leaving the harmonic structure — and therefore the note — untouched. A factor of exactly 1.0 skips the stage entirely, so the LPC round trip costs nothing on a pitch-only change. Lowering the factor moves the envelope down (larger, darker); raising it moves the envelope up (smaller, brighter). Quiet voiced input is warped like loud input; only an effectively silent or non-finite frame passes through unwarped. It operates on decoded mono samples like the other editing helpers.
Related: Editing Basics, Time Stretch and Pitch Shift, Pitch Correction, Editing DSP