keydetect / beatdetect — buildable FFmpeg 8.1.2 package

Everything needed to build a tiny Windows ffmpeg.exe containing only the keydetect and beatdetect audio filters, edit those filters, and rebuild in about twenty seconds.

Nothing else from FFmpeg is compiled in. The result is a single static executable with no DLLs and no installer.

---

What you need

Docker Desktop for Windows. That's the only prerequisite — no compiler, no MSYS2, no Python, no FFmpeg source. Everything else is fetched or built inside the container.

https://docs.docker.com/desktop/install/windows-install/

Start Docker Desktop and wait until it reports Engine running before building.

---

Quick start

build.cmd          first build   — 5-10 min (downloads source, builds toolchain)
test.cmd           run the checks — verifies key and tempo detection

The binary lands in out\ffmpeg.exe.

out\ffmpeg.exe -hide_banner -nostats -i song.flac -af keydetect -f null -
out\ffmpeg.exe -hide_banner -nostats -i song.flac -af beatdetect -f null -
out\ffmpeg.exe -hide_banner -nostats -i song.flac -af keydetect,beatdetect -f null -

Reads flac, wav, mp3, m4a/aac, ogg/vorbis, opus, wavpack, aiff, ac3, mkv.

---

The iteration loop

This is the point of the package:

1. Edit src\af_keydetect.c (or src\af_beatdetect.c) in any editor
2. Run rebuild.cmd — about 20 seconds, recompiles only what changed
3. Run test.cmd, or point out\ffmpeg.exe at your own audio

The FFmpeg source tree lives in a Docker volume and is not re-downloaded or re-configured between rebuilds. If a rebuild fails, it's a compile error in your edit — the message will name the file and line.

---

What's in here

build.cmd            full build; also `build.cmd reconfigure` to re-run configure
rebuild.cmd          fast rebuild after editing a filter source
test.cmd             generate the synthetic corpus and check the binary
clean.cmd            drop the cached source tree; `clean.cmd all` also drops the image
docker\Dockerfile    the mingw-w64 cross-compiler toolchain
docker\build.sh      what actually runs inside the container
src\af_keydetect.c   <- edit these
src\af_beatdetect.c  <-
tests\gen_audio.py   synthesises audio with known keys and tempos
tests\run_tests.ps1  asserts the binary's output against that known truth
out\ffmpeg.exe       the build product
.forgejo\workflows\  CI: builds and publishes a release when a tag is pushed

---

CI and releases

The Forgejo workflow does not build on ordinary pushes. Pushing a tag of the form r* is the release act:

git tag r5
git push origin r5

That builds on an ubuntu-latest runner with the same docker/build.sh as the local flow, bakes NoMercy-keydetect-<tag> into the binary's version string, and publishes a Forgejo Release carrying ffmpeg-8.1.2-NoMercy-keydetect-<tag>.zip plus a SHA256SUMS.txt. The exe hash in that file is the build's identity — it is what the evaluation harness records, so hand the zip and the sums file over together.

Run workflow (workflow_dispatch) in the Actions tab builds without releasing and leaves the binary as the ffmpeg-win64 artifact of the run. The FFmpeg source tree persists in the runner's cache, so builds after the first are incremental; editing the configure block in docker/build.sh invalidates that cache by design.

Changing the build itself

Compiler flags, enabled decoders and demuxers, and the FFmpeg version all live in docker\build.sh. After editing anything in the ./configure block, run build.cmd reconfigure so the change takes effect — a plain rebuild.cmd reuses the existing configuration.

To tag your own builds, set BUILD_TAG before building; it becomes part of ffmpeg -version:

set BUILD_TAG=my-experiment
rebuild.cmd

---

The test corpus

test.cmd synthesises its own audio, so the expected answers are known exactly rather than argued about. Included:

what it checks	how
basic key detection	major and minor progressions in several keys
relative-key traps	progressions where the pitch collection alone is ambiguous
gain invariance	the identical file at two levels must give the identical answer
percussion robustness	75% and 87.5% unpitched content over a known progression
monophonic material	a melody with no chords at all
detuned material	the same progression at A=432 Hz
stereo handling	a stereo copy of a mono case
tempo accuracy	drum tracks at exactly 90 / 100 / 128 / 140 BPM

Two of the key cases are deliberate reproductions of published failure modes: t6a_hello_fminor is an i–bIII–bVII–bVI minor loop that profile-correlation methods classically call the relative major, and t6b_faded_ebminor is the same shape in another key.

Adding your own case is two lines in tests\gen_audio.py plus one entry in the table at the top of tests\run_tests.ps1.

---

keydetect output

lavfi.keydetect.key=Fm
lavfi.keydetect.key_confidence=0.774        Pearson r of the winning key
lavfi.keydetect.key_r_leader=0.811          r of the correlation leader
lavfi.keydetect.key_alt=G#                  runner-up
lavfi.keydetect.key_alt_confidence=0.811
lavfi.keydetect.key_margin=0.037            gap between the top two
lavfi.keydetect.key_candidates=Fm:0.774,G#:0.811,A#m:0.612
lavfi.keydetect.key_reranked=1              chord evidence overrode the chroma
lavfi.keydetect.tonalness=0.523             how pitched the material is
lavfi.keydetect.chord_coherence=1.114       how well the chords fit any key
lavfi.keydetect.bass_tonic=A                strongest pitch class, E1-E3
lavfi.keydetect.bass_chroma=0.574,0.000,... the whole bass band, C first
lavfi.keydetect.chord=Fm                    current stable chord
lavfi.keydetect.chords=Fm-G#-D#-C#-...      progression so far

All of these are also per-frame metadata under the same names. One field is emitted only in the final block, because it costs a correlation pass:

lavfi.keydetect.key_profiles=krumhansl:Am:0.836,temperley:C:0.880,shaath:Am:0.821,edma:Am:0.826

That is every profile's answer off the same accumulated chroma, so a four-way profile comparison costs one run instead of four. Profiles disagreeing is itself a warning about the track.

Reading the confidence fields

key_confidence alone answers almost nothing. Three fields answer three different questions, and a result is only trustworthy when all three agree:

field	question	bad value looks like
tonalness	is there pitched material here at all	low on percussive or sparse tracks
chord_coherence	does that material form a key	at or below 0 on chromatic riffs
key_margin	is the tonal centre within that key decided	near 0 on relative-key ties

r cannot substitute for any of them. It rises on every track once unpitched content stops diluting the chroma, and it rises most on the tracks with the most unpitched content to remove — so the tracks where the key question is least well posed are the ones whose r improves the most. A percussive track with no findable key can report a perfectly healthy r. In the synthetic corpus t2_noise_dilution scores the highest r of any case (0.860) at tonalness 0.191 and chord_coherence 0.000.

Note also that when key_reranked=1, key_confidence is the challenger's correlation, which is by construction below the leader's. Use key_r_leader when comparing r across tracks.

Options

option	default	meaning
profile	shaath	key profile: krumhansl, temperley, shaath, edma
window_ms	190	analysis window length
hop_ms	50	analysis hop
tuning_hz	440	reference A4
detect_tuning	1	estimate tuning deviation from the signal
threshold	0.55	minimum chord template similarity
silence_threshold	-60	silence gate in dB
smooth_ms	1000	chord score smoothing
min_chord_ms	400	minimum chord duration
compression	100	chroma log-compression factor
salience	1.0	weight windows by how tonal they are (0 disables)
rerank	1	let chord evidence break near ties in the key decision
rerank_margin	0.12	outer bound on which keys are worth scoring at all
rerank_gap	0.05	chord score the challenger must win by at zero deficit
rerank_slope	0.5	extra chord score demanded per unit of correlation deficit
rerank_floor	0.5	absolute chord score a challenger must reach to override at all
mode_balance	1	equalise the function weights between a key and its relative
transition	1.0	weight of cadence transitions in the chord score
tonic_bonus	1.0	weight of the first/last tonic-chord bonuses in the chord score
bass	1	accumulate the bass-register chroma (metadata only)

A challenger key must beat the leader's chord score by rerank_gap + rerank_slope * (r_leader - r_challenger), so a photo finish can be settled by the chords while a confident correlation is left alone. rerank_slope=0 restores a flat gap inside a hard window.

rerank_floor is the r4 change: on top of winning the comparison, the challenger's chord score must be good in absolute terms. Winning a comparison between two poor readings says only that one is less bad than the other — that is what sparse, riff-based material produces, and it is exactly when the chroma correlation is the better witness. Corpus measurement put wrong overrides at 0.34 and correct ones at 0.78+; the 0.5 default sits between the populations. The floor only gates the override — failing it leaves the correlation answer standing, it never withholds a key.

tonic_bonus exposes what was previously hardcoded: the first/last-chord tonic bonuses (0.15/0.20). The last committed chord of a fade-out is an accident of where the fade crossed the silence gate, not a cadence, and one measured wrong override traces to exactly that +0.20. It stays at 1.0 (r3 behaviour) until the harness says otherwise — sweeping tonic_bonus=0 vs 0.5 vs 1 is the intended experiment.

Example A/B:

out\ffmpeg.exe -i song.flac -af keydetect=rerank=0 -f null -
out\ffmpeg.exe -i song.flac -af keydetect=profile=edma -f null -
out\ffmpeg.exe -i song.flac -af keydetect=salience=0:compression=50 -f null -
out\ffmpeg.exe -i song.flac -af keydetect=tonic_bonus=0 -f null -

Every r3/r4 scoring change is separately switchable, so each one can be attributed on its own. This reproduces the r2 key decision exactly:

out\ffmpeg.exe -i song.flac -f null - \
  -af keydetect=mode_balance=0:transition=0:rerank_margin=0.06:rerank_slope=0:rerank_floor=0

and this reproduces r3 exactly:

out\ffmpeg.exe -i song.flac -f null - -af keydetect=rerank_floor=0

Turning the changes back on one at a time isolates each one:

arm	adds
mode_balance=0:transition=0:rerank_margin=0.06:rerank_slope=0:rerank_floor=0	r2 baseline
transition=0:rerank_margin=0.06:rerank_slope=0:rerank_floor=0	+ relative-mode balance
rerank_margin=0.06:rerank_slope=0:rerank_floor=0	+ cadence transitions
rerank_floor=0	+ graded rerank requirement (= r3)
(defaults)	+ absolute override floor

Note that the pre-r4 arms all carry rerank_floor=0: the floor postdates them, so reproducing their decisions requires switching it off.

The new metadata fields are emitted in every arm and never affect the key decision, so they can be collected in the same runs.

Full list at any time: out\ffmpeg.exe -hide_banner -h filter=keydetect

---

beatdetect output

lavfi.beatdetect.bpm=128.04

Options: window_size (2048), hop_size (512), peak_threshold (0.3).

On material with no percussion it can lock to a harmonic of the true pulse — a 30 BPM chord cycle read as 120 — which is ordinary tempo-octave behaviour rather than a fault.

---

Troubleshooting

"docker was not found on PATH" — install Docker Desktop, then open a new command prompt so it picks up the updated PATH.

"Docker is installed but not running" — start Docker Desktop and wait for Engine running.

Download or checksum failure — the build verifies the FFmpeg tarball's SHA-256. A mismatch means a corrupt download; run clean.cmd and retry.

A rebuild fails after your edit — it's a compile error; the output names the file and line. clean.cmd is not needed for this.

Starting completely over — clean.cmd all, then build.cmd.

A warning during compilation — the build prints one:

af_beatdetect.c:321: warning: variable 'candidates' set but not used

It is pre-existing in the beatdetect source and harmless — the build succeeds and the filter works correctly. Left unpatched deliberately, so src\ stays a faithful copy of the upstream filter sources.

---

How the filters get into FFmpeg

For reference, docker\build.sh does three things to a stock FFmpeg tree:

1. adds extern const FFFilter ff_af_keydetect; to libavfilter/allfilters.c — configure derives its filter list by scanning this file, so the extern is what makes --enable-filter=keydetect work at all
2. adds OBJS-$(CONFIG_KEYDETECT_FILTER) += af_keydetect.o to libavfilter/Makefile
3. adds keydetect_filter_deps="lm" to configure

Same three for beatdetect. All are idempotent, so re-running is safe.