← All guides
Developers · API comparison

Beat detection and song structure analysis API: the options compared (2026)

October 17, 2026 · 7 min read · every fact checked on the provider's own site on October 17, 2026

Beats, downbeats and sections on one timelineABACtimeline.sectionstimeline.beatstimeline.downbeats (first beat of each bar)4/4 · 117.5 BPM

The short version

1

"Beat detection" means three different things. A BPM number, a list of beat times, or beat times plus downbeats (where each bar starts). Only the third one lets you cut video or lights on the bar.

2

Most hosted APIs stop at tags. Cyanite, ReccoBeats and Sonoteller give you tempo as one number. For beat times and sections you need Music AI, Songbrain, or a model you host yourself.

3

The open options have a licence catch or an install catch. Essentia is AGPL for non-commercial use, librosa is ISC but has no downbeats or section labels, all-in-one is MIT but needs PyTorch, NATTEN, madmom and a GPU to be quick.

4

Songbrain returns the whole grid in one call. Every beat, every downbeat, sections with repeat letters, plus the best moments and word-timed lyrics. 5 songs a month free, then $0.50 a song.

See a real beat grid, no key needed

curl https://api.songbrain.ai/v1/examples/the-fallen-heroes?include=timeline

Get a free API key →

First, what you actually need back

Before you compare providers, decide which of these four you need. Half the "beat detection APIs" out there only do the first.

The table below marks each option on beats, downbeats and sections, then price and licence. The source link is the page the facts were read from on October 17, 2026. If a provider doesn't publish a price, the table says so instead of guessing.

The eight options at a glance

OptionBeatsDownbeatsSectionsPriceLicence
Essentia
C++/Python library, self-hosted
yesno built-in trackerno labelled sectionsfree for non-commercial use; commercial licence from MTG/UPFAGPLv3 (library), CC BY-NC-ND 4.0 (models)
librosa
Python library, self-hosted
yes (dynamic programming, Ellis 2007)nobuilding blocks only (recurrence matrix, agglomerative clustering), no verse/chorus labelsfreeISC
all-in-one (mir-aidj)
PyTorch model, self-hosted
yesyesyes, boundaries and labels (intro, verse, chorus, bridge, outro)free; your GPU timeMIT
Cyanite
Hosted GraphQL API
tempo as a tag, no beat timesnosecond-by-second emotion profile, no structural sectionsnot published; "Start for free" on the sitecommercial
Music AI (Moises)
Hosted modules you chain in workflows
yes (Transcription – Beats, $0.03/min)no separate module listedyes (Transcription – Sections, $0.04/min)pay as you go per minute of audio; Professional $25/month with 5% offcommercial
ReccoBeats
Hosted REST API
tempo only (one BPM number)nonofreecommercial, free tier
Sonoteller
Hosted API, on request
BPM as a tagno"sections tagging" endpoint and a "golden minute" (the song's highlights)not published; access via hello@sonoteller.aicommercial
Songbrain
Hosted REST API + MCP server
yes, every beat in seconds (timeline.beats)yes (timeline.downbeats) + time signatureyes, with repeat letters (A, B, A, C) and energy per section5 songs a month free, then $0.50 a song (25 credits)commercial; results are yours, white-label allowed

Each one in two sentences

Mature beat trackers, key, loudness, hundreds of descriptors. The licence page is explicit: AGPLv3 "for non-commercial applications", contact UPF for commercial conditions, and some algorithms pull in GPL/LGPL dependencies you have to clear yourself.

The easiest start and the most permissive licence. beat_track gives tempo and beat times. For bars and named sections you write the model yourself.

The research model from Kim & Nam (WASPAA 2023). Best open option for structure, but the install is real work: matching PyTorch, NATTEN (built from source on Windows), madmom from GitHub, and it reads audio through Demucs. The speed benchmark on the README uses an RTX 4090.

Built for catalogues: genre, mood, instruments, lyric theme and tempo tags, similarity search by reference track or text. Setting up an integration needs a public webhook URL. Pick it to organise thousands of tracks, not to cut one on the beat.

Also chords ($0.04/min), stems, lyrics with word alignment. Beats plus sections for a 3.5-minute song come to about $0.25. Excellent when you also need stems or chords, which Songbrain doesn't do.

POST /v1/analysis/audio-features takes an MP3, OGG, AIFF or WAV up to 5 MB and returns nine Spotify-style numbers. A fine Audio Features replacement, not a beat or structure API.

Calls itself an experimental engine in beta. Strong on lyrics (themes, moods, explicit flag, DDEX-style metadata). The site says the API needs your own music files; the YouTube demo is for demo use only.

One call also returns the best moments with reasons, word-timed lyrics, key and key changes, LUFS loudness, the song's story and a beat-synced shot plan. No stems, no chords. Example responses are public without a key.

Which one for which job

What the Songbrain grid looks like

This is the real response for one of our own songs. Times are seconds from the start of the file. The full example is public at the URL above.

{
  "timeline": {
    "time_signature": "4/4",
    "beats":     [0.51, 1.02, 1.53, 2.04, ...],   // 350 beats
    "downbeats": [0.51, 2.55, 4.59, ...],         // 88 bars
    "sections": [
      { "start_sec": 0.0,   "end_sec": 16.37, "label": "A", "energy": 0.749 },
      { "start_sec": 16.37, "end_sec": 37.62, "label": "B", "energy": 0.934 },
      { "start_sec": 37.62, "end_sec": 57.91, "label": "B", "energy": 0.934 },
      ...                                          // 12 sections
    ]
  },
  "best_moments": [ { "rank": 1, "start_sec": 134.54, "peak_sec": 137.04, "end_sec": 149.54,
                      "kind": "outro peak", "score": 69, "reason": "...", "sung": "..." } ]
}
import requests, time
API, H = "https://api.songbrain.ai/v1", {"Authorization": "Bearer sb_live_YOUR_KEY"}

song = requests.post(f"{API}/songs", headers=H, files={"file": open("track.mp3", "rb")}).json()
while (doc := requests.get(f"{API}/songs/{song['id']}?include=timeline,best_moments", headers=H).json())["status"] == "processing":
    time.sleep(10)

beats, bars, sections = doc["timeline"]["beats"], doc["timeline"]["downbeats"], doc["timeline"]["sections"]
chorus = max(sections, key=lambda s: s["energy"])

Field reference in the API docs. Limits: 30 s to 10 min, 100 MB, MP3/WAV/FLAC/M4A/AAC/OGG/AIFF. The same tools are available to AI assistants through the Songbrain MCP server.

Common questions

What is the best API for beat detection and song structure analysis?+

It depends on what you need back. For a hosted API that returns every beat, every downbeat and labelled sections in one call, Songbrain ($0.50 a song, 5 free a month) and Music AI (beats $0.03 and sections $0.04 per minute of audio, checked October 17, 2026) are the two that do it without you running models. If you can host your own, all-in-one (MIT) returns beats, downbeats and verse/chorus labels, and librosa (ISC) gives you beats and tempo but no downbeats or section labels.

Which free library does beat detection?+

librosa (ISC licence) has a dynamic-programming beat tracker that returns tempo and beat times. Essentia has beat trackers too, but its licence page says AGPLv3 applies to non-commercial use and commercial use needs a licence from MTG/UPF. all-in-one is MIT and returns beats, downbeats and sections, but needs PyTorch, NATTEN and madmom installed.

Which APIs return downbeats, not just beats?+

Of the eight compared here, downbeats (the first beat of every bar) come from all-in-one (self-hosted) and Songbrain (timeline.downbeats). Music AI's pricing page lists a Beats module and a Sections module but no separate downbeat module (checked October 17, 2026). librosa and ReccoBeats don't return downbeats.

Is ReccoBeats a beat detection API?+

No. ReccoBeats' audio feature extraction endpoint returns nine Spotify-style numbers per file (acousticness, danceability, energy, instrumentalness, liveness, loudness, speechiness, tempo, valence). Tempo is one number for the whole song; there are no beat times, downbeats or sections. It is free and the upload limit is 5 MB.

Does Songbrain return the song structure?+

Yes. timeline.sections lists every section with start and end seconds, a repeat letter (A, B, A, C …) so you can see which parts are the same, and a relative energy value. Next to it come timeline.beats, timeline.downbeats, the time signature and best_moments, the ranked windows that work best in a short clip, with the reason each one works.