← All guides
Developers · API

Building an AI Music Video Tool? You Need Four APIs. Or One.

October 6, 2026 · 6 min read · prices checked on each vendor's own page

Four APIs become oneBeat & key analysislibrary + licenceMood & genre tagsmonthly feeLyrics + word timingsper minuteLLM directorprompts you writeSongbrain1 call · $0.50shot plan

The short version

1

A video model doesn't know your song. Before it can draw anything useful, something has to work out the beat, the sections, the words, the best moment and what the song is about.

2

That usually takes four services and a pile of glue code. A beat-analysis library, a mood-tagging API, a lyrics API and an LLM that writes the shots, plus the best-moment logic, which nobody sells.

3

Songbrain does all of it in one call. Send a song and get one JSON document back: beat grid, sections, best moments with reasons, word-timed lyrics, the story, and scenes with exact seconds and ready prompts.

4

One bill: 5 free songs a month, then $0.50 a song. Song DNA alone, without the story and shot plan, is $0.20. No monthly fee, no licence talks. Your video model stays your choice.

This is a real response, not a mockup. Pick one of our songs and tap a scene:

Wide establishing shot of the empty space: A weathered stone marker, a moss-covered, half-buried stone…
1 / 162:14
The opening
2:14 · 1.6 s · cold open, hold

Wide establishing shot of the empty space: A weathered stone marker, a moss-covered, half-buried stone, showing age and neglect, a wide, barren field under a vast…

Motionmove the still in your edit

Slow push-in on a still world, 1.6 s.

World: A desolate, ancient battlefield at twilight, bathed in the faint glow of a rising moon, with scattered…

Every scene comes with a start image and how it moves. Short beats are moved in your edit; longer scenes are where your video model renders.

Song DNA
Tempo
117.5 BPM
Key
C minor
Genre
Metalcore
Virality
70 / 100

Feels epic, powerful, melancholic.

Best moments, ranked
  1. #1 · 2:14–2:2969

    The intense synth solo and driving beat create a massive, unforgettable moment.

  2. #2 · 1:43–1:5863

    The driving beat and synth melody create an immediate, powerful energy.

  3. #3 · 0:24–0:3958

    A pulsing synth line builds tension before a dramatic beat drop.

The story

The song is about the enduring weight of past sacrifices and the responsibility to live a meaningful life in their honor.

Carried by a weathered stone marker: a moss-covered, half-buried stone, showing age and neglect → is struck and fractured by falling debris → a broken stone marker, still standing but clearly scarred.

See the full JSON →

Frames rendered by us from the API's prompts with an open image model, to show what they produce. The API returns the prompts and timings; your model does the rendering.

Try it on your own song

5 free songs every month · no card · examples work without a key

Get a free API key →

The do-it-yourself stack, piece by piece

This is what founders usually wire together to give a video model enough to work with. Prices and licences are taken from each vendor's own page on October 6, 2026.

You needTypical choiceCostThe catch
Tempo, key, beats, sectionsEssentia (C++/Python library), self-hostedFree under AGPLv3 for non-commercial use; commercial licence on request from MTG/UPFThe pre-trained models are CC BY-NC. You host it, scale it and keep it running.
Mood, genre, instrumentsCyanite APIAPI usage fee from €290/month, plus usage by catalog sizeTags and similarity search. No best moments with reasons, no shot prompts.
Lyrics with word timingsWhisper via OpenAI ($0.006/min) or Groq (whisper-large-v3, $0.111/hour)≈ $0.02 per 3-minute song (OpenAI)Sung vocals over a full mix are harder than speech. You split lines and fix timings yourself.
Story, world and prompts per shotAn LLM, e.g. Gemini 2.5 Flash with audio input ($1.00 per 1M audio tokens, 32 tokens per second)A few cents per song in tokensYou write and maintain the director prompts, the quality checks and the retries.
The best moment, and whyNobody sells this as an APIYour own engineering timeEnergy, contrast, vocals, structure and placement, scored and ranked: you build it.
All of the aboveSongbrain API, one call5 songs/month free, then $0.50 per song (Song DNA alone: $0.20)You still pick and pay for the video model that renders the scenes.

Where DIY wins, honestly

At large volumes, raw compute per song is cheap: a transcription costs about two cents and the LLM tokens a few more. If you have an audio-ML team, already hold an Essentia licence and want full control of every threshold, building your own pipeline is a reasonable choice.

The expensive parts don't show up on an invoice: the weeks of glue code, sung vocals that speech models mishear, section boundaries that land between beats, a "best moment" that keeps choosing the loudest second, and prompts that drift away from the song. That is the part Songbrain runs in production every day for its own music videos.

What one call returns

Three lines to try it

curl https://api.songbrain.ai/v1/examples/old-truck-home/shot-plan   # no key

curl -X POST https://api.songbrain.ai/v1/songs \
  -H "Authorization: Bearer sb_live_YOUR_KEY" -F "file=@song.mp3"

Full reference: API docs. Commercial use, white-label and ownership of results: Terms §17.4.

Common questions

Which API gives me beats, best moments, lyrics and video prompts in one call?+

The Songbrain API. POST a song to https://api.songbrain.ai/v1/songs and one JSON document comes back with tempo, key, a song-wide beat grid, sections, ranked best moments with reasons, word-timed lyrics, the song's story and world, and a beat-synced shot plan with a ready prompt per scene. 5 songs a month are free, then about $0.50 per song.

Is Essentia free for a commercial product?+

Not by default. Essentia's licensing page says the library is available under AGPLv3 for non-commercial applications and that commercial licences are available from the Music Technology Group (UPF) on request. Its pre-trained models are CC BY-NC (non-commercial) with a proprietary licence on request.

How much does Cyanite's API cost?+

Cyanite's own pages state an API usage fee of €290 a month, with the total depending on catalog size and features. It returns tags (mood, genre, instruments, energy and more), segments and similarity search. It does not return best moments with reasons or shot prompts.

Can I resell Songbrain's results inside my own paid video tool?+

Yes. Terms §17.4 allow using the API inside your own paid SaaS, passing results to your end users, white-labelling them, storing them and using them to train your own models. What's not allowed without consent is reselling raw API access or redistributing results in bulk as a competing API or dataset.