Music Video Prompt API

Songbrain API

Turn any song into a beat-synced, scene-by-scene AI video prompt. Songbrain is the prompt layer between a track and Sora, Runway, Kling, Veo, Higgsfield and Opus — so the render cuts land exactly on the drop.

One call returns the best moments, full Song DNA, beat-accurate section shifts, time-stamped lyrics, instrument cues and a virality score — the raw material every AI video tool needs and none of them have.

What you get

⏱️
Best-Moment Detection (Timestamps)
The exact 3–15s hooks with confidence scores and a 'why' — so auto-editors and clip tools start on the breakdown, not the intro.
🎤
Word-Accurate Lyric Transcription
Full transcription with karaoke-level word timestamps, aligned to the best moments — ready for synced captions and lyric-video overlays.
🎨
Visual Ideas from Song DNA
Concrete visual concepts derived from tempo, key, energy, mood, structure and instrumentation — palette, mood, camera energy per section.
🔥
Virality Score + Breakdown
A 0–100 score built from the song's DNA — hooks, energy, structure, lyrics — with a transparent score breakdown showing exactly what drives the number.
🎬
AI Video-Generation Prompt Ideas
Scene-by-scene prompt ideas timed to the track — raw material for Sora, Runway, Kling, Veo, Higgsfield, Pika and Luma.
🧭
Coming later: Visual-Performance Data by Sound Profile
Analyze a song and learn which video styles worked for songs like it.

Intercept every AI video & music tool

The AI tools render anything — but only as well as they're prompted. Songbrain is the prompt layer they're missing.

🎬AI video generators — plug Songbrain in front

Sora / Veo / Runway
Scene-by-scene, beat-timed text prompts so the render's cuts land on the song's drops
Kling / Pika / Luma / Hailuo
Start + end keyframe prompts synced to section shifts for consistent motion between shots
Higgsfield
Camera-motion prompts matched to the song's energy peaks and transients
Opus Clip & auto-editors
Best-moment timestamps so auto-cuts start on the hook — not the first 8 seconds
AI music-video builders
Turn a Suno track into a full shot list with synced lyrics in one call
Faceless & lyric-video tools
Per-line lyric timestamps for perfectly synced captions and karaoke overlays

🎨Short-form, reels & marketing

TikTok / Reels / Shorts tools
Best clip second + visual-energy match for every post
AI slideshow & montage tools
Sync transitions to beat drops and energy peaks automatically
Marketing-kit generators
Caption, hashtags, hook-text overlay and best post-time per clip
Reel reverse-engineering tools
POST a creator Reel URL → cross-referenced clip-timing, caption and visual fixes

📈Music intelligence & A&R

A&R screening
Score entire demo inboxes with a transparent 0–100 breakdown before human ears get involved
Artist dashboards
Enrich artist tools with best moments, Song DNA and score breakdowns per track
Creator-economy platforms
Match creators to songs by genre, mood and energy profile
AI music generators
Validate generated tracks for quality, virality and genre accuracy

🎵Distribution, playlists & music tech

Distributors & labels
Pre-screen tracks for viral potential before release
Playlist curation platforms
Auto-match tracks to playlists by genre, mood and virality
Recommendation engines
Use Song DNA vectors for similarity search and discovery
Streaming analytics
Enrich artist dashboards with deep song-level insights

From song to video prompt

A real example — a metalcore track, "Bleed the Horizon", from upload to a beat-timed prompt to a marketing kit.

// 1. Upload the metalcore track → full analysis
POST https://api.songbrain.ai/api/v1/upload (file: bleed_the_horizon.wav)
{
"title": "Bleed the Horizon",
"virality_score": 88,
"genre": { "primary": "Metal", "subgenre": "Metalcore" },
"song_dna": { "tempo": 182, "key": "Drop C", "energy": 0.96, "loudness_lufs": -6.1, "mood": "aggressive" },
"best_moments": [
{ "start": 48.6, "end": 60.2, "confidence": 0.95, "label": "breakdown", "why": "half-time chug + first scream" },
{ "start": 17.1, "end": 24.0, "confidence": 0.88, "label": "chorus hook" }
],
// beat-accurate section shifts, snapped to the transient
"section_shifts": [
{ "t": 17.04, "from": "verse", "to": "chorus" },
{ "t": 48.61, "from": "build", "to": "breakdown" }, // the drop
{ "t": 60.20, "from": "breakdown", "to": "outro" }
],
// lyrics with per-line timestamps, aligned to best moments
"lyrics": [
{ "start": 48.6, "end": 51.2, "line": "bury me beneath the horizon", "section": "breakdown" },
{ "start": 51.3, "end": 54.0, "line": "i am the storm you can't outrun" }
],
"instruments": ["distorted guitar", "double-kick drums", "bass", "harsh vocals"] // → visual cues
}
// 2. Turn the song into a beat-synced AI video prompt
POST /api/v1/jobs/{job_id}/video-prompt { "target": "kling", "aspect": "9:16" }
{
"shots": [
{
"t_start": 48.61, "t_end": 52.0, "beat": "downbeat of breakdown",
"prompt": "low-angle shot, rain-soaked concrete, a lone figure drops to their knees
as lightning cracks, slow-mo debris, desaturated teal + ember orange,
hard strobe cut on every kick drum",
"camera": "handheld push-in", "cut_on": "double-kick (~0.33s)",
"sync_to": "best_moments[0] breakdown", "lyric_overlay": "bury me beneath the horizon"
}, ...
],
"negative_prompt": "bright, cheerful, pastel, slow, static",
"ready_for": ["Sora", "Runway", "Kling", "Veo", "Higgsfield", "Pika", "Luma"]
}
// 3. Get the marketing kit for the clip
GET /api/v1/jobs/{job_id}/marketing-kit?clip=48.6
{
"caption": "the breakdown that broke my neck 💀",
"hashtags": ["metalcore", "breakdown", "djent", "fyp"],
"hook_text_overlay": "wait for the drop at 0:48",
"best_clip_sec": 48.6, "post_time": "Fri 21:00 EST"
}

Technical Specs

Format
JSON
Response time
< 60 seconds
Input
MP3, WAV, FLAC
Prompt targets
Sora · Runway · Kling · Veo
Protocol
REST API
Callbacks
Webhooks
Status: By request

Get API access now

We are onboarding selected developers and companies.

Why Songbrain API?

AI video tools — Sora, Runway, Kling, Veo, Higgsfield, Pika, Luma — can render almost anything. But a music video lives or dies on one thing: whether the cuts land on the song. Feed them a vague prompt and you get pretty footage that ignores the track. That gap is the whole product.

Songbrain turns a song into a beat-synced, scene-by-scene video prompt— every shot mapped to a section of the track, every cut timed to a transient, the drop getting the biggest visual beat. Under the prompt sits the analysis layer: best moments with a "why", full Song DNA, beat-accurate section shifts, lyrics with per-line timestamps, instrument cues for visuals and a virality score — all from a single call in under 60 seconds.

For Suno and AI artists that's the missing half of the workflow: they can generate the song in minutes, but turning it into a video that cuts on the beat was the hard part — until the prompt comes pre-built. Drop the JSON straight into your render pipeline and the timing is already solved.

Songbrain vs. alternatives

AcousticBrainz was discontinued in 2022. Spotify Audio Features API was deprecated in November 2024. ACRCloud does audio fingerprinting, not analysis. Essentia is a self-host library with no moments and no prompts. None of them turn a song into a video prompt.

Songbrain is the only API that goes from audio to a beat-synced AI video prompt — combining best-moment detection, beat-accurate section shifts, time-stamped lyrics, instrument-driven visual cues and virality prediction in a single hosted call. That makes it the prompt layer for AI video generators, TikTok and Reels automation, lyric-video tools, Opus-style auto-editors and any product that has to put a song on screen.