Video Studio
GitHub

Studio / Engines

Two engines

Remotion is the default. The HTML engine exists for one job: a narrated film where the voice sets the pace. Pick before you plan.

On this page

Which one, when

Remotion (studio/)HTML engine (skills/motion-studio)
Best forKinetic type, titles, UI demos, diagrams, data viz, explainers, captions, reels, templatesNarrated product films where every cut, caption and camera move lands on a spoken word
Written inReact and TypeScript, frame basedOne HTML composition, one function of time
ReviewingRemotion Studio preview, rendered stills and contact sheetsA live HTML player with the real mix, scrubbable; MP4 only on approval
Timing comes fromFrame numbers and a timeline file per videoVoice-over word times (Kokoro voice, Whisper timestamps) on a beat grid
SoundSfx component and built-in synthesised cuesA hits table, a composed music bed, ducked mix at -14 LUFS
Renderingremotion renderHeadless browser, subframe motion blur, piped into ffmpeg
Pick it whenIt is the defaultThe voice sets the pace of the film
  • Remotion (studio/): default. Kinetic type, UI demos, diagrams, data viz, explainers, captions, templates.
  • motion-studio HTML engine (skills/motion-studio): voice-over-driven product films where every cut lands on a spoken word, reviewed live in an HTML player.
  • brag / brag-slim (third-party, third_party/brag): a quick 15 to 25 s launch video of a project you just built, in one command, run from that project's folder. For directed, storyboarded work (a brief, chosen direction, reviewed scenes, house style) use remotion-studio instead.
  • After Effects / Blender: only when the shot needs compositing or real 3D; say so instead of faking it.

What the HTML engine contains

A project is created by skills/motion-studio/tools/new-project.mjs and holds the pieces below.

Composition

comp/ with engine.js, index.html, main.js, style.css. frame(t) sets every style from time alone.

Live player

A review bar over the stage: play, scrub, time readout, subtitle toggle, driven by the real mix.

Hits table

Every sound event is a row; the cue sheet in the composition is a one-to-one copy, and a sync check verifies each cue lands within 40 ms.

Sound pipeline

Voice-over, word times, a beat-grid timeline, a composed music bed and a ducked, loudness-normalised mix.

Render tools

Preview stills and contact sheets, frame-exact video with subframe motion blur, subtitles.

The hits table

The storyboard has a hits table (time | visual event | sound | level). The SFX cue sheet in comp/main.js is a one-to-one copy of it, with times expressed through S/E/L/W helpers so cues move with the edit. Nothing else adds sound effects. The renderer exports the cues to build/comp_timing.json; after the mix, node ~/.claude/skills/motion-review/tools/inspect.mjs sync <mp4> build/comp_timing.json checks every cue lands within ±40 ms and draws a waveform with the hits marked. A hit without a visual event, or a visual accent without a hit, is a storyboard bug: fix the table first, then the code.

Tools in every project

FileWhat it does
mix.mjsSoundtrack: music bed (music.mjs) + voice lines at their timeline times + SFX placed by their peak from the composition's cue sheet, voice-ducked, loudness-normalised. Needs build/comp_timing.json (render.mjs writes it).
music.mjsComposes the music bed in code (Web Audio OfflineAudioContext inside a headless browser) on the exact beat grid of build/timeline.json and writes audio/music.wav (48 kHz stereo). Arrangement comes from script.json "music": chords/roots per bar, intro scenes (soft kick only), breakdown scenes (no drums, dark pad), lift scene (riser before, impact on its downbeat, bright pad), endHit (impact on the last bar for the logo).
render.mjsRenders comp/index.html frame by frame (headless Edge/Chromium) into ffmpeg.
serve.mjsReview server: node tools/serve.mjs [port] → http://127.0.0.1:5199/comp/index.html Serves only the folders the page needs, with HTTP range support (audio scrubbing).
srt.mjsSubtitles from build/timeline.json: node tools/srt.mjs [field=text|de|...] [out=out/name_<field>.srt] Each cue runs from the line start to its end + 0.25 s (or the next line). Dash separators become commas.
timeline.mjsBuilds build/timeline.json: scenes on bar lines, every voice line snapped to the beat grid with authored gaps, word times absolute. Inputs: script.json, audio/vo/durations.json, audio/vo/words.json. Without voice files it estimates durations (draft mode) so design can start early.
vo.pyVoice-over: one Kokoro clip per line of script.json, trimmed and mastered (48 kHz mono). Word times come from words.py. Supports voice blends such as "am_michael:0.5+bm_george:0.5".
words.pyWord timestamps for every voice line (faster-whisper, word level) and a pronunciation check against the script. Writes audio/vo/words.json: {line_id: {"heard": str, "words": [{"w": str, "a": s, "b": s}]}}. Needs a Python with faster-whisper. Model: script.json voice.whisper (base.en for English, small for other languages). Never pass initial_prompt: it made the model hallucinate repeated text.

The contract

  • frame(t) sets every style from t alone. No CSS transitions, no timers, no state between frames.
  • window.__seek(t) awaits image decodes and draws t (renderer). window.__timing exposes {fps, duration, sfx, lines} for the mixer.
  • ?render=1 hides the review bar; ?subs=de turns on subtitles; ?t=12.5 opens at a time.
  • Kinetic type runs on one paused GSAP timeline (master) with a label per voice line; frame(t) calls master.time(t, true) first. Procedural parts (camera, spotlight, cursor, footage) are computed right after.

The live review loop

  1. node tools/serve.mjs (default port 5199) serves only the project folders the page needs, with HTTP range support for audio.
  2. Open http://127.0.0.1:5199/comp/index.html. Play / Space, scrub bar, time readout, subtitle toggle. The bar drives frame() from audio.currentTime of audio/mix.wav (falls back to music.wav, then a silent clock).
  3. The user reviews, asks for changes; you edit and they reload. Only the approved cut is rendered.
  4. For your own QA use stills: node tools/render.mjs preview 0,4.2,13.1,... writes shots/preview_*.png and shots/preview_sheet.png (3 columns). Pick each settled state plus mid-transition moments, and look at every sheet.

The stage is 1920×1080 and is scaled to fit the window with translate(-50%,-50%) scale(k) on an absolutely centred stage (grid centring overflows to the right for oversized children).

Rendering

node tools/render.mjs video out/work/v.mp4 blur=4 fps=60 [from-to] [subs=de]

  • Headless Edge/Chromium via playwright-core, screenshot per subframe piped as JPEG into ffmpeg (image2pipe, libx264 -crf 15 -preset slow -pix_fmt yuv420p).
  • 4 subframes at 60 fps for a 68 s film: about 12 minutes on a laptop. Run it in the background.
  • Fast moves (floods, wipes, drags) want 8 subframes: render that section with blur=8 <from>-<to> and the rest at blur=4, then join the parts, so render time grows only where needed (a shutter of about 0.7 of a frame is the target; per-frame shutter control is not in the template yet). Look at a fast-move frame pair zoomed in for ghost steps (gotchas.md).
  • WebGL in headless Edge: add --use-angle=d3d11 --ignore-gpu-blocklist and preserveDrawingBuffer: true.
  • The renderer writes build/comp_timing.json (sfx cues, line times) for the mixer.

Then: node tools/mix.mjs (music + VO + SFX, ducked, -14 LUFS) and mux: ffmpeg -i v.mp4 -i audio/mix.wav -map 0:v -map 1:a -c:v copy -c:a aac -b:a 256k -shortest -movflags +faststart out/name.mp4

Reference guides

The engine's written method, one file per concern.