Workflows & pipelines

Multi-tool pipelines pieced together from the sources linked under each one. They’re labelled Spotted, not yet tested until we publish our own renders.

Code-only animated explainer: JS canvas → Playwright → FFmpeg

Animation / explainer · Spotted, not yet tested

Free, deterministic animation where the agent draws every frame. Based on Chase AI's walkthrough and the animate / claude-motion skills.

Tools: Claude Code (or any coding agent), Node + Playwright, FFmpeg, faster-whisper, Python (sound)

  1. Brief: subject, format (9:16 / 16:9), length, voice or no voice, music or code score.
  2. Story check: have the agent produce a numbered beat table with timestamps in timeline.json; approve it.
  3. Look check: render 2–4 full-size style frames (or match frames to your references side by side); approve.
  4. Storyboard: one key frame per beat with the sound cue and transition; thumbs up/down per panel.
  5. Build: one HTML page with a <canvas>; draw(t) is a pure function of time with seeded randomness (no Date.now, no Math.random).
  6. Audio: record or TTS each VO line; run faster-whisper for word timestamps and write them into timeline.json; or detect beats from your track (Demucs drums → beat grid); generate SFX/score in Python from the same timeline.
  7. Capture: Playwright opens the page, sets t = frame/fps, screenshots each frame (or streams to ffmpeg via stdin).
  8. Encode: ffmpeg -framerate 30 -i f_%05d.png -i mix.wav -c:v libx264 -pix_fmt yuv420p -c:a aac -shortest out.mp4
  9. Self-review: build a contact sheet (ffmpeg -vf "fps=1,scale=480:-1,tile=6x4") and have the agent check clipping, overlaps, hold times, safe areas and phone legibility; fix and re-render.
  10. Master audio to about −14 LUFS for social and export 9:16, 1:1, 16:9 as separate layouts (not crops).

Sources: Chase AI video · cth9191/animate · whaleyxbt/claude-motion

HyperFrames product launch video from a URL

Motion graphics · Spotted, not yet tested

Use HeyGen's skill router to go from a product site to a rendered launch video.

Tools: HyperFrames CLI + skills, Node 22+, FFmpeg, Firecrawl (optional brand extraction), TTS (media-use skill or ElevenLabs MCP)

  1. npx skills add heygen-com/hyperframes → select Core Skills → restart the agent session.
  2. npx hyperframes init launch && npx hyperframes preview (keep the preview open while prompting).
  3. Optional: extract the real logo, palette and fonts from the site (Firecrawl) and save them as a brand file.
  4. Prompt: '/hyperframes make a 30s launch video for <url>. Brand file: brand.json. One typeface, hex palette, no cuts between scenes 2–4, VO script under 70 words.'
  5. Let the router confirm the brief, then review the generated script and storyboard before animation.
  6. Iterate in the preview; name problems by timestamp ('at 0:07 the headline collides with the UI shot').
  7. npx hyperframes lint && npx hyperframes check, then npx hyperframes render --output final.mp4.
  8. Optional: render 9:16 and 16:9 versions from the same composition.

Sources: HyperFrames prompt guide · Skills guide · Jack Roberts: brand extraction

HyperFrames: talking-head footage → captioned, designed recut

Creator / influencer · Spotted, not yet tested

Your own recorded footage, cleaned up and branded with motion graphics.

Tools: HyperFrames skills, WhisperX or the media-use transcribe skill, FFmpeg

  1. Drop your footage into the project and ask /hyperframes for 'captions or a designed recut'.
  2. Transcribe with word timestamps; have the agent propose cuts for dead air and retakes as a list you approve.
  3. Add motion B-roll and lower thirds keyed to specific words (pattern from Barty-Bart/motion-graphics).
  4. Review stills at every caption change; render.

Sources: HyperFrames skills workflows · Barty-Bart/motion-graphics

Higgsfield MCP + code motion: hybrid cinematic launch

Cinematic + motion graphics · Spotted, not yet tested

Generated cinematic plates from Higgsfield, with type and UI motion layered on top in code.

Tools: Claude + Higgsfield MCP, Remotion or HyperFrames, Refero / Mobbin references

  1. Collect 3–5 design references (Refero, Mobbin) and write a one-page style spec.
  2. Connect https://mcp.higgsfield.ai/mcp in Claude (Settings → Connectors). Check the credit balance via the agent.
  3. Generate hero stills first (e.g. Seedream / Soul Cinema); approve one starting frame per shot.
  4. Animate approved stills to short plates (Seedance 2.5 or Kling); keep motion simple: push-in, orbit, a single action.
  5. Import plates into a Remotion/HyperFrames composition; layer kinetic type, UI and logo in code.
  6. Generate score/SFX in code or via Higgsfield/ElevenLabs audio; sync to the timeline.
  7. Render, contact-sheet review, deliver.

Sources: Jack Roberts: Claude Motion Design · Jason Lee: Opus 5.5 motion graphics · Higgsfield MCP help

Higgsfield: consistent fictional spokesperson with lip-sync

UGC / talking head · Spotted, not yet tested

Higgsfield's documented chain for a talking AI avatar. Fictional characters or consenting people only.

Tools: Higgsfield MCP, Soul 2.0, Soul ID, Seed Audio, Seedance 2.5

  1. Generate a fictional character and a reference set with Soul 2.0 (varied angles, expressions, one full-height shot).
  2. Ask Claude to train a Soul ID on 20–80 of those images (~10 minutes).
  3. Pick a preset voice in Seed Audio (or clone your own voice, with consent).
  4. Generate the starting frame with the Soul ID by name; spend time on this frame.
  5. Seedance 2.5: starting frame + audio track + short performance prompt → lip-synced clip.
  6. Label synthetic spokespeople where platform rules or law require.

Sources: Higgsfield talking-avatar guide · @maxxmalist on starting frames

Higgsfield Katana: rebuild a reference edit with your product

Product / ads · Spotted, not yet tested

Reuse the pacing of an edit you own with new characters or products.

Tools: Claude + Higgsfield MCP, Katana

  1. Choose a reference edit you own or have rights to (including its music), or a Katana library preset.
  2. Upload up to five clean photos of the product, character or clothing.
  3. Ask Katana to recreate the edit; it reads shots, pacing and audio.
  4. Correct in chat ('swap shot 3 for a macro of the logo'); each revision builds on the last.
  5. Make variant B with a different product in the same conversation.

Sources: Higgsfield Katana · @maxxmalist demo

ComfyUI Wan 3.0: multi-shot sequence from one reference set

Cinematic / UGC · Spotted, not yet tested

Consistency via reference tags and last-frame chaining.

Tools: ComfyUI (latest), Wan 3.0 Partner Nodes, VideoHelperSuite

  1. Template Library → 'Wan 3.0' → reference-to-video.
  2. Connect references in a fixed order: Image1 = character, Image2 = product, Video1 = motion ref, Audio1 = voice/rhythm.
  3. Prompt with tags: '@Image1 picks up @Image2 from the counter, mirrors the gesture in @Video1, handheld 35mm.'
  4. Generate shot 1 at short length; extract the last frame (VideoHelperSuite) and use it as the first frame for shot 2.
  5. Keep seed, references and lens language constant across shots; assemble in an editor.

Sources: ComfyUI Wan 3.0 docs · Pixraft multi-shot advice

Music video: own the track, cut on the beat

Music video · Spotted, not yet tested

Generate or license the music, extract beats, and have both generated clips and code motion snap to them.

Tools: ACE-Step 1.5 (or a licensed track), Demucs, librosa / beats.py, WhisperX (lyrics), Remotion / HyperFrames / any i2v model

  1. Generate a track with ACE-Step 1.5 (set BPM, key and duration) or use one you've licensed.
  2. Split stems with Demucs; detect beats/bars/drops on drums (claude-motion's beats.py does this).
  3. Time lyrics word by word with WhisperX on the vocal stem (the per-character lyric timing seen in the Skillry gallery).
  4. Write timeline.json: beats, sections, lyric words.
  5. Generate clips per section at the section's length; place code typography on lyric timestamps.
  6. Land the biggest visual change on the drop; silence or hold before the payoff.

Sources: ACE-Step 1.5 · claude-motion sims/beats · Skillry gallery

Own-footage character replacement (consent-based)

VFX · Spotted, not yet tested

Keep a performance and swap in a fictional character on footage you own. Based on the open Genjustsu workflow, with the filter-avoidance steps left out.

Tools: Demucs, Depth Anything (video), SAM 3, Seedance via an API provider

  1. Use footage you shot, with the performer's consent; prepare fictional character references.
  2. Normalize to 24 fps; generate a depth pass; segment the subject with SAM 3.
  3. Review the full-clip mask; approve it (the repo binds approval to file hashes).
  4. Generate a draft; approve before paying for 1080p.
  5. Restore your original audio in the final export.

Sources: Genjustsu repo

How to turn a workflow into an agent skill

Patterns taken from the strongest skills we looked at: animate, claude-motion, HyperFrames and Skillry.

  1. Run the workflow by hand once, end to end, and write down every decision you made (format, references, model, settings, what you rejected and why).
  2. Split it into stages with cheap approval gates (brief → beats → look frames → storyboard → build → review). The animate skill is a good model.
  3. Turn taste into numbers: easing curves, stagger, hold times, type sizes, palette hex codes, loudness target. claude-motion's motion-design skill shows the level of detail that works.
  4. Make one timeline file the single source of truth (beats, words, scenes) that picture, sound and review all read.
  5. Add a self-review step the agent can actually perform: contact sheets of stills, measured checks (text overlap, clipping, beat alignment, LUFS).
  6. Write SKILL.md: a front-matter name and a one-line description of when to use it, then the steps, file layout, commands and a checklist. Put scripts and templates beside it.
  7. Include one finished reference piece and its source, so the agent can take it apart.
  8. Declare costs and keys up front (which steps call paid APIs), and require explicit confirmation before any paid generation.
  9. Add guardrails in the skill itself: owned or licensed references only, fictional or consenting likenesses, no filter-evasion steps.
  10. Install it where your agent looks (e.g. .claude/skills/<name>/ or via npx skills add <repo>) and test it on three new briefs. Fix what breaks, version it, and publish.
my-video-skill/
  SKILL.md          # name, when-to-use, steps, checklist
  timeline.schema.json
  templates/        # starter composition, style file
  scripts/          # render.mjs, contact-sheet.sh, beats.py
  examples/01/      # one finished piece + its source

Starter tool chains

Our original outlines, kept for reference.

Cinematic product spot

  1. Higgsfield: Generate hero shots from a shot list (one prompt per shot).
  2. Topaz Video: Upscale and clean up the selected takes.
  3. ElevenLabs: Record the voiceover from your script.
  4. Submagic: Add captions for social cutdowns.

Talking-head explainer in multiple languages

  1. HeyGen: Create the presenter video from a script.
  2. Descript: Tighten the edit and add B-roll.
  3. Rask AI: Translate and dub into target languages.

Long-form to shorts

  1. OpusClip: Find and cut candidate short clips.
  2. Captions: Style captions and reframe for vertical.
  3. Suno: Generate a music bed (check license terms).