PopTok: a fully local pipeline from one prompt to a lip-synced music video (ACE-Step, LTX-2.5, InfiniteTalk)

Julian Harris shared PopTok on Oct 25, and LTX featured it the same day in its weekly community roundup. The repo (Apache-2.0) is new and has few stars, but the design is worth studying. There are three ways in: a finished song plus lyrics; lyrics plus a music style, where ACE-Step 1.5 makes several takes and keeps the one Whisper transcribes best; or a single prompt, where three LLM 'songwriters' write the lyrics and style first.

Then a 'directors' workshop' of three simulated music-video directors writes the visual treatment: one universe style that leads every image prompt, a shot for every lyric line and every 6–8 s of instrumental, and where the performer appears or sings. Stills come from MOSS-Image and Qwen-Image-Edit, with an automatic check that re-rolls bad frames. LTX-2.5 with an LTX-2.3 Transition LoRA turns them into one continuous take. Scene changes are then moved onto downbeats, and InfiniteTalk lip-syncs only the sung passages, cut in from the vocal stem. One flag stops the run after the stills, so you can review them before hours of video rendering.

Hardware is the catch. The author says the full pipeline wants a 128 GB DGX-class box. Parts run on an RTX 4090 (analysis, stills, int8 lip-sync) and an M5 Max Mac (LTX video), and the README publishes per-step timings for each machine. The local LLM is Qwen 3.8 27B on llama.cpp.

Why it's on ReelStack: beat-snapped cuts, Whisper-aligned lyrics and lip-sync only where the singer is on camera are the right instincts for music videos, whatever models you use. A lyric_loop tool re-spells words the singer garbles until Whisper hears them correctly. That's a neat trick for any AI-music workflow.

Our take: most people should borrow the planning and beat-alignment steps and use hosted models for the heavy renders. If you do music videos for clients, use your own performer photo with consent.