Claude for video editing,end to end

Claude does not watch the video the way an editor does. It reads what can be measured: a word-level transcript, loudness, silence, face positions, frame counts. Then it writes the edit as commands and checks the result the same way. That is why it is good at the passes editors repeat on every episode, and why you still look at the output.

This is how Sprites cuts its own podcast and talking-head footage with Claude, simplified. Each step has its own page with the prompt and the commands.

1. Transcribe, primed for fillers

   Whisper with word timestamps and a prompt full of "um, uh, like", so the disfluencies are written down. A plain transcript hides them.

2. Tighten

   Fillers, stutters, restatements and pauses over 0.4 s come out. Never change meaning. See remove filler words and remove silence.

3. Place the cut points

   Transcript word ends run 100 to 300 ms early. Cut points come from phoneme-level forced alignment and a spectrogram, in real energy valleys. Audio edges stay exact; the picture snaps to frames.

4. Clean the audio

   High-pass, soft expander, automix, breath reduction, denoise, EQ, 3:1 compression, de-esser, limiter. -14 LUFS, true peak under -1 dBTP. See audio cleanup.

5. Pace each speaker

   Shorts go toward about 185 words a minute per speaker, capped so nobody sounds rushed, pitch preserved. Picture sped by the same factor.

6. Reframe and caption behind a gate

   Face centred per shot, captions with shrink-to-fit fonts, and every card built and pixel-checked before the render is allowed to start. See reframe to 9:16 and auto captions.

7. Thumbnails

   16:9, 9:16 and the two LinkedIn crops from one layout. See thumbnails.

8. Run the QA gauntlet

   Transcribe the final file primed and count fillers, scan every join for repeated audio, check lip sync in 2 s windows, re-measure loudness on the mp4, and look at a contact sheet end to end.

ffmpeg -i episode.mp4 -vn -ac 1 -ar 16000 work/episode_16k.wav

import whisper, json
model = whisper.load_model("turbo")
r = model.transcribe("work/episode_16k.wav", word_timestamps=True,
                     condition_on_previous_text=False,
                     initial_prompt="Umm, so, uh, I was like, uhh, you know, hmm... Um, yeah. Uh, we, um, like, uhm, basically, uh, okay.")
json.dump(r, open("work/words.json", "w"))

ffmpeg -i out/clip.mp4 -af ebur128=peak=true -f null - 2>&1 | grep -A12 Summary

ffmpeg -i out/clip.mp4 -vf "fps=1/2,scale=320:-1,tile=6x5" -frames:v 1 qa/sheet.png

1. Look at the output. A render that finished without an error is not proof the picture is right. Stills and contact sheets, every time.
2. Silence is not success. Check exit codes. A check that crashes prints nothing, and nothing looks like a pass.
3. Measure, do not model. Head tops, lip sync and loudness are measured off the picture and the encoded file, not estimated.
4. Originals are read-only. Everything new goes to a separate folder. Nothing overwrites the camera files.

A connector moves the edit into the vendor's app and onto its plan and credits. Running ffmpeg yourself keeps the files, the rules and the checks on your machine.