| name | clip-forge |
|---|---|
| description | Turn a long YouTube video, podcast, gameplay recording, match or tutorial into ready-to-post, fully edited vertical Shorts / Reels / TikToks. Finds the most viral moments (from the transcript for talk, from action, sound and scoreboard or kill-feed changes for gameplay and sport), shows a scored table for approval, then renders 1080x1920 clips with a real edit - colour grade, punch-in zooms, slow motion, sound effects, speaker tracking, an animated headline and word-by-word captions (English and Malayalam). Use whenever the user gives a YouTube link or a video file and wants shorts, reels, clips, highlights, "viral parts", "best moments", "cut this into short videos" or "edit this clip", even if they do not name this skill. Also use when a message starts with /clip-gameplay, /clip-podcast, /clip-sports, /clip-tutorial, /clip-video or /clip-edit. |
ClipForge
Long video in, a handful of finished vertical clips out. The pipeline is:
check_setup.py– what can this machine do?- Get the video (three routes, below)
- Find the moments – how depends on the mode
- Show a scored table, the user approves
- Plan the edit for each approved clip, then
render.py - Titles, captions and hashtags for each clip
The standard to hit is a clip that looks like an experienced editor made it:
full-screen framing, a designed headline, the picture and sound reacting on
the moments that matter. A plain cut with a label on top is not the goal.
references/editing.md says how; read it before step 5.
All scripts live in scripts/ next to this file. Run them with the absolute
path to that folder. Keep everything for one video in one work folder
(<workdir>), with finished clips in <workdir>/shorts.
Only work on videos the user owns or has permission to reuse. If they are clipping someone else's channel, mention once that re-uploading needs the owner's permission, then carry on with what they asked.
Mode
The kind of video decides how moments are found, framed and captioned. Read the one reference file for the mode before step 3, and follow it.
| Command | Mode | Reference | Use for |
|---|---|---|---|
/clip-gameplay <link> | gameplay | references/gameplay.md | Game recordings and streams, with or without a facecam |
/clip-podcast <link> | podcast | references/podcast.md | Podcasts, interviews, talking head: the moment is in the words |
/clip-sports <link> | sports | references/sports.md | Matches and broadcasts: goals, wickets, points, finishes |
/clip-tutorial <link> | tutorial | references/tutorial.md | How-to, lessons, coding, screen recordings: one complete idea per clip |
/clip-video <link> | video | references/video.md | Everything else: vlogs, reviews, events, travel, reactions |
/clip-edit <file or link> | edit | references/edit.md | The user already has the clip or the times and wants only the edit; replaces steps 3 and 4 |
- If the message starts with one of the commands (or the user names the kind of video in words), the mode is settled. Do not second-guess it.
- With no command, decide from the title, channel and a contact sheet
(
peek.py --atwith five times spread over the video). If it is still unclear, ask in one line which mode it is.
Anything after the link is a request to honour, for example
/clip-gameplay <link> top 3, neon captions, watermark @myname.
1. Check the machine
python scripts/check_setup.py
Read mode, todo and speech_to_text. Install what todo lists (they are
copy-paste commands; ask before installing on the user's own PC).
mode: local– YouTube is reachable (Claude Code on the user's own PC). The whole pipeline runs from just a link.mode: cloud– YouTube is blocked (Claude chat / cloud sessions). The download has to happen on the user's computer; everything after that runs here. Do not retry the download through other tools or mirrors, the block is at the network level.speech_to_text: false– no transcript can be made here unless the video has subtitles. Podcast mode needs one; say so before starting.
2. Get the video
A. Local mode, link given
python scripts/fetch.py "<url>" --out <workdir>
It prints the video path and any subtitle files. For videos over ~90 minutes
ask whether they want the whole thing or a section (--section 600-3000).
B. Cloud mode, link given
Say plainly, in one or two sentences, that this chat cannot download from YouTube itself and the download takes one step on their PC. Then:
- Send them
assets/pc/get_video.bat(once; they keep it in a folder of their choice, e.g. a project folder they have connected to the chat). - They double-click it and paste the link. It saves the video, subtitles
and info file into an
inboxfolder next to the .bat. Default is 720p to keep the file small;get_video.bat <url> 1080gives sharper crops. - If computer-folder tools are available in this session (names containing
remote-devices), list theinboxfolder, copy the.mp4and every.json3/.vttbeside it into the workspace, and continue. Those tools usually cap one file at about 400 MB: if the video is bigger, ask them to re-run the .bat at 480, or to tell you which part of the video matters. - If there are no such tools, ask them to attach the video and subtitle file to the chat (works for small files only), or to run this skill in Claude Code on the PC where route A works end to end.
C. A video file is already here (attached, or in a connected folder) – use it directly. A subtitle file with the same name beside it is picked up automatically.
3. Find the moments
Follow the reference file for the mode. The tools it draws on:
| Tool | What it gives |
|---|---|
transcribe.py <video> --out <workdir> [--lang en\|ml\|auto] [--model small] | Word-timed transcript: subtitle file → faster-whisper → sherpa-onnx. Exit code 3 = no transcript possible |
scout.py <video> --out <workdir> [--top 12] [--len 30] [--kind sports] [--roi x,y,w,h] | The stretches with the most sound and motion, ranked, each with an 8-frame filmstrip in <workdir>/scout/, plus end.jpg for the last 45 s. --roi is the kill-feed or scoreboard box: every change there is counted and timed |
track.py <video> --clips <workdir>/clips.json --write | Who is on screen and who is talking in each clip; writes a suggested layout, face positions and a speaker-following crop path into clips.json |
analyze_audio.py <video> --out <workdir> | Single loud instants (a laugh, a shout) with their lift in dB |
peek.py <video> --at 754,760 --out x.jpg | Frames at exact times |
peek.py <video> --range 998-1032 --n 8 --out x.jpg | A filmstrip of one stretch, start to finish |
peek.py <video> --at 600 --grid --out x.jpg | One frame with a pixel grid, for reading the box of a facecam, kill feed or scoreboard |
peek.py <video> --at 600,640 --crop x,y,w,h --out x.jpg | One region enlarged, for reading a score or other small on-screen text |
Notes on transcription:
exact_word_timing: falsemeans word timings are estimated inside each line. Captions still work, the highlight is just a little less tight.- Speech-to-text on a long video is slow on a small machine (roughly real
time with
smallon 2 CPUs). Run it in the background and tell the user. Do not go belowsmallfor Malayalam, and tell the user Malayalam captions will need a read-through. - After it finishes, read
<workdir>/transcript.txtin full. It is the compact[mm:ss] textversion; the JSON is for the renderer.
scout.py takes about two minutes for a 17-minute 1080p video. Its action
number is a rank inside that one video: it tells you where to look first, not
how good the moment is. Open the filmstrips and judge with your own eyes.
4. Score and get approval
Aim for 8–12 candidates on a long video (fewer if it is short), each 20–60 s; action moments can be as short as 12 s. Do not overlap them.
Score each out of 100:
| Part | Points | What earns them |
|---|---|---|
| Hook | 30 | First 3 seconds make a stranger stop: a claim, a question, or action already under way |
| Payoff | 25 | It delivers: the answer, the punchline, the kill, the fail |
| Stands alone | 20 | Makes sense without the rest of the video |
| Emotion | 15 | Surprise, laughter, tension, awe, or strong disagreement |
| Pace | 10 | No dead stretch; the length fits the content |
Be honest with the scores. A long video with three good moments should get three high scores and several mediocre ones, not twelve 85s. If you judged a moment from still frames only, say so.
Show the table and stop for approval:
| # | Time | Len | Score | Hook line (on screen) | Why it works |
|---|---|---|---|---|---|
| 1 | 12:34–13:10 | 36s | 88 | He quit a ₹40 lakh job for this | Bold claim up front, clean ending |
Ask which numbers to render (and offer "top 5"). Do not render before the user answers; rendering is the slow part and their taste decides. In the same message, ask once for anything not yet known that changes the render: the channel handle for the watermark, and whether they want a different look from the mode's default edit.
The hook line is the 3–8 word headline shown for the first 3.5 seconds. Put
*stars* around the one or two words that carry it; they take the accent
colour. Write it to create curiosity without lying about the clip: only
claim what you saw or read. Use the language the speaker uses (Malayalam hook
for a Malayalam clip).
5. Plan the edit and render
Read references/editing.md now. Then, for each approved clip, decide the
layout, the edit, the headline and the beats, and write
<workdir>/clips.json:
{"clips": [
{"id": 1, "start": 754.2, "end": 790.6, "hook": "He quit a *₹40 lakh* job",
"slug": "quit-job", "layout": "auto", "edit": "story", "tighten": true},
{"id": 4, "start": 2210, "end": 2236, "hook": "1 HP. *Four enemies.*",
"layout": "center", "edit": "hype", "captions": false,
"beats": ["auto", {"t": 2229.4, "fx": ["punch", "flash"], "text": "Clutch"},
{"t": 2231.0, "fx": "slowmo", "len": 1.2}]}
]}
Edits (the mode reference says which suits the mode):
| Edit | What it does |
|---|---|
hype | Vivid grade, slow push-in, punch + shake + flash with a bass thump on each beat, condensed all-caps headline, progress bar |
premium | Punchy grade, slow push-in, punch-in with a soft thump on each beat, bold headline (default) |
story | Soft grade, zoom cuts on sentence changes, animated captions |
clean | Soft grade and animated text only |
raw | Plain cut, boxed headline, static captions |
Beats are the moments the picture reacts to. Leave "beats" out to have
them found from the sound, or list your own (t in source seconds; fx of
punch, shake, flash, slowmo; optional text for a callout word).
Layouts (the mode reference says which to pick):
| Layout | Result |
|---|---|
speaker | 9:16 crop centred on the detected face; follows "focus_path" from track.py when present |
auto | speaker if a clear face is found, otherwise blur |
center | 9:16 crop that fills the screen; "focus_x": 0–1 slides it, "focus_path" moves it over time |
stack | Two speakers, one above the other; face positions come from track.py or "faces" |
split | Facecam on top, game below; needs "facecam": [x, y, w, h] |
blur | Whole width kept in the middle, blurred copy above and below; "zoom": 1.0–2.5 enlarges the sharp picture. Last resort: it leaves the picture small |
Looks, set for every clip on the command line or per clip in clips.json:
| Option | Per-clip key | What it does |
|---|---|---|
--edit hype\|premium\|story\|clean\|raw | "edit" | The editing treatment |
--style bold\|clean\|neon\|fire | "style" | Caption look and accent colour. bold: white capitals, yellow accent (default). clean: smaller sentence case, pale blue. neon: green accent, purple outline. fire: largest, orange accent |
--outro "Follow for more" | "outro" | Tag that pops over the last 1.6 s |
--watermark "@name" | "watermark" | Small see-through channel tag for the whole clip |
--progress | "progress" | Thin bar along the top that fills as the clip plays |
python scripts/render.py <video> --clips <workdir>/clips.json --out <workdir>/shorts [--edit hype] [--style fire] [--outro "Follow for more"] [--watermark "@name"]
One JSON line per clip (ok, file, layout, edit, seconds, and
beats with the times the effects landed). A full render takes about one
and a half times the clip's length per clip on an ordinary PC, so say how
long it will take and run it in the background. When the framing or a beat
time is a guess, run once with --preview first: it makes small fast drafts
(*_preview.mp4) to check before the full render.
Then check your own work before showing it: pull one frame from each clip and look at it.
ffmpeg -y -loglevel error -ss 2 -i <clip.mp4> -frames:v 1 <workdir>/check_01.jpg
Look at the headline (2 s), one beat, and the last second. Confirm the
subject is in frame, the hook is readable, captions do not sit on a face or
on the part of the game that matters, and the clip does not run into an end
card or a menu. Fix by changing the
layout, zoom, focus_x or facecam and re-render just that clip with
--only <id>.
6. Post copy
For each rendered clip write, in the clip's language:
- Title (under 60 characters, the hook rephrased, no clickbait the clip does not pay off)
- Caption (1–2 lines plus a question or call to action)
- Hashtags (3–5: one broad, the rest specific to the topic or game)
Save them as <workdir>/shorts/posts.md and show them in the reply.
7. Hand over
- Local mode: say where the
shortsfolder is. - Cloud mode: send each
.mp4andposts.mdto the user as files. If the video came from a connected folder on their computer, also save the clips into ashortsfolder beside theinbox.
Finish with one line per clip (number, length, hook) and offer changes: different start or end, another layout or caption style, a new hook line, more clips from the remaining candidates.
Things that go wrong
- Download fails in local mode with a sign-in or bot message – update
yt-dlp (
python -m pip install -U yt-dlp) and retry once. If it still fails, ask the user to download the video another way and use route C. transcribe.pyexits 3 with a "could not find module ... dll" error – the speech engine is installed but cannot load.check_setup.pyprints the fix (on Windows, the Visual C++ runtime). This is a machine problem, not a silent video: say so rather than treating the video as speechless.- Captions are wrong words – the transcript is wrong, not the renderer.
Fix the words in
transcript.jsonfor that clip's time range and re-render, or re-runtranscribe.pywith a bigger--model. - Face crop follows the wrong person –
track.pyguesses the speaker from mouth movement and can be wrong. Switch that clip tostack(both people on screen), orcenterwith afocus_x. - The edit feels like too much – drop to
premiumorclean, or give an explicit shortbeatslist in place of the automatic ones. assfilter missing – ffmpeg was built without libass;check_setupsays how to install a full build.
