Claude Desktop App with Claude Code Desktop App Only
Installed on your laptop and opened Claude Code inside it. Do not use Claude inside a web browser.
Build an AI Video Editor
with Claude Code.
You can build your own autonomous AI video editor using Claude Code and 5 free tools. Everything you need is laid out right here. Follow the 4 steps below, copy the 3 prompts, and explore the verified GitHub repositories at the end to inspect each tool.
Prerequisites
Before You Begin
Tap each requirement as you prepare it. We save your progress automatically so you don't lose your spot.
Claude Account with Code Access
An active Claude account permitted to run Claude Code (typically requires an official paid subscription).
Node.js v22 or Newer
Node.js 22+ installed on your laptop. Strictly required for HyperFrames to compile HTML compositions.
3 to 5 Instagram Reel Links
Public Instagram videos with the pacing, captions, and editing rhythm you want Claude to learn in Step 3.
1 Raw Video Recording
One unedited video clip of yourself speaking with natural pauses. You will feed this clip into Step 4.
Step 01
Set Up the Workspace
Create 1 root folder containing 3 subfolders, then open it in Claude Code so Claude can immediately navigate your workspace.
- On your computer, create a new root folder named
AI Video Editor. - Inside that folder, create 3 subfolders:
Footage,References, andOutput. - Open the Claude Desktop app, then start a new session in Claude Code.
- Select Local mode, then choose the
AI Video Editorfolder you just created.
🎬 raw-talking-head.mp4
🎙️ voice-memo-take2.m4a
Drop zone: Place all raw, unedited camera files, b-roll, and screen recordings here.
Claude never overwrites files in this folder.
🔗 reel-hook-sample-01.mp4
🔗 reel-subtitles-sample-02.mp4
📄 custom-brand-sheet.md
Inspiration hub: Prompt 2 instructs yt-dlp to download Instagram references here.
Claude dissects them to compile your custom brand sheet.
✨ reel-final-export-v1.mp4
📋 storyboard-breakdown.json
Export vault: Every rendered MP4, storyboard draft, and final asset lands here,
organized and ready to post.
DIR // 01
Footage
Your raw clip drop-zone. Untouched video & voice recordings.
DIR // 02
References
Where Claude downloads Instagram videos and saves your style rules.
DIR // 03
Output
Where your finished, polished, and exported Reels are delivered.
Step 02
Install the Tools
Paste Prompt 1 into Claude Code. Claude installs, configures, and self-tests all 5 engines automatically. Wait until it outputs its readiness confirmation.
Prompt 1: Install Tools
I want you to be my video editor. Install these tools on my laptop, then test them one by one: 1. Remotion: github.com/remotion-dev/remotion 2. HyperFrames: github.com/heygen-com/hyperframes 3. FFmpeg: github.com/FFmpeg/FFmpeg 4. Whisper (whisper.cpp): github.com/ggml-org/whisper.cpp 5. yt-dlp: github.com/yt-dlp/yt-dlp When everything is working, let me know that you're ready.
Under the hood: Claude verifies binary paths, downloads FFmpeg/Whisper models, clones
required toolchains, and verifies installation commands. Wait for its clean readiness response before pasting
Step 3.
Step 03
Teach Claude Your Editing Style
Paste your Instagram links below. The prompt updates live with your URLs, ready to copy into Claude Code with a single tap.
Prompt 2: Learn My Style
Paste Your Reference Reel URLs (Updates Live Below):
Study the editing style of these videos: [PASTE INSTAGRAM LINK 1] [PASTE INSTAGRAM LINK 2] [PASTE INSTAGRAM LINK 3] 1. Download all videos into my References folder. 2. Analyze the common patterns across these videos: fonts, colors, subtitle styles, cuts, zoom effects, pacing, graphic animations, music, and sound effects. 3. Build me a brand sheet (style guide) that I can review first. Make it inspired by these videos, not an exact copy. Once I approve it, use that style guide every time you edit my videos.
Review the style guide first: The instruction "inspired by these videos, not an exact
copy" guarantees your content retains its own distinctive originality while leveraging battle-tested
social media pacing.
Step 04
Record, Then Edit
Type in your raw video filename. The prompt updates live with your file path, ready to generate your first Reel.
Prompt 3: Edit into a Reel
Your Raw Video Filename inside Footage/:
Edit my new video into an Instagram Reel: [FILENAME] in my Footage folder 1. Cut out all silent pauses longer than 0.1 seconds. 2. Follow my approved style guide. 3. Add subtitles, background music, and motion graphic animations. 4. Use anything useful from my References folder. Show me the storyboard (scene plan) first for my review. Once I approve it, save the final video to the Output folder.
Review the Storyboard: Claude outlines every scene cut, zoom, and subtitle timing first. Once
you say "Approved", Claude triggers Remotion to render the final composition directly into
Natural Language Tweaks: If you want changes, simply talk like a director: "Make the intro hook 20% faster and scale the first title up."
Output.
Natural Language Tweaks: If you want changes, simply talk like a director: "Make the intro hook 20% faster and scale the first title up."
Visual Comparison: Silent Pauses Cut
✨ AI Cut Mode: 7 awkward pauses removed (>0.1s), dynamic word captions auto-synced, camera
zoom added to high-energy phrases.
The Engine
The 5 Core Tools & GitHub Repos
These 5 open-source engines power your video editor. Prompt 1 handles their installation automatically. All repository links verified.
| Tool | Role in Your Pipeline | Repository |
|---|---|---|
| Remotion | Creates programmatic videos using React code, built for frame-perfect cuts, typography, and motion graphics. | remotion-dev/remotion ↗ |
| HyperFrames | HeyGen's open-source tool that converts HTML directly into MP4 video, designed specifically for autonomous AI agents. | heygen-com/hyperframes ↗ |
| FFmpeg | The industry standard multimedia engine for silence trimming, audio merging, transcoding, and video assembly. | FFmpeg/FFmpeg ↗ |
| Whisper (whisper.cpp) | High-performance local speech-to-text inference engine that generates millisecond-accurate subtitle timings. | ggml-org/whisper.cpp ↗ |
| yt-dlp | Reliably downloads reference reels from Instagram and other media platforms directly to your workspace. | yt-dlp/yt-dlp ↗ |
Note: The FFmpeg GitHub repository is an official mirror; its primary home is hosted at ffmpeg.org. Each tool operates under its own open-source license, so review repository terms before employing them in commercial client deliverables.
# Click a prompt button above to simulate how Claude Code executes the
workflow
claude-code:~/AI Video Editor$ _
Best Practices
Tips for Exceptional Results
Four practical rules to guarantee high audience retention, zero hallucinated cuts, and clean final renders.
Pick Harmonious Reference Videos
Select 3 to 5 videos that share a similar visual cadence and typography. Clashing styles confuse the brand sheet generator.
Always Scrutinize the Storyboard
Review the timeline plan before approving. Catching pacing mismatches or subtitle typos early avoids unnecessary re-rendering.
One Iteration at a Time
Keep your revision requests simple and focused. E.g., "Make the intro title bigger" works far better than long multi-part demands.
Keep Folder Conventions Strict
Raw footage always lives in Footage, finished renders always in Output.
Predictable paths prevent file errors.
Troubleshooting & Questions
Frequently Asked Questions
Quick answers to common setup hurdles and workflow fine-tuning.
What if yt-dlp fails to download an Instagram Reel?
+
Instagram frequently adjusts rate-limits. If yt-dlp stumbles on a link, simply download the video manually
using your browser or a trusted media downloader and drop the
.mp4 directly into your
References/ folder. Claude will analyze it identically.
Can I use my own royalty-free music instead of downloaded audio?
+
Yes. Drop any
.mp3 or .wav into your References/ folder, then in
Prompt 3 add: "Use audio.mp3 from my References folder for the background audio at -18dB."
Do I need to install Node.js globally?
+
Yes, Node.js version 22 or newer must be accessible from your system PATH so both Remotion and HyperFrames
can compile rendering pipelines without dependency conflicts.
How fast does the render process typically take?
+
For a standard 30 to 60-second vertical Reel with captions and jump cuts, local rendering usually
completes within 10 to 35 seconds depending on your laptop's CPU/GPU and video resolution.