Building an AI video clipper in Streamlit

The first version of this clipper was a Jupyter notebook (clipper.ipynb in my repository). It took a video, transcribed it and used Large Language Models (LLMs) to find and cut segments. It worked, but a notebook is not a friendly interface for a tool I wanted others to use.

So I turned the notebook into a fully interactive Streamlit web app. My AI collaborator, Google Gemini 2.5 Pro Preview, did the refactoring and wrote the new code, guided by a comprehensive system instruction and my iterative architecture decisions. This was an exercise in my AI-first development philosophy, and it mirrors the rapid re-platforming I described when I transformed a CLI tool into an Electron GUI.

The goal

The goal was an app where users could:

1. Input a YouTube URL or upload a local MP4 file.
2. Choose a transcription method:
   • YouTube's API, for speed with YouTube videos.
   • Local Whisper, for privacy and control, using a local GPU when one is available. This is similar to my work with NVIDIA GPUs and Ollama for local AI.
   • OpenAI's Whisper API, for potentially the highest accuracy through the cloud.
3. Let an AI analyze the transcript to find key topics.
4. Have another AI extract video segments for each topic, within the duration limits the user sets.
5. Generate the clips with FFMPEG.
6. Download the snippets.

This required moving beyond a linear script to a modular, service-oriented architecture. That pattern also suits more complex systems, such as my Desktop AI Assistant.

Refactoring the monolith into modules

The core of this build was less about me writing code and more about me designing the system and prompting the AI to implement it. The process had four parts.

1. Define the new architecture. We decided on this structure:
   • app.py: the Streamlit UI.
   • core/: Pydantic data models (models.py) and shared constants.py.
   • services/: distinct functionality (system, video, transcription, LLM, FFMPEG).
   • orchestrators/: pipeline logic (common_steps.py, youtube_pipeline.py, local_mp4_pipeline.py), similar to how n8n orchestrates complex AI workflows.
   • utils.py: a thin facade to the orchestrators for app.py.
   My role was to define these boundaries and each module's responsibilities. The AI generated the Python code for each file from those specs and the logic in the original notebook. A clear plan matters, much like nailing down user stories before development.
2. Implement each service. For each service, I gave the AI the relevant snippets from the old notebook and described the required functions, inputs, outputs and error handling. For transcription_service.py, I outlined the three methods and prompted for each one, including the FFMPEG audio extraction step for the OpenAI API.
3. Build the orchestration layer. The orchestrator functions in utils.py, which call into orchestrators/, manage the end-to-end flow. They decide which services to call based on user input. Careful prompting ensured correct data handoff between services and proper error propagation.
4. Craft the Streamlit UI (app.py). With the backend modular, the AI built the interface. It includes the sidebar for user configuration, display areas for processing logs, GPU information and results, input handling that triggers the pipeline, and clip information with download buttons. My prompts focused on layout, widget types and session state management. Streamlit's ease of building UIs is a key part of my AI development power stack.

Key technical hurdles

We hit several challenges during the build and solved them with the AI's help.

• FFMPEG accessibility in Streamlit. The ffmpeg.exe bundled in the video-clipper-streamlit app directory had to be found reliably by subprocess calls. The fix lives in services/system_service.ensure_ffmpeg_is_accessible().
• Local Whisper on the GPU. Getting Local Whisper (via stable-whisper) to use the NVIDIA GPU correctly took work. The AI wrote debugging logs that pinpointed discrepancies, much like the detailed setup needed for running Ollama with NVIDIA GPUs.
• LLM prompt engineering for segment extraction. At first the LLM struggled with strict duration limits. Iterative refinement of the prompt in services/llm_service.py fixed it, much as code2prompt or Context7 prompts need tuning to get the output you want.
• Handling None returns in orchestration. Orchestrator functions had to handle potential None returns from services gracefully.
• Python imports in a modular structure. Managing imports across core, services and orchestrators was detailed work, and the AI handled it well.

The outcome is a working modular AI tool

The result is the video-clipper-streamlit app we have been working on. It now:

• Accepts YouTube URLs or MP4 uploads.
• Offers three transcription methods.
• Uses LLMs to identify topics and then extract relevant segments.
• Generates downloadable video clips.
• Provides real-time feedback through a processing log.

The modular structure is easier to maintain and extend than the original monolithic notebook.

Learnings from an AI-first build

• With clear architecture and precise prompting, LLMs can write most of the application code. This is the evolution of "Fluid Dev," where conversational AI becomes the new IDE.
• Modularity matters, even for AI. Breaking problems into smaller modules helps both the human architect and the AI coder.
• System instructions come first. A detailed system instruction up front is essential. I covered this principle in my 15 tips for AI-first development in AI Studio.
• The human role shifts. My main work was design, defining interfaces, prompting and validating.
• Debugging is collaborative. Giving the AI errors and context helps it produce fixes.

The "AI hammer" approach suits trivial tasks. This project used AI to build a larger, more useful application.

Next steps for the clipper

• Further refining LLM prompts for more consistent segment extraction.
• Adding more UI and UX polish, perhaps more granular progress indicators.
• Extensive testing with a wider variety of video inputs.

If you want help with AI-driven development or automation, get in touch.