How I Built FlashCut: An AI Video Highlight Generator with YOLOv8 and Kotlin
🚀 Published App: FlashCut is live on the Google Play Store! You can View FlashCut on Google Play → or explore its case study in my Portfolio Selected Work.
The Problem I Wanted to Solve
Every time I recorded a long video — whether a sports match, a lecture, or a gameplay session — I faced the same tedious task: scrubbing through 30+ minutes of footage just to locate the 2-3 minutes worth keeping.
Manual video scrubbing is slow, repetitive, and consumes valuable time. That process is what FlashCut was built to eliminate. It is a native Android application that analyzes video content and audio energy to automatically detect key moments and compile them into a concise highlight reel — eliminating manual scrubbing.
Why Native Android with Kotlin & Jetpack Compose
When planning FlashCut, I chose Native Android (Kotlin + Jetpack Compose) because the application was targeted directly at the Android platform.
Directly building native Android provided clear architectural advantages:
- Direct Control Over Platform & Media APIs: Native integration with Android's media components, file system, and background execution services.
- Granular Lifecycle Management: Precise control over memory usage and state restoration during asynchronous processing.
- Modern Declarative UI: Jetpack Compose provided reactive, clean state management when rendering real-time progress indicators and interactive clip controls.
By keeping the mobile client strictly native, the mobile layer handles UI and playback efficiently while delegating heavy computational tasks to a dedicated backend.
System Architecture & End-to-End Flow
FlashCut operates on a decoupled client-server architecture:
[ Android App (Kotlin + Compose) ]
│ (Video Upload)
▼
[ Flask REST API (Python Backend) ]
│
├──► 1. PySceneDetect ──► Scene Cuts & Segments
├──► 2. YOLOv8 ─────────► Object Detection on Sampled Frames
├──► 3. librosa ────────► Audio Energy & Loudness Analysis
│
▼
[ Highlight Scoring & Selection ]
│
▼
[ MoviePy / OpenCV ] ──► Compiled Highlight Video ──► [ Returned to Android App ]
The Step-by-Step Processing Pipeline
- Video Upload: The user selects a video in the Android app. The app uploads the media file to the Python Flask backend hosted on Cloud Infrastructure.
- Scene Segmentation (PySceneDetect): Before running computer vision models, PySceneDetect analyzes the video stream to detect visual scene transitions, breaking the raw video into logical scene segments.
- Object Detection (YOLOv8): Sampled frames within each scene are processed using YOLOv8 for object detection — identifying key visual subjects or targets across frames.
- Audio Energy Analysis (librosa): In parallel, librosa extracts the audio track to measure volume dynamics and audio energy peaks (such as cheering, crowd noise, or sudden exclamations).
- Highlight Scoring: Visual object detections and audio energy signals are combined into a score for each scene.
- Selection & Video Generation: Top-scoring scenes are compiled using MoviePy and OpenCV into a final highlight video file returned to the user.
Highlight Scoring: Combining Visual & Audio Signals
Relying on computer vision alone is rarely enough for sports or event videos. A subject standing still might look visually identical to a subject during an action moment.
To address this, FlashCut combines visual detection signals with audio energy analysis when ranking potential highlights:
1. Visual Signals (YOLOv8)
YOLOv8 scans sampled keyframes within a scene to detect the presence and concentration of primary targets (e.g., people or specific subjects). Scenes with clear target presence receive positive visual relevance signals.
2. Audio Energy Signals (librosa)
Audio analysis using librosa evaluates audio energy dynamics across time:
- Loudness and energy levels identify sudden spikes in audio volume.
- High-energy audio segments (cheering, loud reactions, impact sounds) indicate moments of heightened activity.
3. Combined Segment Ranking
Each candidate scene segment is assigned a score based on visual target presence and audio energy. Segments exceeding the highlight threshold are automatically selected for extraction, ensuring that visually relevant moments supported by audio excitement make the final cut.
Technical Engineering Challenges
Building FlashCut required solving several real-world production challenges:
1. Asynchronous Task Feedback
Processing long videos against YOLOv8 and librosa takes time. Freezing the UI or letting HTTP requests time out was unacceptable. I implemented an asynchronous job processing flow on the backend paired with polling endpoints, allowing the Jetpack Compose UI to display real-time progress state without hanging.
2. Trustworthy UX & Manual Overrides
An automated AI feature is only as useful as the controls given to the user when edge cases occur. To ensure reliability, FlashCut includes an interactive clip editor in Compose: users can review automatically selected clips and adjust boundaries before exporting.
3. Production Deployment & Authentication
- Backend Infrastructure: Hosted on Oracle Cloud Infrastructure to provide steady computational resources for video processing.
- Authentication: Integrated Firebase Authentication to manage user identity and secure storage access.
- Production Builds: Transitioning from local debug builds to Google Play Store release configurations required configuring proper API endpoints, keys, and release build security.
What Shipping Taught Me
FlashCut is a real, published application on Google Play — not a tutorial project. Shipping a production app to the store highlighted real engineering insights:
- Edge-Case Resilience: Handling interrupted network connections mid-upload gracefully.
- Corrupted Media Handling: Validating media headers on the server before starting heavy pipeline runs.
- Debug vs. Production Builds: Ensuring native dependencies and API endpoints resolve correctly under production build configurations.
Tech Stack Summary
- Mobile Client: Kotlin, Jetpack Compose, MVVM Architecture
- AI & Computer Vision: YOLOv8 (Object Detection), Python, OpenCV
- Scene & Audio Analysis: PySceneDetect, librosa
- Backend Services: Flask (REST API), Python, MoviePy
- Infrastructure & Auth: Oracle Cloud Infrastructure, Firebase Authentication
- Distribution: Google Play Store (View App Listing)
