Leobit
AI-Powered Video Analysis and Tagging Solution
Custom development of a PoC that automates the processing of video materials across industries
ABOUT the project
- Client:
- Leobit's Internal Project
- Location:
-
USA
- Company Size:
- 100+ Employees
- Industry:
-
Information Technology
- Solution:
- Custom Software
A proof of concept for a machine video intelligence (MVI) system that runs a comprehensive video content analysis. The tool can segment scenes from the videos, detect objects, activities, and analyze behavior. Upon processing the video, the solution delivers a detailed analytical report with timestamps, confidence scores, and visual overlays.
We managed to build a solid framework for real-time video analysis using computer vision. The right combination of models enables precise tagging and segmentation of videos in the short term. In the future, we may improve this framework, enhancing token consumption and processing speed.
Customer
We built this PoC to explore the capabilities of computer vision models for real-time video analysis. As a result, we developed a framework that can be reused in other projects across industries.
Business Challenge
Manual video analysis is time-consuming and resource-intensive, particularly for large volumes of footage. Organizations across legal, security, sports, healthcare, and construction sectors struggle to extract actionable insights from video content efficiently. Existing solutions often lack precision, customization, or fail to provide detailed metadata like timestamps and confidence scores. There’s a clear need for automated, intelligent video analysis that delivers comprehensive, searchable analytics without human review bottlenecks.
Project
in detail
We built the solution in around 4 weeks. Although it is a proof of concept, the tool offers a convenient user experience and can be explored by multiple stakeholders.
We started with an analysis of existing computer vision models and architectural patterns. As a result, we decided to build an Azure-based tool relying on the Ultralytics YOLOv8 model for real-time object detection and the HuggingFace Transformers model for activity recognition.
We integrated the models with a . NET-based back-end that orchestrates the models. The models are configured to process video streams asynchronously and generate structured analytical reports. The architecture is quite simple, but it handles its primary functions effectively.
The Leobit team constructed a responsive interface using Angular and Tailwind CSS that simplifies video upload, processing, and results exploration. Users can easily navigate detected objects, activities, and events with precise timestamps and confidence scores.
Advanced Object Detection
YOLOv8 detects and classifies objects in real time, providing bounding boxes, labels, and confidence scores. The system is configured to detect and categorize different object categories. By facilitating the detection of people, vehicles, tools, and equipment, it enables precise tracking across video sequences.
Activity & Behavior Recognition
HuggingFace Transformers analyzes temporal sequences to identify activities, interactions, and behavioral patterns. The system segments scenes based on activity type, enabling users to quickly locate relevant events within long video files.
Searchable Analytics & Timestamps
All detections are indexed by timestamp, object type, activity, and confidence score. Users can search and filter results to locate specific events instantly, eliminating manual frame-by-frame review.
Technology Solutions
- Integration with Ultralytics YOLOv8 and HuggingFace Transformers that ensures real-time object detection and activity recognition.
- Simple yet efficient .NET back end responsible for the system’s orchestration.
- Simple and intuitive UI built with Angular and Tailwind CSS.
Value Delivered
- Reduction in manual video review time across legal, security, sports, and healthcare.
- Rapid event discovery and evidence extraction with searchable timestamps.
- Scalable, cloud-native framework for deployment across enterprise environments.
- Detailed, confidence-scored analytics that support data-driven decision-making.
