6Sense
Edge-native CCTV anomaly detection. A two-stage on-device pipeline — YOLOv11 pre-filter plus a vision-language model — with zero frames leaving the machine.
- edge-ai
- computer-vision
- yolo
- tensorrt
- vlm
- fastapi
Final-year major project: an on-device AI CCTV anomaly detection system built for Indian security environments. Live RTSP streams go through a two-tier cascade and raise alerts on a guard-station PWA — and no video frame, embedding or detection ever leaves the host machine.
The problem
Cloud video analytics ship every frame off-site: expensive bandwidth, unusable on a bad connection, and a privacy problem for anyone being filmed. The constraint here was the opposite — make it work on a single 6 GB consumer GPU, offline.
What I built
- Tier 1 — reflex. YOLOv11n (TensorRT FP16) plus MediaPipe pose and ByteTrack rejects roughly 90% of normal frames before anything expensive runs.
- Tier 2 — reasoning. Forwarded keyframes go to MiniCPM-V (INT4, llama.cpp), which returns a schema-validated JSON threat assessment instead of free text.
- Dispatch. FastAPI + async SQLite + a WebSocket alert channel, a 45-second dead-man's timer, and GSM SMS escalation when the network itself is down.
- Guard station. Next.js 15 PWA, plus a single-command orchestrator that auto-detects podman or docker.
Results
22.3 FPS at 1080p on an RTX 3050, 1.8–2.4 s p90 motion-to-alert latency, 3.77 GB peak VRAM. Part 1 shipped and benchmarked in October 2026; field hardening runs through February 2027.
What I learned
Most of the work was not the model. A strict three-consecutive-frame rule was silently dropping two thirds of real events; replacing it with a relaxed M-of-N latch and a geometric fall heuristic mattered more than any accuracy tweak.
Built with Prem, Prashant and Sarthak — I led the AI pipeline and system architecture. The repository is private while Part 2 is under development.