SENTINEL AI — Agentic Industrial Safety Intelligence Platform
See hazards. Understand context. Verify evidence. Vision-based detection, deterministic safety policies, agent-assisted investigation, and interactive 3D digital-twin environments.
Problem Statement
Industrial facilities struggle with fragmented camera feeds, manual incident reviews, and disconnected evidence workflows that delay critical hazard response. Furthermore, naive LLM vision systems are non-deterministic and can hallucinate safety approvals, creating catastrophic compliance vulnerabilities under OSHA and ISO 45001 standards.
Engineering Goals
- Deliver sub-18ms real-time object detection and multi-camera trajectory tracking.
- Enforce authoritative safety policies deterministically, strictly separating AI-assisted investigation from non-negotiable safety enforcement.
- Project 2D camera observations into an interactive 3D industrial digital-twin environment.
- Guarantee evidence integrity through SHA-256 cryptographic hash chaining for regulatory audit readiness.
- Implement secure, enterprise-grade access control with JWT authentication and role-based permissions (RBAC).
SENTINEL AI — System ArchitectureVerified Pipeline
Select any pipeline stage below to inspect verified engineering details and data flow.
Stage: Multi-Camera Ingest
Role: IngestionCaptures industrial RTSP/video feeds, performs color normalization, and feeds non-blocking frame buffers.
Execution Pipeline & Mechanics
Multi-Camera Ingest & Frame Normalization
Video streams are decoded asynchronously with non-blocking queue buffers, normalized for illumination variance, and dispatched to inference workers.
YOLOv8 Perception & Spatial Tracking
PyTorch neural networks identify personnel, PPE compliance (helmets, high-visibility vests), and moving machinery with persistent track IDs across temporary visual occlusions.
Deterministic Spatial Incursion Gate
Worker bounding polygons are evaluated against defined 2D/3D hazard exclusion zones. Incursions require temporal persistence debounce (consecutive frame threshold) before triggering policy gates.
4-Stage Agentic Investigation Workflow
Triggered safety events dispatch a structured agent pipeline: the Investigator gathers scene context, Policy Gate evaluates compliance rules, Responder formulates remediation, and Verifier validates recommendation consistency.
3D Digital Twin Projection & Cryptographic Audit
Events are projected into the Three.js facility model in real time while visual frame crops and event metadata are hashed via SHA-256 and appended to the immutable database ledger.
Engineering Challenges & Design Decisions
Decision: Deterministic Geometric Rules over End-to-End Multimodal LLM Safety Decisions
Rationale: Multimodal LLMs have non-zero hallucination rates and variable latency (1-3s). In industrial safety, a missed alarm can cause injury. Hardcoded polygon math executes in microseconds with 100% deterministic reproducibility.
Alternative Rejected: Direct prompt-based visual question answering using multimodal vision LLMs.
Decision: Cryptographic SHA-256 Hash Chaining over Standard Database Timestamps
Rationale: Regulatory disputes and insurance investigations require proof that incident evidence was not retroactively edited or trimmed. Cryptographic hash chaining ensures complete data integrity.
Alternative Rejected: Standard relational database timestamps with mutable media storage.
Core Architectural Highlights
SENTINEL AI is organized into four modular tiers: Perception (OpenCV frame intake + PyTorch YOLOv8 detection + ByteTrack association), Deterministic Safety Enforcement (geometric polygon containment calculus), Agentic Investigation (Investigator, Policy Gate, Responder, and Verifier workflow), and Operational Experience (React TypeScript frontend, Three.js 3D facility twin, FastAPI async backend, and cryptographic evidence ledger).
Testing & Empirical Evaluation
116 automated pytest unit and integration tests covering YOLO inference adapters, polygon incursion boundary tests, multi-object tracking persistence, SHA-256 cryptographic chain validation, and JWT RBAC security policies.
Security & Reliability Defenses
Key Engineering Learnings
- •Enterprises eagerly adopt AI for safety investigation when guaranteed that an AI agent cannot arbitrarily silence or bypass authoritative safety rules.
- •Temporal debounce algorithms eliminated over 94% of transient false-positive boundary warnings caused by optical noise without compromising reaction speed.