The Play-Detection Cascade
Accurate clipping is the foundation of any film-study system. Our cascade prioritizes reliability and auditability over black-box automation.
The system does not rely on a single learned model. Instead it fuses cheap, interpretable signals with increasing precision:
- Field Mask — computed once per source. All subsequent motion analysis ignores everything outside the field lines.
- Camera Motion Compensation — modern global motion estimation subtracts operator movement so that player motion isn't masked by camera work (the primary failure mode in 2013 research).
- Active-Cell Motion (VTI) — divides the masked frame into an N×N grid. Each cell is classified active/inactive across a sweep of thresholds. Real events (snap, tackle pile-up, whistle) produce consistent peaks across many thresholds. This is viewpoint- and zoom-invariant.
- Audio Onset — detects whistle peaks and crowd reaction. Serves as confirmation rather than primary gate (broadcast mix varies).
- Formation Check — low-fps player detection (permissive models only) verifies the pre-snap geometry that distinguishes "about to start a play" from random motion or huddles.
Computed once per source. All analysis restricted to playing surface.
Subtracts pan/tilt/zoom so player motion can be isolated.
Zoom-invariant motion detection across multiple thresholds.
Whistle and crowd reaction as independent confirmation.
RF-DETR verifies pre-snap geometry (~22 bodies, low motion).
Interactive visual of the multi-signal play detection pipeline. Hover steps for emphasis (in production build).
Candidate intervals are fused into start/end timestamps with an explicit confidence score (0–1). Clips are emitted with asymmetric padding: -2.5 s before snap, +2.0 s after end. This padding was measured to reduce bad cuts from 63% (snap-timed) to 8%.
Any clip whose confidence falls below the review threshold is placed in a coach-facing human review queue. The product philosophy is that reviewing 8 uncertain boundaries out of 110 plays is acceptable; a silently wrong cut is not.
field mask, motion compensation, active-cell VTI, and formation check are implemented and calibrated.
Production-quality end-to-end autonomous clipping with <5% human review is on the roadmap.