Skip to content

Video analytics (YOLO): resource usage

The Video analytics (YOLO) trigger analyses the cameras in the automation-vision-worker service. This page sums up what it uses and how to work out the server it needs.

These are approximations. The values were measured on one specific machine (see How it was measured) and can vary ±30–50 % with the processor, the scene and the concurrent load. Use them for sizing, then confirm with the live camera status in the node editor (actual FPS and analysis time).

Quick summary

  • What weighs most is the model inference. Its cost grows with the model size and with the FPS you set.
  • Each model is loaded once in memory, however many cameras use it.
  • Each camera adds a video decoder (~55 MB of RAM and very little CPU on the sub-stream).
  • Going from 5 to 2 FPS cuts the model's CPU usage by more than half.

Usage per model

Latency of one picture (preparation + inference + results) and the memory a model adds once loaded.

Model Input File Model RAM Latency (1 core) Latency (4 cores) Cores per camera at 5 FPS Cameras per core at 5 FPS
YOLOX Nano 416 px 3.7 MB ~35 MB ~24 ms ~12 ms ~0.14 ~7
YOLOX Tiny 416 px 20 MB ~80 MB ~80 ms ~38 ms ~0.43 ~2
D-FINE Nano 640 px 15 MB ~130 MB ~150 ms ~76 ms ~0.77 ~1.3
YOLOX S 640 px 36 MB ~180 MB ~315 ms ~123 ms ~1.6 0.6
D-FINE S 640 px 42 MB ~255 MB ~425 ms ~200 ms ~2.1 0.5
RT-DETRv2 R18 640 px 81 MB ~335 MB ~710 ms ~350 ms ~3.6 0.3
YOLOX M 640 px 101 MB ~340 MB ~760 ms ~390 ms ~3.8 0.25

How to read it:

  • Latency (1 core) is the actual CPU cost per picture. Cores per camera = latency × FPS ÷ 1000, plus the decoder.
  • Latency (4 cores) is how long one picture takes when the model uses 4 threads. It improves the response time of a camera, not the total number of cameras: the total CPU used is the same or a little higher.
  • Cameras per core refers to physical cores. Hyper-Threading threads add little to inference.
  • The 640 px models and the transformers (D-FINE, RT-DETR) on CPU are for few cameras or low FPS. For many cameras with those models, use a GPU.

Which model?

Need Recommended model
Many cameras on a modest server, objects close by (corridors, doors, checkouts) YOLOX Nano
General use on CPU (perimeters, medium parking lots) YOLOX Tiny
Crowds (fewer duplicate boxes) on CPU, few cameras D-FINE Nano
Small or distant objects (large perimeters, cars far away) YOLOX S or D-FINE S, with a GPU for several cameras
Maximum accuracy RT-DETRv2 or YOLOX M, with a GPU

Usage per camera (on top of the model)

Item CPU RAM
Decoding the sub-stream (640×360, 15 fps) ~2 % of a core ~55 MB
Decoding the main stream at 720p (15 fps) ~8 % of a core ~60 MB
Decoding the main stream at 1080p (15 fps) ~10 % of a core ~70 MB
WebRTC connection, object tracking and rules < 1 % ~5 MB

Recommendation: use the sub-stream. The model works at 416–640 px, so the main stream only helps with very small objects.

Fixed usage of the service

Item Approximate value
Worker process with the inference runtime loaded ~35–60 MB
Docker image (no models) ~750 MB on disk
Installed models Their file size (3.7 to 101 MB each)
Annotated picture per event 70–150 KB, kept 7 days by default (VISION_SNAPSHOT_RETENTION_HOURS)

Disk example for pictures: 500 events a day × 100 KB × 7 days ≈ 350 MB.

How to size the server

CPU (physical cores)

cores ≈ Σ per camera [ latency_1_core_ms × FPS ÷ 1000 + 0.02 ]  × 1.3

The 1.3 is a margin: with every core busy, the processor drops its turbo frequency.

RAM

RAM ≈ 60 MB + Σ models in use (model RAM) + cameras × 60 MB  (+30 % margin)

Examples

Scenario CPU calculation Cores RAM
16 cameras, YOLOX Nano, 5 FPS 16 × (24×5/1000 + 0.02) × 1.3 ~3 ~1.3 GB
8 cameras, YOLOX Tiny, 5 FPS 8 × (80×5/1000 + 0.02) × 1.3 ~4.5 ~0.8 GB
8 cameras, YOLOX Tiny, 2 FPS (dwell/occupancy) 8 × (80×2/1000 + 0.02) × 1.3 ~2 ~0.8 GB
4 cameras, YOLOX S, 2 FPS (distant perimeter) 4 × (315×2/1000 + 0.02) × 1.3 ~3.4 ~0.6 GB
2 cameras, D-FINE S, 5 FPS 2 × (425×5/1000 + 0.02) × 1.3 ~5.6 ~0.6 GB

Settings that control usage

Variable Default Effect
Trigger FPS 5 Linear on the model's CPU
VISION_MAX_INFER half the cores Parallel inferences. More parallelism = more cameras served, if there is free CPU
VISION_THREADS min(4, cores) Threads per inference. Keep VISION_MAX_INFER × VISION_THREADS at or below the physical cores
VISION_MAX_STREAMS 32 Cap on simultaneous cameras. Cameras beyond it are not analysed (logged)
VISION_FRAME_WIDTH 640 Width the picture is decoded to. Raising it only helps 640 px models with small objects

Signs the server cannot keep up: the camera status shows actual FPS below the configured value, or an analysis time far above the table. Lower the FPS, use a smaller model, disable cameras, or give it more CPU or a GPU.

GPU

There is an NVIDIA image (Dockerfile.vision.gpu, CUDA 12 + cuDNN 9, VISION_EP=cuda). There are no GPU measurements yet. As a general reference, onnxruntime + CUDA usually runs 640 px models in a few tens of milliseconds per picture, which allows tens of cameras per GPU. Measure it on the real hardware before committing to a capacity.

How it was measured

  • Hardware: Intel Core i9-11900H (8 cores / 16 threads, laptop), Linux container (Docker Desktop, WSL2).
  • Software: onnxruntime 1.29 (CPU, the one shipped in the worker image); ffmpeg 4.4 from Ubuntu 22.04.
  • Test picture: a street photo scaled to 640 px wide, as the worker decodes it. Median of 40 runs after 3 warm-up runs.
  • Memory: process RSS before and after loading the model and analysing (one process per model).
  • Decoder: ffmpeg with the exact arguments the worker uses, on a 30 s H264 clip at 15 fps.
  • Reproducible: the TestBenchmarkCatalog test in the module.automation repository (backend/pkg/vision/infer/bench_test.go).

Servers with lower-clocked Xeon or EPYC processors can be up to 30–50 % slower per core, even with many more cores.