Edge AI (on-device inference)
Edge AI runs inference on the device where data is captured, such as a camera or an on-premise appliance, instead of sending it to a remote service. Latency drops to the local network, the footage never leaves the building, and the system keeps working when the internet connection does not.
What changes when inference moves to the edge?
| Edge | Cloud | |
|---|---|---|
| Round-trip latency | Local network only | Internet round trip per request |
| Behaviour during an outage | Continues | Stops |
| Where footage travels | Stays on site | Leaves the premises |
| Bandwidth cost | None for inference | Proportional to streams |
| Model update | Deployed to each device | Central, immediate |
The bandwidth arithmetic decides most video deployments before privacy is discussed. Sending fifty camera streams to a remote service is a continuous upload commitment; running the model where the frames already are turns it into occasional metadata.
What hardware does edge inference need?
- An accelerator sized to the number of streams and the frame rate each one is analysed at, not to the camera count alone.
- Enough memory to hold the model and the enrolled gallery, which for face recognition grows with enrolment.
- Thermal headroom for a device that runs at load continuously rather than in bursts.
- A physically secured enclosure, because a device holding templates is now part of the attack surface.
Sizing is driven by analysed frames per second rather than by cameras. A site that analyses two frames per second on each of forty cameras is a smaller workload than one analysing fifteen frames per second on ten, which is why a camera count on its own does not specify a deployment.
When is the cloud the better choice?
Edge is not automatically correct. Workloads that are occasional rather than continuous, that need a model far larger than any on-site device can hold, or that must aggregate across many locations in real time are better served centrally. The honest position is that placement follows the workload, and most real estates end up with both.
The one requirement that settles the question is a legal or policy constraint on data leaving the site. Where that exists, the placement is decided before any performance comparison begins.
The hybrid arrangement most sites arrive at puts inference at the edge and everything else centrally: models are trained and packaged centrally, devices analyse locally, and only events, thumbnails and health metrics travel back. That keeps the bandwidth and privacy properties of edge processing while retaining one place to manage fleets from.
Frequently asked questions
- What is edge AI?
- Edge AI runs inference on the device where data is captured, such as a camera or an on-premise appliance, rather than sending data to a remote service. The result is local-network latency, footage that stays on site, and operation that continues through an internet outage.
- What is the difference between edge AI and cloud AI?
- Placement. Edge AI processes data where it is captured; cloud AI sends it to a remote service. Edge wins on latency, on bandwidth for video, and on keeping data on site; cloud wins on model size, on central updates and on aggregating across locations in real time.
- Does edge AI mean the system never connects to the internet?
- Not necessarily. Most edge deployments still connect for management, alerting and model updates while keeping inference and raw data local. A deployment where nothing connects at all is an air-gapped one, which is a stricter requirement and a separate design.
- How much hardware does edge video analysis need?
- Sizing follows analysed frames per second, not camera count. Forty cameras analysed at two frames per second is a smaller workload than ten analysed at fifteen. Memory must also hold the model and, for face recognition, the enrolled gallery, which grows with enrolment.
- Is edge AI more private than cloud AI?
- It removes one specific exposure: the raw data never leaves the premises, so there is no third-party copy and no cross-border transfer to justify. It does not by itself address retention, access control or who can query the results, which remain policy questions wherever the inference runs.
- Can models be updated on edge devices?
- Yes, though it is a distribution problem rather than a deployment switch. Updates are pushed to each device, usually with staged rollout and a rollback path, and in air-gapped installations they are carried in on physical media under a change-control process.