Skip to content

Accuracy and liveness at Ayonix

Ayonix publishes no accuracy percentage for its face recognition products, because a figure without its threshold, dataset and demographic breakdown cannot be reproduced. Accuracy is established for a specific deployment by a pilot on the site's own cameras, and liveness is established by testing against named presentation attack instruments.

Why does Ayonix not publish an accuracy percentage?

A percentage is only a fact if it can be reproduced, and reproducing one requires the threshold it was measured at, the dataset it was measured on, the gallery size, the image quality and the demographic breakdown. None of those appear in a headline number, and changing any one of them moves the result far enough to describe the same system as excellent or unusable.

  • Threshold: the setting that trades false accepts against false rejects, and the one most often omitted.
  • Dataset: visa-quality portraits and a corridor camera produce different results from identical software.
  • Gallery size: an identification error rate measured at a thousand enrolments does not transfer to a million.
  • Demographics: error rates vary across groups, and an aggregate figure averages that variation away.
  • Capture conditions: pose, lighting and occlusion move results more than most differences between algorithms.

The practical test of any published figure is whether a buyer could repeat the measurement from what is printed next to it. Where they could not, the number is a marketing claim in the shape of a result.

How is accuracy established for a specific deployment?

  1. Survey the cameras: measure the pixels across a face at the point people actually pass, not at the centre of the frame.
  2. Enrol under the conditions the site will use, so enrolment quality reflects the real process rather than a staged session.
  3. Run for a defined period on live streams, recording every comparison rather than only the alerts.
  4. Count both errors: people the system failed to match, and matches to the wrong person. One without the other describes nothing.
  5. Tune the threshold against what each error costs at that site, and record where it was left and why.
  6. Repeat the measurement after any change to camera placement, lighting or enrolment process.

The output of that process is a number that describes one installation, which is the only kind of accuracy figure a buyer can act on. It is not comparable between sites, and Ayonix does not present it as though it were.

What independent evidence exists about face recognition algorithms?

The strongest independent evidence in this field comes from the evaluations run by the US National Institute of Standards and Technology, which measures submitted algorithms on sequestered data under fixed conditions and publishes the results whatever they show. NIST does not endorse vendors and issues no certification, so a report is evidence about an algorithm rather than a recommendation of a supplier.

A laboratory result and a site result answer different questions. The evaluation says how an algorithm behaved on that data; the pilot says what this installation will do with these cameras, this lighting and these people. Both belong in a procurement decision, and neither substitutes for the other.

How is liveness established?

Liveness claims are only meaningful against named attack instruments. ISO/IEC 30107-3 defines how presentation attack detection is tested and reported, so the question to ask is which instruments were presented, at what level, and by whom - not what detection rate was reported against unnamed attacks.

StatedWhat it tells a buyerIf absent
Attack instruments usedWhich attacks the result coversThe result covers nothing specific
Who performed the testingWhether it was independentIt was probably self-assessment
Date of testingWhether it reflects the current buildIt may predate the shipped version
Where the check runsWhether compression discarded the cuesLaboratory conditions may not apply on site
What a liveness claim should state, and what it means when it does not.

ISO/IEC 30107-3 compliance describes a test methodology, not a pass mark. There is no certificate to hold, so a vendor claiming one is describing something that does not exist.

What should a buyer ask any face recognition vendor?

  • Which algorithm identifier did you submit to an independent evaluation, and on what date?
  • At what threshold is your quoted figure measured, and on which dataset?
  • What is the demographic breakdown behind the aggregate you publish?
  • Which presentation attack instruments has liveness been tested against, and by which laboratory?
  • Will you run a pilot on our cameras, and will you report the failures as well as the matches?
  • Where do templates live, and what happens to recognition when the internet connection drops?

A vendor that can answer all six has evidence. A vendor whose answer to any of them is a percentage has restated the claim rather than supported it, and that is the distinction this page exists to make.

Frequently asked questions

Why does Ayonix not publish an accuracy rate?
Because an accuracy percentage without its threshold, dataset, gallery size and demographic breakdown cannot be reproduced or compared, and a buyer cannot act on it. Ayonix points to independent evaluations for algorithm comparison and to a pilot on the buyer's own cameras for the figure that describes their site.
How accurate is Ayonix face recognition?
Accurately answering that requires naming a threshold, a dataset and a deployment, which is why the honest answer is a pilot rather than a number. Ayonix runs evaluations on the customer's own cameras and reports both error types: people missed, and people matched to the wrong identity.
Is Ayonix NIST certified?
No company is. NIST runs evaluations and publishes reports on how submitted algorithms performed; it does not endorse vendors, issue certificates or maintain an approved list. Any vendor claiming NIST certification is describing something that does not exist.
What is a realistic face recognition pilot?
A defined period on live streams from the site's own cameras, with enrolment done the way the site will actually do it, every comparison recorded rather than only the alerts, and both error types counted. A pilot that reports only successful matches has measured half of the system.
How should a liveness claim be evidenced?
By naming the presentation attack instruments tested, the level, the laboratory and the date, following ISO/IEC 30107-3. A detection rate quoted against unnamed attacks says nothing, because the result only covers the attacks that were actually attempted.
Does a high independent evaluation rank guarantee site performance?
No. It shows how an algorithm behaved on that dataset under laboratory conditions. Site results depend on camera placement, pixels across the face, lighting, throughput and gallery size, none of which an evaluation covers. The rank narrows a shortlist; the pilot decides.

Jan Mocary β€” Chief Technology Officer, Ayonix AI

Leads engineering for Ayonix face recognition and the ATLAS agent platform, including their on-premise and air-gapped deployment modes.