Summary
Most commercial veterinary AI products publish no performance data at all. A 2026 audit of 71 products sold in North America found that 45 of them (63.3%) disclosed none of the validation or safety metrics it looked for, and only one vendor described the species, breed, age and sex mix of its training data. The ACVR and ECVDI position statement on AI goes further: it says no commercially available diagnostic imaging product currently meets its standards. That leaves the checking to the buyer. These six questions give a radiologist a structured way to do it before any contract is signed.
What did the 2026 transparency audit find?
David Brundage at the University of Wisconsin-Madison audited the public information behind 71 commercial veterinary AI products: 47 generative and ambient tools, 19 diagnostic imaging tools and 5 specialised tools. Each product was scored against 25 items covering data provenance, performance and validation, safety and risk management, and documentation.
Three findings matter to anyone buying imaging AI:
- 45 of 71 vendors (63.3%) disclosed no metric from the framework. For those products there was no public information on how the model was built, validated or monitored.
- Imaging tools did better, but not by much. Of the 19 diagnostic imaging vendors, 36.8% cited peer-reviewed evidence and reported specific accuracy metrics. The rest did not.
- One vendor (1.4%) disclosed the signalment of its training data. Without that, nobody outside the company can judge whether a model will behave the same way on a cat, a giant breed or a geriatric patient.
The audit also notes that North America has no federal pre-market approval requirement for AI tools used in veterinary medicine. Nothing outside the sale itself forces a vendor to publish these numbers, so the questions below are often the only way to get them.
Question 1: Which species, regions and modalities does the validation set cover?
A model trained mostly on canine thoracic radiographs has seen few feline abdomens. That is a property of the training data, and no software update fixes it unless the data changes. Ask the vendor to list the species, anatomical regions, modalities and conditions in the validation set, with case counts for each.
"Trained on hundreds of thousands of images" is not an answer. If the vendor cannot break the number down, either the data does not cover your caseload or nobody characterised it closely enough to know.
Question 2: Can you provide performance figures for each classifier?
An overall accuracy figure hides how a tool performs on less common findings. A product can score 95% overall by handling frequent, easy findings well while missing a rarer finding that matters clinically. The 95% is true and still misleading.
Ask for sensitivity, specificity and area under the curve for each classifier separately, along with the reference standard used to decide what was correct. If the vendor can only give an aggregate, you cannot interpret it. This is the same evidence appraisal a radiologist applies to a journal paper, applied to a purchase.
Question 3: How does the tool handle breed and body-type variation?
What counts as normal on a thoracic radiograph of a giant breed differs from a toy breed or a cat. Cardiac silhouette size, tracheal position and diaphragmatic contour vary enough that a model validated on one body type can be systematically off on another.
Ask whether the validation set included the range of breeds, body conformations and ages your service actually reads. Ask too what the tool does with a patient outside that range. A model can be confidently wrong on an unusual patient, and a confident wrong answer is harder to catch on a busy overread than an obvious failure.
Question 4: What happens after go-live?
Validation describes performance on one dataset at one point in time. When a practice changes its DR panel, adjusts its exposure protocols or takes on a different caseload, performance can drift with no visible sign at the workstation.
The ACVR and ECVDI statement treats this as a deciding factor. It says systems that "do not allow postimplementation monitoring as defined by good machine learning practices" should not be used in veterinary practice. Ask the vendor:
- How do you track sensitivity and specificity once the tool is in clinical use?
- What change in performance triggers a review or retraining?
- How are model versions numbered, and how are users told when one changes?
- How can a radiologist report an error or a near miss, and what happens to that report?
If the answers are not documented, ask for them in writing as part of the contract.
Question 5: What exactly does each named tool do, and what evidence stands behind it?
One product name often covers several classifiers with different validation histories. A thoracic AI package might include one well-studied detector and another with no published evidence at all, and the marketing will not separate them.
Because there is no veterinary pre-market approval to lean on, "is it cleared?" cannot be the test. Ask instead for a list of every algorithm in the product and, for each one, whether it has peer-reviewed validation, internal validation only, or none. Then ask where each one appears in the workflow and whether a radiologist can tell which classifier produced a given flag. A named tool with a defined input, a defined output and a version logged against each study can be audited. "AI-assisted reading" as a general feature cannot.
We hold NewLumen to the same test. It has two named AI features, each with a stated scope. Priority Triage flags urgent studies in the worklist and does not diagnose. Report Assist pre-structures report sections by species and modality, and the radiologist writes and signs the report. The AI Marketplace bills each feature per clinic, so a service switches on only what it has decided to use. Put these six questions to us as well.
Question 6: Where does the radiologist sit in the workflow?
The ACVR and ECVDI statement says "AI systems should always be used with a qualified veterinary professional in the loop," and that "board-certified radiologists are best suited to evaluate the output of computer-aided diagnostic tools."
Ask the vendor to show you, in the product rather than a slide deck, where a qualified person reviews the AI output and whether anything can reach the referring veterinarian before that review happens. Some products return AI-generated findings straight to the practice. Know which kind you are buying.
In NewLumen the report goes out when the radiologist signs it, by link email, PDF or automated delivery to a connected practice management system. A finalised report cannot be edited; any change goes in an addendum with a reason, linked to the original.
Before you sign
- Get written answers to all six questions.
- Check the vendor's claims against the papers it cites, and note which classifiers have no citation.
- Run a pilot in which a radiologist reviews every AI output independently before wider rollout, and record where the tool and the radiologist disagree.
- Agree in the contract how model changes will be communicated and how errors are reported.
For a wider look at what imaging AI does well today and where the evidence runs out, see AI in veterinary radiology: hype vs real clinical value and Can AI read veterinary X-rays?.
FAQ
What does the 63.3% figure actually measure?
It comes from Brundage's 2026 audit in Frontiers in Veterinary Science, which reviewed the public information behind 71 commercial veterinary AI products in North America. 45 of those vendors (63.3%) disclosed none of the 25 validation, safety and documentation items the audit scored. The figure covers all 71 products, including generative and ambient tools, not only imaging. Among the 19 diagnostic imaging products, 36.8% cited peer-reviewed evidence with specific accuracy metrics.
What does the ACVR and ECVDI position statement say about AI?
The statement, published in JAVMA in 2025, says AI systems should always be used with a qualified veterinary professional in the loop and that board-certified radiologists are best suited to evaluate AI output. It states that no commercially available diagnostic imaging product currently meets its standards, calls for unbiased third-party evaluation, and says systems that do not allow post-implementation monitoring should not be used. It is guidance from the two specialist colleges, not a regulation.
Is veterinary imaging AI cleared by the FDA?
There is no federal pre-market approval requirement for AI tools used in veterinary medicine in North America, so a veterinary AI product does not go through the clearance process that human medical AI does. Ask for published, peer-reviewed validation for each algorithm instead.
How can I tell whether an AI tool is properly scoped?
Ask the vendor to list every AI function by name and show where it runs in the workflow. A scoped tool has a defined input, such as lateral and VD thoracic radiographs, a defined output, such as a flag with a confidence level, and a version number recorded against each study. If the vendor describes "AI-assisted reading" without naming tools or showing where they appear, ask for a workflow map before going further.
What post-deployment monitoring should a vendor provide?
At minimum, the vendor should track performance once the tool is in clinical use, tell you when sensitivity or specificity moves past an agreed threshold, announce each model update and what changed, and give radiologists a way to report errors. Without monitoring, the first sign of drift may be a missed finding.
Sources
- Brundage D. A systematic audit of transparency and validation disclosure in commercial veterinary artificial intelligence. Front Vet Sci. 2026;13:1761038. doi:10.3389/fvets.2026.1761038
- Appleby RB, DiFazio M, Cassel N, Hennessey R, Basran PS. American College of Veterinary Radiology and European College of Veterinary Diagnostic Imaging position statement on artificial intelligence. J Am Vet Med Assoc. 2025;263(6).