Skip to main content

FDA AI Device Orders Compared: Rules vs. Real Accuracy

FDA published four rulebooks in six days. Only one contains an accuracy number, and the decision summaries show what that number cost.

VerdictA special control says what a maker had to prove, not how well the product did. Before you buy one of these four, ask for the decision summary and the specificity.

7 min readBy Johnie T YoungLast reviewed
FDA AI Device Orders Compared: Rules vs. Real Accuracy

Source: Pexels stock photo, free to use under the Pexels license.

Say a clinic group sits through two demos in one week: a handheld that scores skin lesions, and a program that reads chest CT scans for lung fibrosis. Both vendors say "FDA De Novo, class II." That phrase tells you less than it sounds like.

On October 30 and November 5, 2026, FDA published four final orders that put algorithm-based devices into the Code of Federal Regulations, two on each date. I read all four, and then the FDA decision summary behind the first product of each type. The orders say what a maker must show. The summaries say what that first product did show, and the two are not the same kind of document.

Four orders, all codifying January 2024 decisions

Every order is a codification of a decision FDA had already made. I found nothing in them that changes what a clinic already using one of these tools has to do.

Device type Regulation Federal Register order First product (De Novo decision)
Skin lesion second-read software 21 CFR 878.1830 Oct 30, 2026, 91 FR 68465 DermaSensor, DEN230008 (Jan 12, 2024)
Lung fibrosis referral software for CT 21 CFR 892.2085 Oct 30, 2026, 91 FR 68467 Fibresolve, DEN220040 (Jan 12, 2024)
MCI prognostic software 21 CFR 882.1454 Nov 5, 2026, 91 FR 69690 BrainSee, DEN220066 (Jan 11, 2024)
Cervical cytology slide system with AI 21 CFR 864.3900 Nov 5, 2026, 91 FR 69687 Genius Cervical AI, DEN210035 (Jan 31, 2024)

Checked November 8, 2026 against the Federal Register orders on govinfo.gov and the FDA De Novo database page for each DEN number. Counting from each decision date, the orders came 33 months and 5 to 25 days later (our arithmetic).

All four say FDA has not exempted the type from 510(k), so the next maker still has to file one. I found no duty on a user in any of the four, only labeling statements the maker has to carry. One more detail: only the cytology order uses the words "artificial intelligence." The other three say software, algorithm or model, though the DermaSensor summary does call its algorithm "artificial intelligence/machine learning."

What the text requires: one number across four orders

I laid the special controls side by side, paragraph by paragraph.

What the order's text requires Skin Lung MCI Cytology
A named comparator in clinical testing Aided users must beat unaided users, (b)(1)(i) "An appropriate clinical control," (b)(1)(ii) None named, (b)(1)(i) Non-inferior to a comparator FDA accepts, (b)(2)(v)
A numeric accuracy floor 90 percent sensitivity, (b)(1)(iii)(A) No No No
Test data stated as independent of training No No Yes, (b)(1)(i) Yes, (b)(2)(iii)(C)
Human factors study Yes, (b)(8) No Yes, (b)(3) Yes, (b)(2)(vi)
Postmarket data Yes, (b)(1) Yes, (b)(1) Yes, (b)(1) No
Cybersecurity or architecture documentation No No No Yes, (b)(1)(vii)(D), (b)(2)(iii)(D)
Patient-facing labeling No No Yes, (b)(5) No

"Yes" and "No" mean the word or the duty appears or does not appear in the order's codified text. Read from the govinfo HTML of each order on November 8, 2026. For postmarket data, all three orders add "unless FDA determines" it is not required.

The skin order has the only number, and it comes with a way out: "at least 90 percent sensitivity of the device output for lesions with high metastatic potential, or an alternative clinical consideration must be provided to justify lower sensitivity." The same item adds that "clinical justification must be provided for the reported specificity." Keep that second sentence in mind.

Federal Register page 68467 with the special controls for the skin lesion device: aided users must beat unaided users, and standalone testing must show at least 90 percent sensitivity for lesions with high metastatic potential or justify lower sensitivity

Source: Federal Register, vol. 91, no. 209, page 68467 (October 30, 2026), cropped by us. U.S. government work.

The comparator row is the one I would read hardest. "Superior" for skin, "non-inferior" for cytology and a vague "appropriate clinical control" for lung are three different bars, and the MCI order names no comparator at all, only a reference standard.

The last thing I searched for was any sentence about what happens when the maker updates the model. I looked in all four texts for "retrain", "predetermined", "change control" and "model update" and found nothing about the device. The FDA De Novo page for each product has a field called "Predetermined Change Control Plan Authorized." It says No on all four. In my reading, none of the four decisions came with a pre-agreed route for model updates. Only the Hologic summary says what the maker did instead: the algorithm "will remain locked" and "will not be continually trained and improved" after authorization. I did not find that statement in the other three summaries.

What the first product showed: the half the orders leave out

The decision summaries are FDA's own write-up of the study behind each product. The table has the headline results, in each device's own test population.

Device Compared with Sensitivity Specificity Test set FDA's own caveat
DermaSensor (skin) Primary care physicians 95.5% vs 83.0% 20.7% vs 54.2% 224 malignant, 1,355 benign lesions Specificity "deemed clinically acceptable" because a missed melanoma is worse
Fibresolve (lung), on-label slices up to 3 mm Expert panel of 5 55% vs 20% 82% vs 95% 137 cases "The Agency disagrees with these endpoints"
BrainSee (MCI) None, reference standard only 72.9% (CI 64.9 to 80.9) 96.3% (CI 92.1 to 100) 198 patients Lower sensitivity bound below the 70% goal; accepted anyway
Genius (cytology), HSIL+ threshold Manual glass slide review 81.5% vs 74.0% 94.8% vs 97.2% 858 positive, 5,022 negative Specificity lower at all four thresholds

Sources: FDA De Novo decision summaries DEN230008 (Table 11, page 18), DEN220040 (page 5), DEN220066 (Tables 4 and 5, page 7), DEN210035 (Table 12, page 21). Read November 8, 2026.

Three of the four beat their comparator on sensitivity and lost to it on specificity (cytology at LSIL+, ASC-H+ and HSIL+). That is my count from the tables, not FDA's framing. The fourth, BrainSee, has no comparator, and its summary says the algorithm "is tuned to favor specificity over sensitivity" instead.

FDA De Novo summary DEN230008, page 18, with Table 11: DermaSensor sensitivity 95.5 percent against 83.0 percent for primary care physicians, specificity 20.7 percent against 54.2 percent

Source: FDA De Novo summary DEN230008, page 18, cropped by us. U.S. government work.

The skin result is the one to sit with. The 90 percent line in the order is met by a device that, in the study, called 20.7% of benign lesions negative, which means it flagged about 79 of every 100 of them. FDA's own summary is fair to the device on one point: at the same 95.5% sensitivity, the primary care physicians' interpolated specificity was 9.0%, against the device's 20.7%. Both things are on page 18.

The lung summary is the most direct page of the four. FDA says splitting sensitivity and specificity endpoints across two comparators "favors the device in both comparisons," and still finds "a small probable benefit" for a device used in a referral pathway to a Multidisciplinary Discussion. It also notes the specificity lower bound of 73%, under the 80% goal, in the on-label slices.

FDA De Novo summary DEN220040, page 5, where FDA writes that the Agency disagrees with the study endpoints, followed by the device and expert panel results by CT slice thickness

Source: FDA De Novo summary DEN220040, page 5, cropped by us. U.S. government work.

For BrainSee, the arithmetic in Table 4 holds: 86 of 118 patients who did progress were caught, which is 72.9%. That leaves 32 who progressed and were labeled non-converters. FDA accepted it because patients with a false negative output "are likely to receive continued follow-up," and it requires a postmarket study of groups the validation study under-represented.

My read: ask for both numbers

The sensitivity figure is the one vendors quote, and in these four summaries it is half a result. The 90 percent in the skin order is the most repeatable line in the whole set of orders, and the specificity beside it decides how many of your patients get sent on. I would not sign anything on a sensitivity number alone, and I would not treat "FDA De Novo, class II" as an accuracy claim, because the orders only fix what must be shown. FDA has already published what was shown.

One more judgment, from the model-update point. The cytology maker locked its algorithm. For the other three I found no statement either way. If a tool you buy keeps learning, nothing in these four decisions covers that version.

What I would ask for this week

This is the desk's judgment, not medical or legal advice. Ask for each item in writing.

  1. The De Novo number and the decision summary. Read the results tables and the benefit-risk section before the next demo. The Fibresolve summary also has a "Study Limitation" paragraph, and it is the most direct one.
  2. Sensitivity and specificity, with confidence intervals and the number of cases, in the population on the product's label, not the whole dataset. Fibresolve's headline across all 300 cases was 41% and 87%, while the on-label slices were 55% and 82%.
  3. The comparator, named: unaided clinicians, an expert panel, a microscope or none.
  4. What changes when the vendor updates the model, and whether the version you test is the version you get. The FDA database lists no change control plan for any of the four.
  5. For the three types with postmarket data in (b)(1), what that data has shown so far.

What I could not confirm

I read the four orders and four decision summaries, not the products' labels or instructions for use, and I did not use any of the tools. The summaries describe studies as authorized in January 2024. I do not know whether any product has since changed, or whether any postmarket study has reported. The De Novo database pages were last updated November 2, 2026, and show the change control plan field as "No" for each product. I did not ask FDA what that field means beyond its label. I did not check whether other makers have used these types as a predicate for 510(k) clearance.

Legal disclaimer

This article is published for informational and educational purposes only. It does not constitute legal, regulatory, or professional compliance advice and should not be relied upon as such. AI governance requirements vary by jurisdiction, industry, and organizational context. Always consult a qualified legal or compliance professional before implementing policies or making decisions with regulatory implications.

About the author

Johnie T Young

Governance & Regulation Desk

Covers AI laws, regulator orders and court rulings, and what each one means for a team of ten. Posts start from the statute or the order itself, with every fact traced to a row in a source log.

  • Starts from the statute or order
  • Every fact traced to a source row
  • Dated checks on each post