The AI Label Is Coming Off. Now What?

Every week I read a lot of AI news. Most of it is noise. This week three items landed together, and they tell one story: the hard part of AI in healthcare is not the demo. It is what happens after go-live, and who is still asking questions.

1. The federal “nutrition label” for EHR AI is going away

ONC’s HTI-5 final rule removes the model-card and risk-management requirements for predictive decision support in certified EHRs. It takes effect December 28, 2026.

ONC’s reasoning is that it found no evidence the labels were being used or that they helped patients. That may be fair. But the rule itself notes that a large majority of commenters opposed removing them, and groups like the ACR asked for another way to keep transparency.

Here is what changes in practice. Hospitals can no longer count on a federal requirement to surface how a predictive tool was built and tested. If they want that information, they will have to ask vendors for it directly. We do not yet know whether the major EHR vendors will keep publishing model cards on their own.

My take: a label nobody reads is a design problem, not proof that transparency doesn’t matter. Health systems that build their own “show me the model” habit now will be in a better spot than those waiting for a rule.

2. An AI agent for oncology notes: faster, cheaper, and a reality check

A study in Nature Cancer tested an AI agent that reads oncology notes looking for immunotherapy side effects. It went through three phases, including a randomized crossover with real trial staff. With the agent, staff were faster, more accurate, and more consistent, at about two cents per note.

That is impressive. The part I want everyone to notice is the drop. Performance measured as F1 was 0.92 on past records. In live silent use it fell to between 0.72 and 0.79. Humans still verified every output.

The study did not report effects on patient outcomes, and we don’t have results from other hospitals yet.

This is the gap between the lab and the floor, measured honestly. Any AI tool you see should come with that second number, not just the first.

3. AI inbox drafts: each message faster, the whole inbox not so much

JAMA Network Open looked at nearly a million patient portal responses at one academic center. When a clinician used an AI draft, that reply was about 30% faster (29.5%). But adopting the tool was linked to 8.6% slower overall turnaround, and it did not cut after-hours inbox work.

It is observational and from one system. The vendor and model were not named, and the explanation isn’t clear yet.

Still, the lesson is clear. Speeding up one step does not automatically fix a workflow. The burnout promise of AI inbox tools has to be measured, not assumed.

The thread that ties them together

Transparency rules are loosening. Live performance is lower than lab performance. Local speed gains do not always become system gains. Put together, the responsibility is shifting to the people inside health systems to keep asking: How was this built? How does it do here, today? Did it actually make our day better?

AI can help. It just doesn’t grade itself.

This is educational commentary, not medical advice.

Sources

• ONC HTI-5 final rule, Federal Register
• Nature Cancer, AI agent for immunotherapy adverse events
• JAMA Network Open, AI-drafted portal replies

Originally published on my Substack. Planning a board or leadership session on AI governance? Explore my healthcare AI keynotes.