Healthcare AI for Boards: 10 Questions Every Board Member Should Ask

By Harvey Castro, MD, MBA

Board members keep telling me the same thing in different words. They know AI belongs on the agenda. They do not know what they are supposed to ask, and they do not want to spend their one question of the meeting on something that makes them sound naive.

Good news. The useful questions are not technical. Every one of them below is a governance question that a director without a computer science background can ask, and each one has a wrong answer you will recognize immediately.

I have written a separate piece on the operational framework, which maps AI oversight onto medical staff credentialing and privileging. This one is written for the boardroom, where the job is fiduciary rather than operational.

1. How many AI tools are running in this organization today, and who keeps that list?

Start here, because the answer sets the tone for everything else.

A good answer names a number, names an owner, and names the last time the inventory was refreshed.

A bad answer is a number with no owner. In a Black Book Research survey of 182 hospital leaders in late 2025, 33 percent said unclear ownership across IT, quality, safety, and compliance slowed governance progress, and only 29 percent had enforced policies covering AI model inventory, lineage, and sign-offs (Black Book Research release).

2. Which of those tools touch a clinical decision, and which touch a payment decision?

The risk profile differs sharply between the two, and so does the law.

These laws cover different decisions and actors.

Indiana's HB 1271 added a medical-necessity downcoding provision effective July 1, 2026: an insurer may not use automation, including AI, as the sole basis without an employee or contractor reviewing the covered person's medical record. That clause does not specify a licensed clinician, and the chapter excludes Medicaid and Medicaid managed-care organizations.

Maryland's HB 1563, effective June 1, 2026, concerns emergency-room coverage decisions and post-acute-care studies; it is not a general AI human-review mandate.

Utah's SB 319, effective January 1, 2027, addresses adverse preauthorization determinations involving clinical or medical necessity. The decision-maker must know the condition or consult a knowledgeable specialist, exercise independent medical judgment subject to the specified specialist-consultation exception, and not rely solely on another source's recommendations. Separate provisions govern licensed review of specified appeals.

Georgia's SB 444, effective January 1, 2027, requires a qualifying natural person to conduct utilization review with a clinical peer participating before AI or software issues an adverse determination; the software cannot supersede that peer's judgment.

If the organization operates across state lines, ask how the policy accounts for the differences.

3. What did local validation show before each clinical tool went live?

This is the single highest-yield question a director can ask.

A good answer cites performance on the organization's own patients with a date attached.

A bad answer cites the vendor's published figure. The gap between those two can be enormous. The Epic Sepsis Model reached hundreds of US hospitals before independent evaluation showed a hospitalization-level area under the curve of 0.63, with 67 percent of sepsis cases missed and 18 percent of hospitalizations hypothetically flagged. The analysis was retrospective, used a score threshold of at least 6 for these misses and hypothetical flags, and excluded hospitalizations eligible for actual alerts. According to PubMed, that is Wong A, Otles E, Donnelly JP, et al., JAMA Internal Medicine, 2021 (DOI).

4. Who signed the approval, by name?

Accountability without a name is not accountability.

Ask whether a practicing clinician signed, and whether that person still works here. Turnover quietly orphans approvals.

5. What is our override rate, and which direction is it moving?

Override rate is a vital sign for the whole program.

Very low override rates warrant review; they do not alone prove automation bias. Ask whether any dashboard, quality metric, or compensation formula penalizes a clinician for overriding a model. Those incentives deserve scrutiny as part of oversight.

6. Have we measured performance across our patient populations?

Stratify by language, payer, race, and site.

A model that performs on a commercial population and fails on a Medicaid population is a quality failure first, a reputational failure second, and a legal failure third. The board should see the stratified numbers, not a summary statistic.

7. What is our process for turning a model off, and have we ever used it?

Most AI policies have no off switch and no precedent for using one.

A good answer names the person with authority, describes the trigger, and cites at least one tool the organization has retired. If nothing has ever been retired, ask why a portfolio that only grows should be assumed to be working.

8. If a regulator or payer requested a complete AI audit trail tomorrow, how many days would it take?

Ask for a number in days.

Only 22 percent of hospital leaders in the Black Book survey were highly confident they could produce that trail within 30 days, and among small hospitals the figure fell to 15 percent. This is also now a certifiable domain: The Joint Commission launched its voluntary Responsible Use of AI in Healthcare certification on June 1, 2026, covering governance, data management, risk and bias reduction, monitoring and validation, and transparency with education and training (Joint Commission announcement). Ask management whether the organization intends to pursue it and, if not, what the reasoning is.

9. What percentage of the IT and quality budget funds AI oversight?

The median across surveyed hospitals is 4.2 percent of 2026 IT and quality or safety budgets, with large systems at 6.8 percent and smaller hospitals at 2.3 percent. Only 26 percent of those organizations plan to raise governance budgets by two or more percentage points in 2026, and 18 percent plan no increase at all.

If your organization sits well below the median while its AI portfolio grows, the board is carrying an unfunded control.

10. What is our exposure if a vendor changes a model without telling us?

Silent model updates are the risk directors most often miss. A vendor retrains, ships an update, and the hospital is running a materially different tool that nobody re-approved.

Ask three follow-ups. Do our contracts require notification of model changes. Do they require ongoing performance reporting from the vendor. Can we retrieve our data and audit logs if we terminate.

Note that 41 percent of surveyed hospital leaders named insufficient vendor documentation as their primary obstacle to audit readiness. That is a contracting failure, and contracting is squarely within the board's remit.

The two mistakes I see boards make

Delegating the whole subject to the IT committee. Clinical AI creates quality risk, workforce risk, liability risk, and reputational risk. Routing it to technology governance alone means the people who understand patient safety never see it. Put it on the quality committee, with IT and legal in the room.

Treating one education session as oversight. A guest speaker, a slide deck, and a nodding board is not a control. I say that as someone who is regularly the guest speaker. A keynote can change the framing of a conversation. Only a standing agenda item with numbers attached changes the organization.

There is a third one worth naming, because it is subtler. Boards sometimes decline to ask hard questions about AI out of a worry that they will look uninformed or appear to obstruct innovation. The directors I respect most ask the plain question anyway. In my experience, when one person in the room admits they do not follow something, three others exhale.

How to use these in an actual meeting

Do not fire all ten at once. Management will produce a deck, and a deck is how questions go to die.

Pick three for this quarter: the inventory, local validation, and the off switch. Ask for written answers before the meeting. Put AI oversight on the quality committee's standing agenda rather than treating it as an annual education topic. Then ask the same three questions next quarter and watch whether the answers improve.

Directors are not expected to understand model architecture. They are expected to know what the organization owns, who is accountable for it, how performance is monitored, and what happens when it fails.

That has always been the job. The subject matter is just newer than most of the rest of the agenda.

Harvey Castro, MD, MBA (DR GPT™) is a board-certified emergency physician, 5x TEDx speaker, Advisor to the CEO of Phantom Space, and author of books on AI, medicine and leadership.

Related DR GPT™ reading

• AI Governance in Healthcare: A Physician's Framework for Boards and Executives
• What Every Hospital CEO Should Know Before Buying AI

More from DR GPT™: who DR GPT™ is, healthcare AI keynotes, board advisory, speaking, TEDx talks, books, and the media kit.

Book Harvey Castro, MD, MBA, known as DR GPT™, for a healthcare AI keynote, board retreat, or executive session. Start the conversation.

Worksheet version: September 23, 2026. Use your browser’s Print command to save a copy for the meeting.

A decision worksheet for your next AI proposal

Before approving an AI purchase or expanding a pilot, ask management to complete one decision record. Use a separate record for each proposed use. A tool that helps an administrator draft a document deserves a different discussion from one that influences a clinical decision. Describe the actual workflow before discussing the product.

This original DR GPT™ worksheet turns the questions above into a practical meeting document. It draws on the voluntary NIST AI Risk Management Framework, which organizes risk management around Govern, Map, Measure and Manage. It is a discussion aid, not a NIST assessment or certification.

Complete this record before the decision

Copy the table into your board materials. Management should supply the evidence and recommend an action within the organization's approval process. The board or delegated committee should review decisions appropriate to its authority. Put unresolved items in writing with a named person and a due date.

AI proposal decision record

Field	What to record
Decision requested	Explore, pilot, expand, continue or stop. State the budget, duration and approval authority.
Use case	The task, intended users, permitted data, expected benefit and activities the tool must not perform.
Accountable owner	One named owner, the operational team and the person who can pause use.
Evidence	Links to relevant evaluations, known limitations, product version and vendor change notification arrangements.
Local validation	Test setting, comparison method, acceptance criteria, results, unresolved failures and test date.
Human review	Who checks the output, what they check, when they check it and how they reject or correct it.
Escalation	Where users report problems, who responds and how work continues during an investigation.
Stop criteria	Specific failures or changes that trigger a pause, plus the conditions for restarting.
Review date	Next review, reporting owner and changes that require an earlier review.

NIST's AI RMF Core addresses documented responsibilities, evaluation in conditions similar to deployment, ongoing monitoring and the ability to disengage an unsuitable system. The table applies those ideas to an organizational decision; each team still needs criteria suited to its own task and risks.

Hypothetical example: drafting meeting summaries

Consider a team testing AI to draft summaries of routine facilities planning meetings. This is an illustrative scenario, not a reported deployment. The proposed pilot uses synthetic meeting notes and excludes patient information, personnel matters and confidential board discussions. The operations director owns the pilot; a designated meeting secretary reviews every draft before anyone shares it.

The team compares the drafts with summaries written by its staff. Reviewers check whether the tool invents decisions, omits disagreements or assigns tasks to the wrong person. They also record correction time. Before testing, management defines which errors would make the tool unacceptable. Any invented approval or unintended disclosure triggers a pause and investigation. Ordinary note taking remains available throughout the pilot.

At the scheduled review, the team presents the corrected drafts, error log and time measurements. Management can then recommend continuing, changing the scope or ending the pilot. A polished demonstration alone would leave these questions unanswered.

Bring the decision into the room

For your next meeting, choose one live proposal and ask its owner to complete the record in advance. Use the discussion to identify missing evidence and assign the next decision. Accountable Intelligence: Who decides, and who answers is Harvey Castro's keynote for boards and executive teams. Request a keynote or executive session built around your audience's AI governance questions.