Patient trust in AI starts with human accountability

DR GPT™ perspective

Trust is not a property of a model. It is a property of a system, and systems have names attached to them.

The instinct when trust is low is to improve the model. Higher accuracy, better benchmarks, a more impressive demo. That instinct is wrong, and the arithmetic shows why. A model that is right 95 percent of the time, sitting inside a system that reliably catches the other 5 percent, is safer than a model that is right 99 percent of the time where nobody can tell which answers are the wrong ones. Detectability beats accuracy. Buy for detectability.

The practical form of that is a named human on every output. Not a committee, not a policy, and not the phrase the algorithm recommended. A person, with three things: a name, the authority to override, and enough time to disagree. Strip any one of those and what is left is not oversight. A clinician with eleven seconds and no authority to push back is liability laundering, and patients feel the difference even when they cannot name it.

Trust also behaves asymmetrically, which is the part institutions underestimate. Deposits are slow and withdrawals are instant. One visible failure can damage an entire category of care for years, well beyond the organization that caused it. That is why caution here is not timidity. It is arithmetic.

Originally published on KevinMD. Read the full essay at the link below, or see how I build these arguments for a room as a speaker.