The software makes a recommendation. The doctor reviews it, or doesn’t. The patient never knows an algorithm was involved. This is the quiet revolution reshaping clinical medicine.
The economy of prediction
We have built an entire economy around clinical prediction. Startups promise to catch cancers earlier, flag deteriorating patients sooner, identify drug interactions faster. The pitch decks are compelling. The evidence base is thinner than the marketing suggests.
AI in healthcare is often framed as augmentation. But there is nothing augmenting about a system that shapes decisions without accountability.
The pressure to adopt is immense. Deploying AI is cast as innovative, necessary, inevitable. Questioning it is reframed as Luddism or, worse, as resistance to progress that could save lives.
The cost of automation
Clinicians who work with these systems often describe a peculiar form of deskilling. The recommendation appears, confident in its probability score, yet somehow the clinical reasoning that produced it remains opaque. It has been processed, weighted, fitted into models that serve purposes far removed from this patient.
The alert fatigue is predictable. The automation bias, well-documented. But even trust can become a burden when it flows in one direction from systems that cannot explain themselves.
Toward a different practice
There must be ways to harness computational power without surrendering clinical judgment. Ways to build decision support that supports rather than supplants. Ways to deploy AI that do not require patients to become unwitting participants in unvalidated experiments.
These alternatives are harder. They are slower. They do not generate the valuations that drive health tech investment. But they might, in the end, serve both innovation and the people it claims to help.