AI to Enhance Care

Session Focus

Coburn introduced the session as confronting AI in the clinic directly: decision making and decision support.

Bostwick, a physician and medical geneticist turned venture capitalist, framed the shift. Over recent years AI has transformed clinical notes and administrative burden. Recently it has crossed a new threshold into clinical decision-making. The panel’s question: what happens when a non-deterministic model enters high-stakes patient care, and how is rapid innovation balanced against safety, liability and trust.

Is Anything Ready for Autonomy

Rebecca Mishuris MD answered plainly that she is not sure the field is there for anything yet. Many workflows exist or are emerging, but there is always a human at the end, if not in the middle or the beginning.

Even a patient-facing triage chatbot has a person scheduling the visit at the end, and the system escalates concerning conditions or symptoms to a human. Mass General Brigham has nothing truly fully autonomous or fully agentic in clinical care. Call routing and self-scheduling are different, and she noted there is actually not much AI in those processes.

Her more interesting question was whether the field should get there, and she decomposed it into three levels of decision.

Societal. What is society ready to allow a computer to help navigate or treat, which depends on public understanding of AI and agents, and on trust.

Health system. What is an organization ready to allow, weighed against risk to the patient, patient safety, risk to the organization and risk to individual clinicians making those decisions. This turns on accountability, understanding how the systems work and how they are monitored.

She made a pointed observation about current practice here. Healthcare already monitors care safety, but outside of inpatient vital signs it is largely not real-time. In ambulatory care, monitoring means the patient returning for a visit or calling about a side effect, which is less proactive than anyone would want. These technologies create the opportunity for proactive monitoring, and the field has to decide in a risk-based way what monitoring each AI-enabled workflow requires.

Clinicians. Also a trust question, plus education about what these tools can do, should do, cannot do and should not do.

Her analogy was pharmaceutical. People use AI tools on-label and off-label, and the implications of off-label use need to be understood. A model built to treat hypertension in pregnant women applied to a 50-year-old man is a very different population.

She closed with a reframing of a phrase used elsewhere at the Forum. AI moves at the speed of trust, and it also moves at the speed of what is available, given that anyone can sit on their couch on a Saturday morning and build their own agent. What the field needs is AI at the speed of responsibility and accountability.

Automation Complacency

Bostwick asked Allon Bloch where AI augments clinical decision-making versus where it risks the muscle memory of clicking yes repeatedly until vigilance decays.

Bloch set the scale first. K Health operates in 10 major hospital systems including Mass General, where AI is part of the care team and tens of thousands of patients have been seen, alongside Mayo Clinic, Penn Medicine, Northwell and Novant.

His argument for moving forward rather than waiting concerned what patients are already doing. They use ChatGPT and Claude extensively, considerably more than doctors would like, drawing their own conclusions without their medical history and without integration into the system. Using those tools, he said, you feel the intelligence but you also feel the bravado. Even when a model states it is not deciding for you, it often sways patients.

On complacency itself, he identified the temptation directly: having seen 999 patients and always agreed, concluding there is no zebra here and pressing the button. Much of the countermeasure is interface design, because the work involves engaging the physician as much as the patient.

Two further points. This is not one AI bot, a word he dislikes because it is like saying compute. There are roughly a dozen agents orchestrated together and monitored along the way. And their AI is trained to say it does not know, just as a good physician would, rather than guessing.

Auditing Machine Behavior

Haider Warraich MD introduced himself as a cardiologist practicing in Boston, formerly at the FDA, now leading the ADVOCATE program at ARPA-H, which he described as the largest US government investment in agentic AI.

Its goal is bringing patient-facing technologies to market through the regulated process as clinician extenders performing tasks clinicians do today, with a focus on heart failure.

He positioned the program as an argument against the premise that one must choose between being safe and being innovative. Well-designed programs focused on the real barriers to a technology can do both.

He agreed the technology is not ready for autonomous functionality today, and said that is exactly why the investment is being made: to bring frontier labs, startups, large AI companies, health systems, academic labs and professional societies into the room together to build the full pyramid beneath agentic clinical AI.

Supervision is a central focus. Monitoring after deployment matters, and while not everything requires real-time monitoring, a technology that can take actions needs a monitoring system that keeps pace. Supported teams include one led by Stanford University with OpenAI and the University of Washington, with another to be announced joining that supervisory layer.

His reasoning: health systems need assurance of transparency into what they are deploying and redundant layers of safety before these technologies reach the bedside.

Chaotic Patient Input and the Advantage of Time

Muthu Alagappan MD, an internist leading an AI-native primary care company delivering care by messaging, addressed how non-standard patient language gets interpreted.

He separated two questions: whether AI history taking produces better or worse outcomes than human history taking, and whether chat as a modality has advantages over video or in-person.

On the form factor, he reported that more patients are turning to messaging as their entry point, often starting on ChatGPT or Claude even if that is not where their journey ends.

He identified two genuine advantages messaging holds over the clinic.

Anonymity and reflection. A patient in clinic who feels rushed or judged may not report accurately how many drinks a week they have or whether they actually quit smoking. The relative invisibility of messaging produces more truth and honesty.

The dimension of time. Rather than taking a history only in the present moment, a patient presenting with shoulder pain can be assessed now, again in two hours, again in six, and again the next morning. That tracks a condition toward a better diagnosis or triage decision, much as an emergency department reevaluates after an initial protocol.

Key Takeaways

1. Nothing in clinical care is fully autonomous today, and a human sits at the end of every workflow discussed.

2. Autonomy is three separate decisions: what society permits, what a health system accepts, and what clinicians trust.

3. AI moves at the speed of availability as much as trust, which is why responsibility and accountability are the needed pace.

4. Off-label AI use is the underexamined risk. A model built for one population applied to another carries implications clinicians must understand.

5. Patients already use general models extensively, drawing conclusions without their medical history, which argues against waiting.

6. Monitoring must keep pace with action. ARPA-H is funding a supervisory layer alongside the technology itself.

7. Messaging has clinical advantages over the visit: more honest histories, and the ability to observe a condition across time.