UCSF Study Identifies Varied Benefits from AI Scribes with Implications for Return on Investment

Findings show AI scribes reduce documentation burden, but benefits vary by physician baseline efficiency

A new study led by UCSF researchers and published in the American Journal of Managed Care (health IT special issue) offers new insights into how AI scribes impact physician work experience and documentation time. The research, which examined over 300 ambulatory physicians across multiple specialties at UCSF, reveals a more complex picture than previously understood about who benefits most from this emerging technology.

The study, titled "Subjective and Objective Impacts of Ambulatory AI Scribes," found that while 86.5% of physicians perceived reductions in documentation time after adopting AI scribe technology, the actual time savings were modest and not strongly correlated with perceived benefits. This suggests that different groups of physicians may be deriving different types of benefits from the technology.

Most significantly, the research identified that physicians with higher baseline documentation time—those who spent more time on clinical notes before AI scribe adoption—realized the majority of actual time savings. 

"Our findings suggest that AI scribes are widely perceived by physicians as a valuable tool and therefore help promote physician wellbeing. A different type of benefit—objective time savings—are concentrated among physicians with higher baseline documentation time," said lead author Julia Adler-Milstein, PhD, chief of the UCSF Division of Clinical Informatics and Digital Transformation (DoC-IT). 

The research team also raises important questions about how to maximize the value of AI scribes across all physicians, including understanding which encounter types benefit most from the technology and how physician trust in the tool evolves over time.

The full study is available in The American Journal of Managed Care. The work was supported in part by a gift from Ken and Kathy Hao to establish the Impact Monitoring Platform for AI in Clinical Care (IMPACC) at UCSF. 

Read the study

Authors: Julia Adler-Milstein, Orianna DeMasi, Hossein Soleimani, Sarah Beck, Maria Byron, Aris Oates, Robert Thombley, Jinoos Yazdany, Sara Murray.

Another study supported in part by IMPACC, "Ambient Artificial Intelligence Scribes and Physician Financial Productivity," was recently published in JAMA Network Open. Read more here. 

To learn more about AI scribes and their use at UCSF, check out this Q&A with Sara Murray, MD, MAS, UCSF Health’s chief AI officer.