UCSF and UCSF Health Receive Pivotal Donation to Support the First Continuous AI-monitoring Platform in Clinical Care

The UCSF Division of Clinical Informatics and Digital Transformation (DoC-IT) and UCSF Health have received a $5 million gift from Ken and Kathy Hao to develop a cutting-edge, real-time, continuous, and automated artificial intelligence (AI) monitoring platform for clinical care.

The Impact Monitoring Platform for AI in Clinical Care (IMPACC) aims to bridge the gap between the rapid evolution of AI technologies used by clinicians and the essential need for robust, ongoing assessment of their efficacy, safety, and equity. 

Julia Adler-Milstein, PhD, chief of the UCSF Division of Clinical Informatics and Digital Transformation (DoC-IT), and Sara Murray, MD, MAS, chief health AI officer at UCSF Health, will lead the pioneering collaboration.

“This philanthropic gift is transformative in many ways,” said Adler-Milstein. “It comes at a critical juncture as the healthcare industry more broadly integrates AI into clinical practice. Through IMPACC and this collaborative effort, we are poised to improve patient care at UCSF while advancing the science of how to assess AI tools in real-world use.”

Currently, the healthcare field lacks established protocols for ongoing AI monitoring, leading to risks of adverse outcomes for patients and healthcare providers that go undetected. While assessments are conducted to determine the suitability of new AI technologies for safe integration into clinical environments before deployment, once they are deployed, health systems need a way to promptly identify any issues in their real-world performance.

IMPACC will fill this urgent need by shifting from planned, periodic, manual monitoring of a focused set of measures to real-time, continuous, automated, and longitudinal monitoring across a broad measure set with specified criteria for escalation to human review and intervention.

“By building IMPACC, we will take a major leap forward in how we analyze AI’s performance in healthcare,” said Murray. “As we deploy new AI technologies, this novel, scalable platform will provide our health system with direct and actionable insights into ongoing performance, ensuring not only the effectiveness of these new tools but also safety across the system and benefit for patients.”

In addition to monitoring both performance and impact on targeted outcomes over time – such as whether an AI tool improves clinical outcomes for patients – IMPACC will be used to inform healthcare leaders on decisions about scaling, refining, or turning off a tool for their systems. Specifically, it will report on whether a tool is achieving its intended results or if it requires improvement, as well as flag if a tool is potentially dangerous or risks worsening health disparities, prompting immediate action when necessary.

After development and testing, IMPACC will be piloted at UCSF Health on an initial set of current AI tools. This effort will be in collaboration with the UCSF Health AI Oversight Committee, which makes recommendations about the safety and efficacy tools and whether they should be deployed more broadly across the health system. The team also will explore building a dashboard to allow patients to track when AI has been used in their care.

While simple in concept, in practice, IMPACC requires deep and diverse expertise, including highly technical development and social and behavioral sciences, making DoC-IT and UCSF Health together optimally equipped to undertake this innovative work. It is only because of the Haos’ visionary philanthropic support that UCSF and UCSF Health are able to pursue vital research and its translational applications.

___

About the UCSF Division of Clinical Informatics and Digital Transformation (DoC-IT)
DoC-IT, a newly formed division at UCSF, serves as the academic home for applied clinical informatics researchers in the UCSF Department of Medicine. Included within the division is the country’s only research center – the Center for Clinical Informatics and Improvement Research (CLIIR) – that is dedicated to using metadata to measure real-world use and impact of digital tools. The nationally recognized center specializes in how to measure user interactions with the electronic health record (EHR), including what data is viewed, what actions are taken, and how the introduction of AI tools alters these behaviors.

About UCSF Health
UCSF Health is internationally renowned for providing highly specialized and innovative care and a part of UCSF, one of the top universities in the nation for health sciences research and higher education. By bringing together the world's leading experts in nearly every area of health, UCSF Health drives advancements in treatment and technology that benefit patients everywhere. The UCSF Health AI team has created the AI governance process and built the HIPAC platform, which is the infrastructure on which AI models run and integrate with the UCSF Health EHR.

Contact
Mika Rivera, Director of Communications