Katie covers the impact of health technology on patients, clinicians, and businesses. Her stories explore the price tag of clinical AI, digital health at the FDA, and the boom in direct-to-consumer telehealth. Confidential tips can be sent on Signal at palmer.01.

Across medicine, doctors are increasingly using artificial intelligence models to predict patient risk, from sepsis to falls to death. Often embedded directly into electronic health records, these predictions can be easy to incorporate into care — and their performance can be easily taken for granted. 

James Deardorff, a geriatrician and assistant professor in the division of geriatrics at the University of California San Francisco, has developed several models that aim to make predictions for older adults, from mortality to the need for nursing home care. In geriatrics, he says, it’s important for clinicians to be aware of both the performance of an algorithm — including in subgroups like older patients — and how to responsibly use its output.

This month, Deardorff penned a commentary on a large analysis of Epic’s proprietary end-of-life prediction model published in JAMA Network Open, highlighting how a model can contribute to a poor outcome even if it’s accurate. If a patient’s one-year mortality prediction is used to prompt an open-ended conversation about the goals of care, he and his co-author wrote, there are few downsides. But if it’s used to inform a higher-stakes decision like transplant priority, the impacts could be profound.