Why clinical AI must be designed for humans, not just trained on data

As AI becomes part of everyday radiology, experts argue that successful implementation depends as much on human factors, workflow integration, and communication as on algorithm performance.

Photo

Artificial intelligence continues to transform radiology, but technical accuracy alone no longer determines whether an AI tool succeeds in clinical practice. As AI becomes increasingly integrated into routine workflows, attention is shifting beyond algorithm performance to the human factors that determine whether AI succeeds in practice: how clinicians interact with AI, how information is communicated, and whether technology genuinely supports clinical decision-making. Human-centered design is therefore becoming a critical component of safe and effective AI implementation.

The problem: AI can introduce new cognitive risks

AI is designed to support radiologists, yet poorly implemented systems can unintentionally create new sources of diagnostic error. Experts highlight cognitive biases such as automation bias and complacency bias, both of which can lead clinicians to rely too heavily on AI recommendations.

Although these biases may produce similar outcomes, they arise through different cognitive processes. As a result, they require different mitigation strategies. Experts argue that simulation-based learning, targeted training, debiasing techniques, and thoughtful interface design can help clinicians develop appropriate trust in AI rather than simply trusting, or distrusting, it by default.

Instead of asking whether clinicians should rely on AI, the more important question becomes when they should rely on it and when they should challenge its recommendations.

The design lens: technology must fit clinical workflows

Successful AI implementation requires far more than accurate algorithms. Systems must also reflect the realities of clinical practice, supporting the way radiologists think, work, and make decisions under everyday conditions.

Woman sitting at a desk and looking at medical scans disyplayed on a monitor in...

Experts recommend studying real clinical workflows before introducing AI tools, identifying where technology can reduce cognitive workload rather than create additional complexity. Testing systems in realistic simulation environments can uncover usability problems before implementation. Insights from aviation and other safety-critical industries demonstrate the value of designing technology around human behavior rather than expecting people to adapt to technology.

As Patricia Trbovich, Associate Professor at the University of Toronto and Badeau Family Research Chair in Patient Safety and Quality Improvement (Canada), explained during a session at RSNA 2025: “The success of an AI is going to be less dependent on how well you can model the accuracy of the AI and more so on how well you can get the design of this interaction downright.”

Experts also caution against overwhelming clinicians with more information. Instead, AI should transform complex data into meaningful clinical insights that help radiologists recognize patterns, prioritize findings, and make informed decisions more efficiently.

The communication layer: context makes AI actionable

Even highly accurate AI systems can lose clinical value if their outputs are poorly communicated. Experts argue that AI recommendations should be structured, framed, and delivered in ways that fit naturally into clinical workflows, allowing radiologists to interpret and act on them when they are most useful.

Providing confidence measures, communicating uncertainty, and presenting relevant clinical context can improve interpretability and help clinicians judge when AI recommendations should be trusted and when they should be reviewed more carefully. Rather than relying solely on performance metrics or heat maps, AI should explain its reasoning in ways that support clinical judgment and make recommendations genuinely actionable for both physicians and patients.

The payoff: better collaboration, less burden, safer deployment

Group of medical professionals sitting around a desk shaking hands...

Ultimately, the goal of clinical AI is not to replace radiologists but to strengthen human decision-making. When AI is designed around clinicians' needs, integrated into existing workflows, and communicated clearly, it can reduce cognitive burden, support shared decision-making, and improve patient care without compromising professional oversight.

As Elizabeth A. Krupinski, Professor and Vice Chair of Research at Emory University (US), concluded at a RSNA 2025 session: “Perceptually, cognitively, and ergonomically, AI tools really have to be human-centered, intuitive and easy to use.”

As AI adoption continues to accelerate, improving algorithms alone will not determine success. Long-term value will depend on building systems that clinicians can understand, question, and confidently integrate into everyday practice, allowing AI to become a trusted clinical partner rather than simply another source of information.


This article is based on presentations and statements from the RSNA 2025 session "Best Practices for Human-AI Collaboration”.

Channels

related articles
popular articles