Educational content on VJHemOnc is intended for healthcare professionals only. By visiting this website and accessing this information you confirm that you are a healthcare professional.

Share this video  

IMS 2026 | Addressing the risk of over-reliance on AI in clinical decision-making

Adrián Mosquera Orgueira, MD, PhD, Santiago Clinic Hospital CHUS, Santiago, Spain, discusses the need for critical thinking skills to challenge artificial intelligence (AI) outputs. Dr Mosquera Orgueira notes that overreliance on AI technology can erode critical thinking and individuals’ ability to evaluate AI results, citing examples in which doctors failed to detect intentionally introduced mistakes in AI responses. This interview took place at the 23rd International Myeloma Society (IMS) Annual Meeting in Glasgow, UK.

These works are owned by Magdalen Medical Publishing (MMP) and are protected by copyright laws and treaties around the world. All rights are reserved.

Transcript

At the very beginning, I used to think that clinicians who wanted to use AI, they should not know a lot about AI. I still think they should know the basics of AI, not a lot. Obviously, there must be some regulations. There is a new ISO guideline for machine learning tools in clinics. And I think once we validate these tools with these standards, clinicians should be relatively comfortable using them if they know the basics of AI and statistics...

At the very beginning, I used to think that clinicians who wanted to use AI, they should not know a lot about AI. I still think they should know the basics of AI, not a lot. Obviously, there must be some regulations. There is a new ISO guideline for machine learning tools in clinics. And I think once we validate these tools with these standards, clinicians should be relatively comfortable using them if they know the basics of AI and statistics. But I have a higher concern about the nervous skilling. And it’s starting to be proved that fellows who are trained with AI tools, they are unable to adopt new skills as before. So there is a debate on whether you should previously know how to perform a task to be able to then evaluate what the AI tool is doing. Because if you are unable to do clinical tasks by yourself with a critical mind, then you will be unable to challenge what the AI is telling you. So this is a very hot topic. Nobody knows very well how to manage it. And I think it’s probably the most dangerous thing that we lose our critical mindset and we lose our individual capacity to challenge what AI says and just perform what it says. There are many, many, many examples of this. Many, many projects where AI was implemented and with a human in the loop who had to validate the outputs of the AI, and even when they purposefully introduced some mistakes into the responses, only a minor part of doctors detected it because they were over-reliant on the technology, so this is a very big problem.

This transcript is AI-generated. While we strive for accuracy, please verify this copy with the video.

Read more...