ASH 2025 | AI‑driven virtual tumor board enhances precision care in AML: refining AI use in hematology

Yeah, yeah. So, you know, obviously hematology is very complicated. You know, it’s ever-changing. You know, we have new diagnostic classifications, especially in myeloid malignancies. There are multiple different classifications that sometimes can compete with each other. You know, prognostication is ever-changing. It’s more molecular-based. It’s more sophisticated, but that leads to, you know, prognostication is ever-changing, it’s more molecular-based, it’s more sophisticated, but that leads to you know, that point of care ability to use it really quickly and get the information that you need fast in a busy practice, and then treatment decisions, you know, are, you know, we are doing well to get a lot of new therapies, but that makes the subtleties of treatment decisions that much more challenging. And so, ultimately, how do we bridge the gap of the changing landscape in myeloid diseases? And the suggestion is, can we use AI, specifically large language models, to help do that? We know physicians are using LLMs in their everyday practice, primarily though right now for administrative tasks. Things like writing patient letters, you know, generating, you know, administrative scheduling, you know, maybe making PowerPoints. Things that I would consider are relatively simple tasks that you could do by yourself, but ultimately maybe saving time, making things a little bit more efficient. But I think that’s just scratching the surface. How do we move into more complex decision-making, bringing in genomics, clinical data, and making it easier for patients? So, can we use general LLMs to ultimately do that? And so, that’s what we initially asked. In our first presentation, we looked at general LLMs, large language models like ChatGPT, Claude, DeepSeq, some of the common ones that we use, looking at complex medical decision-making. And what we found, and what others have found before us, is they really right now don’t perform that well. They can hallucinate, which means generate false information. They often are delayed and provide guidelines or even diagnostic options that are a little bit behind the curve of the rapidly changing landscape. And so, that sort of is a challenge. It might turn off people from using AI in general for clinical care. And I think right now that’s pretty appropriate. I think if you’re using AI for clinical care right now in the current state, you’re going to get a lot of bad information. But how do we move to the next state of this? How do we move beyond just a single general large language model to help with these tasks? And that’s really where we’re going with our next, you know, set of models and our next set of validations. And so, you know, we previously published this on MDS and now we’re publishing it on AML at ASH, but we are starting to use AI agents. And so, you know, I’ve talked about this a little bit before, but what is an AI agent? So, it is a task-oriented piece of software. You can think about it as a more complex chatbot that is really powered by a large language model. You can pick whichever one you want to for a specific individualized task. And we’ve found that when you train an agent on a task, it has memory, it just, it’s a little bit more task-oriented and a little bit more focused on what you’re trying to achieve, and ultimately using an AI agent, you know, outperforms a general large language model in a lot of the complex tasks that we do day to day as a physician and physician. And so, how do we take this one step further? So, we know that one AI agent is good, but if you string together multiple AI agents, each with an even more specific task that they’re trying to accomplish, you have a much better output, a much better product at the end. And so, that’s what we did with the AML Virtual Tumor Board and what we’ve done with our MDS Virtual Tumor Board in the past is combine AI agents. And so, what happens with this particular system is that a physician would be seeing, let’s say, seeing a patient in clinic. And ultimately, they have a lot of information and they want a decision support tool like UpToDate or NCCN that they can reference and ultimately get information that they can use for evidence-based, guideline-based recommendations. But in this case, you feed a patient-specific case to this framework, and it works like a real-life tumor board, except it’s all AI and automated. And so, you give an unstructured patient case, you have a moderator agent that structures the information, you go to a diagnostic agent in an AML case that references the ELN and WHO guidelines, you have a prognostic agent that references the ELN 2022 and 2024 prognostic classification systems. And then you have a treatment agent that references NCCN and other treatment guidelines. Ultimately, they work autonomously, similar to what you would see in a real-life tumor board. And it’s really interesting. They are communicating back and forth and then ultimately providing back to the moderator the information to generate a nice summary, and that’s what we got. And so, what we did is actually take those summaries for 20 complex AML cases and then blindly provided them to experts to review against general large language models like ChatGPT, and what we found is using this multi-agent framework, we were able to get, based on a scoring above four to five out of five on a Likert scoring system, about 93 percent concordance with an expert recommendation, so the experts felt like this was 93 percent concordance with the guidelines in what they do, and the general large language models across the board were about 40 to 60% at the highest, concordant with an expert opinion. And so, you know, the other big thing is I talked about hallucinations. So, hallucinations are a big problem with general large language models. With our agentic framework, we actually saw zero, you know, fully, you know, factually incorrect outputs in this particular version of the framework, which, you know, is, you know, builds on the trust of clinicians that they’re trying to use this. And so, basically what we showed and what we built is an autonomous multi-agent framework that works like a virtual tumor board to take clinical information and provide expert-level recommendations. Ultimately, the physician at the end of the day is the one that’s gonna, you know, pick the treatment, but it really is a decision support tool similar to UpToDate or referencing NCCN that can make your life a lot more efficient and getting what you need to take care of an individual patient. And so, you know, I think, you know, right now we are, you know, sort of in the early days of it. But, you know, we can see expanding on our work, integrating EMR data, and ultimately, you know, building this out to be able to be a tool that others can use that might not see as much AML or, you know, MDS ultimately in their clinic to get expert recommendations in real-time and hopefully that will ultimately improve patient care and patient outcomes.

This transcript is AI-generated. While we strive for accuracy, please verify this copy with the video.