Decoded #5: Guidelines Change: How Does Your AI Keep Up?

Good news: your tool is completely built, firing on all cylinders, and spitting out excellent outputs. Bad news: the ADA just released an entirely new set of guidelines that changed a key diabetes treatment algorithm that must be accounted for. This is a question to ask software developers, because medicine will inevitably evolve.

Clinical guidelines often update on regular patterns, but tools built without this foresight will struggle to adapt. Non-clinician-driven LLM tools will obtain the guideline (with proper permissions), plug it into their AI model's knowledge base, maybe remove the old guideline, and call it a day. This will leave them susceptible to hallucination and drift as they may have done their best to refine the software with past literature but lost the refinement as the old guideline was removed. Or they leave the old guideline and now have contradictory guidance in place that confuses AI reasoning and runs the risk of a major hallucination.

Clinical-grade AI tools will scour the guideline like a grade school teacher holding up the answer key in marker to spot differences where the bubbles don't match. Once the mismatch is identified, the clinical leads will weigh the ramifications of the change and impact on day-to-day practice and adjust or advise accordingly.

Now, take this one step further and think back to last week's Decoded post, "Pharmacist in the Loop". Understanding where that clinician is involved is key in times of change. The team that has the clinician as "The Assembler" will understand the interconnection of the guideline changes and know directly what to alter to synchronize the machine. They'll know every bolt to replace or tighten to ensure the tool is always in perfect harmony with guidelines and clinical practice.