Medicine

Evaluating medical AI under missing information: same-provider judges and human raters change apparent safety

Koyar Afrasyab

Featured July 28, 2026

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

When medical AI models are asked questions with missing details, how they are judged matters a lot: the paper shows that AI judges are often too easy on other AIs, especially their own kind, compared to real doctors.

In depth

The paper extends medical AI stress-testing to open-ended clinical conversations under missing information, where safe behavior means recognizing absent data and declining to over-commit. It rigorously demonstrates that the choice of LLM judge significantly alters apparent safety scores due to moderate inter-judge agreement and a quantifiable same-provider bias. Crucially, the study reveals that LLM judges are systematically more permissive than human clinicians, leading to optimistic safety estimates.

Key Takeaways

1
The paper introduces a novel missing-information robustness probe for open-ended medical conversations, where models are tested on their ability to recognize absent clinical details and avoid unsafe over-commitment.
2
It rigorously quantifies how LLM judge choice and a 'same-provider preference' bias materially alter the apparent safety ranking of frontier models, highlighting the need for evaluator-reliability analysis.
3
The study establishes that LLM judges are more permissive than human clinicians in assessing appropriate uncertainty, suggesting that LLM-judged safety rates are often optimistic relative to expert human standards.

Conceptual Flow

HIGH LEVEL

Methodology: How LLMs are Stress-Tested for Safety

The study tests AI doctors by giving them incomplete patient stories and seeing if they admit they don't know enough, instead of guessing.

Full Patient Story

Medical Question

Remove Key Details

Incomplete Patient Story

AI Doctor's Answer

Results: Why AI Judges See Safety Differently

The study found that AI judges are much nicer than human doctors when scoring AI answers, and they even favor answers from their own company's AI.

AI Doctor's Answer

AI Judge's Score

Human Doctor's Score

Compare Scores

AI Judges More Lenient

AI Judges Favor Own Brand