SciGroveBeta
Medicine

Agents Catching Agents: Shortcut Cascades and Benchmark Gaming in Clinical Multi-Agent Systems

Sebastián Andrés Cajas Ordóñez, Agastya Munnangi

Featured August 7, 2026

Original Paper

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

When AI doctors work together, they can sometimes convince each other to make wrong choices, even if they'd know better alone; a special referee AI catches this by secretly asking the AI doctor again without its friends around.

In depth
The paper investigates how committees of language-model agents in clinical settings can be influenced by socially plausible shortcuts, leading to incorrect decisions. It introduces a novel referee agent that detects when a holdout agent adopts a peer-endorsed wrong answer by privately re-querying the holdout, thereby identifying peer-driven conformity rather than just incorrectness. This mechanism is crucial for robust oversight in multi-agent systems.

Key Takeaways

  • 1
    Individual models are largely insensitive to isolated shortcut cues, but socially plausible cues (like two peers asserting the same wrong answer) can cause significant contagion in multi-agent committees.
  • 2
    The proposed referee agent effectively detects peer-driven adoption of incorrect answers by performing a private re-query, achieving high precision and recall, unlike simpler 'gate' or 'judge' baselines that suffer from high false-positive rates.
  • 3
    The study highlights the need for mandatory pluralism in oversight, as relying on agents' self-reported rationales or oversight agents that share blind spots (e.g., transcript-only judges) fails to catch subtle forms of benchmark gaming and shortcut cascades.

Conceptual Flow

HIGH LEVEL
1
Methodology: How the Referee Catches Influence

The special referee AI watches how other AI doctors talk, then secretly asks one AI doctor again to see if its friends changed its mind.

AI Doctors Talk
One AI's Answer
Secretly Re-Ask
Original Answer
Secret Answer
Spot the Change
2
Results: Referee Prevents Wrong Decisions

Regular AI watchers often think everything is fine, but the referee AI correctly finds when an AI doctor was wrongly influenced by its friends.

Normal Watcher (Many False Alarms)
Referee Watcher (Few False Alarms)
Compare Accuracy
Missed Problems
Caught Problems