SciGroveBeta
Medicine

Towards Expert-level Medical AI for Real-time Video Consultations

Mahvish Nagda, Jihyeon Lee, Matthew Thompson, Chunjong Park, Tim Strother, Valentin Liévin

Featured August 14, 2026

Original Paper

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

A new AI system called AMIE (Video) can now talk to patients and understand their body language in real-time video calls, performing as well as human doctors for many tasks, thanks to its multi-agent design.

In depth
The paper introduces AMIE (Video), a Gemini-based multi-agent system designed for real-time clinical video consultations. This system integrates low-latency dialogue, clinical reasoning, and real-time audio-visual perception through specialized Talker, Planner, and Perception agents to achieve expert-level performance, overcoming the inherent limitations of text-only medical AI by capturing crucial non-verbal cues.

Key Takeaways

  • 1
    AMIE (Video) achieves expert-level performance in real-time video consultations, matching or exceeding primary care physicians (PCPs) in history-taking, diagnosis, and management.
  • 2
    The system utilizes a novel asynchronous multi-agent architecture (Talker, Planner, Perception agents) to balance low-latency dialogue with thorough clinical reasoning and real-time audio-visual perception.
  • 3
    Patient actors significantly preferred AMIE (Video)'s interface over text chat for communicative effectiveness and feeling understood, highlighting the importance of audio-visual cues in telehealth.

Conceptual Flow

HIGH LEVEL
1
AMIE's Multi-Agent Design for Real-time Video Consultations

The AI uses three smart helpers working together to talk, think, and see during a video call, just like a team of doctors.

Patient Video Call
Split & Process
Talker Agent
Planner Agent
Perception Agent
2
Expert-Level Performance in Clinical Video Consultations

The AI performed as well as or better than human doctors in video calls, and patients liked talking to it more than text-only AI.

AI Video Consult
Human Doctor Consult
AI Text Consult
Compare Performance
AI Video: High Scores
Patients Prefer Video AI