SciGroveBeta
Medicine

Towards Expert-level Medical AI for Real-time Video Consultations

Mahvish Nagda, Jihyeon Lee, Matthew Thompson, Chunjong Park, Tim Strother, Valentin Liévin

Featured August 14, 2026

Original Paper

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

A new AI system called AMIE (Video) can now talk to patients and understand their body language in real-time video calls, performing as well as human doctors for many tasks, thanks to its multi-agent design.

In depth
The paper introduces AMIE (Video), a Gemini-based multi-agent system designed for real-time clinical video consultations. This system integrates low-latency dialogue, clinical reasoning, and real-time audio-visual perception through specialized Talker, Planner, and Perception agents to achieve expert-level performance, overcoming the inherent limitations of text-only medical AI by capturing crucial non-verbal cues.

Key Takeaways

  • 1
    AMIE (Video) achieves expert-level performance in real-time video consultations, matching or exceeding primary care physicians (PCPs) in history-taking, diagnosis, and management.
  • 2
    The system utilizes a novel asynchronous multi-agent architecture (Talker, Planner, Perception agents) to balance low-latency dialogue with thorough clinical reasoning and real-time audio-visual perception.
  • 3
    Patient actors significantly preferred AMIE (Video)'s interface over text chat for communicative effectiveness and feeling understood, highlighting the importance of audio-visual cues in telehealth.

Conceptual Flow

HIGH LEVEL
1
AMIE's Multi-Agent Design for Real-time Video Consultations

The AI uses three smart helpers working together to talk, think, and see during a video call, just like a team of doctors.

Patient Video Call
Split & Process
Talker Agent
Planner Agent
Perception Agent
2
Expert-Level Performance in Clinical Video Consultations

The AI performed as well as or better than human doctors in video calls, and patients liked talking to it more than text-only AI.

AI Video Consult
Human Doctor Consult
AI Text Consult
Compare Performance
AI Video: High Scores
Patients Prefer Video AI

This breakdown was generated by SciGrove. Get the same analysis — intuition, storyboard, peer review, a runnable prototype and a glossary — on any paper you upload or paste a DOI for.