SciGroveBeta
Medicine

Towards Expert-level Medical AI for Real-time Video Consultations

Mahvish Nagda, Jihyeon Lee

Featured August 24, 2026

Original Paper

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

A new AI system, AMIE (Video), acts like a super-smart doctor on video calls, using different AI brains to see, hear, and think, performing as well as or better than human doctors in simulated patient visits.

In depth
The paper introduces AMIE (Video), a Gemini-based multi-agent system that achieves expert-level performance in real-time clinical video consultations. This system integrates low-latency dialogue, clinical reasoning, and real-time audio-visual perception through a novel asynchronous three-agent architecture. It was evaluated against primary care physicians (PCPs) in a randomized Objective Structured Clinical Examination (OSCE) study, demonstrating comparable or superior performance in key clinical competencies.

Key Takeaways

  • 1
    AMIE (Video) is the first AI system to demonstrate expert-level performance in real-time clinical video consultations, matching or exceeding PCPs in diagnostic accuracy, history-taking, and physical observation.
  • 2
    The system employs a novel asynchronous multi-agent architecture (Talker, Planner, Perception agents) to balance low-latency dialogue with thorough clinical reasoning and real-time audio-visual perception.
  • 3
    Patient actors preferred AMIE (Video)'s interface over text chat for communicative effectiveness and feeling understood, highlighting the critical role of audio-visual cues in virtual care.

Conceptual Flow

HIGH LEVEL
1
Methodology: How AMIE (Video) Works

The AI uses three specialized brains working together to talk, think, and see during a video call, just like a human doctor.

Patient Speaks
Patient Moves
AI Processes
Speaking AI
Thinking AI
Seeing AI
2
Results: AI Matches or Exceeds Doctors

In tests, the AI performed as well as or better than human doctors in understanding and treating patients, especially when using video.

AI Doctor
Human Doctor
Compared Performance
AI: Better Diagnosis
AI: Better Exams
AI: Equal Communication

This breakdown was generated by SciGrove. Get the same analysis — intuition, storyboard, peer review, a runnable prototype and a glossary — on any paper you upload or paste a DOI for.