SciGroveBeta
Neuroscience

EmoMind: Decoding Affective Captions from Human Brain fMRI

Bilal A. Mohammed, Lin Gu, Ruogo Fang

Featured May 26, 2026

Original Paper

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

A new brain-to-text system translates fMRI signals into emotional captions by first describing what's seen, then adding feelings using a detailed brain-decoded emotion "recipe," outperforming simpler label-based methods.

In depth
The paper introduces EmoMind, a novel pipeline that decodes affective captions directly from fMRI signals. It achieves this by first retrieving a neutral scene description and then rewriting it using a continuous 34-dimensional emotion vector also decoded from the fMRI. A key innovation is the use of classifier-free guidance with an identity-preserving null branch, enabling fine-grained control over the balance between semantic content and affective expression in the generated captions.

Key Takeaways

  • 1
    EmoMind is the first end-to-end pipeline to generate affective captions directly from fMRI, using a continuous, high-dimensional emotion vector.
  • 2
    The method employs a novel Bernoulli-switched reconstruction-supervision training objective and classifier-free guidance to balance semantic fidelity and affective expressivity.
  • 3
    The approach significantly outperforms label-prompted GPT-4, especially on metrics requiring person-specific affective structure, demonstrating the value of continuous brain-decoded affect.

Conceptual Flow

HIGH LEVEL
1
Methodology (The 'Logic')

The system takes brain scans, figures out what someone is seeing and how they feel, then uses those feelings to rewrite a simple description into an emotional one.

Brain Scan
Split Information
What is Seen
How it Feels
2
Results (The 'Impact')

This new method creates much more personal and varied emotional descriptions from brain activity compared to just using simple emotion labels, showing how unique each person's feelings are.

Brain Scan
Old Way: Simple Labels
Generic Emotional Text

This breakdown was generated by SciGrove. Get the same analysis — intuition, storyboard, peer review, a runnable prototype and a glossary — on any paper you upload or paste a DOI for.