Neuroscience

BrainJanus: A Unified Model for Understanding and Generation across Brain, Vision, and Language

Haitao Wu, Qirui Zhang, Zhouheng Yao, Shangquan Sun, Qihao Zheng, Mianxin Liu, Chi Zhang, Wanli Ouyang, Chunfeng Song, Changqing Zhang, Jiamin Wu

Featured July 7, 2026

Original Paper

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

A new computer brain model called BrainJanus learns to turn brain signals, pictures, and words into a shared secret code, letting it understand and create any of them from each other, like translating thoughts into images or text.

In depth

The paper introduces BrainJanus, a pioneering unified brain model that integrates brain signals, vision, and language within a single framework. It achieves this by first quantizing continuous neural dynamics into discrete tokens using a Unified Brain Tokenizer, aligning them with visual and linguistic representations in a shared 'Omni space'. Building on this, an All-in-One autoregressive architecture leverages next-token prediction to enable seamless any-to-any generation, encompassing both encoding (image/text to brain) and decoding (brain to image/text) tasks.

Key Takeaways

1
The authors developed the first unified autoregressive framework, BrainJanus, which bridges brain, vision, and language modalities through a shared discrete token space, enabling flexible any-to-any generation.
2
A novel Unified Brain Tokenizer is introduced to convert continuous neural signals into discrete tokens, which are then aligned with existing vision and language tokens in a common 'Omni space'.
3
The model demonstrates superior performance across diverse encoding and decoding benchmarks, exhibiting zero-shot generalization and preserving interpretable biological topography, suggesting its potential as a general-purpose brain modeling paradigm.

Conceptual Flow

HIGH LEVEL

Methodology: Unifying Brain, Vision, and Language

The model turns brain signals, pictures, and words into a shared secret code, then uses a smart predictor to guess the next part of the code, allowing it to switch between them easily.

Brain Signals

Pictures

Words

Turn into Shared Code

Unified Code Stream

Results: Any-to-Any Generation

By using its shared code, the model can accurately turn brain signals into detailed pictures or words, and also turn pictures or words into brain signals, showing it understands how they connect.

Brain Code

Picture Code

Word Code

Predict Next Code

Generated Brain

Generated Picture

Generated Text