Neuroscience

STSBench: A Large-Scale Dataset for Modeling Neuronal Activity in the Dorsal Stream of Primate Visual Cortex

Ethan B. Trepka, Ruobing Xia, Shude Zhu, Sharif Saleki, Danielle Abreu Lopes, Stephen J. Niño Cital, Konstantin F. Willeke, Mindy Kim, Tirin Moore

Featured July 25, 2026

Original Paper

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

A new, massive dataset of monkey brain activity while watching videos helps train AI models to predict what neurons see and even recreate simple images from brain signals, especially focusing on movement and basic shapes.

In depth

The paper introduces STSBENCH, a large-scale dataset of single-neuron recordings from the primate dorsal visual stream (STS/MT/MST) while macaques viewed natural videos. This dataset enables the development and benchmarking of deep learning models for predicting neuronal responses (encoding) and reconstructing visual stimuli from neural activity. A key innovation is the adaptation of conditional latent diffusion models to generate images based on neural firing rates, demonstrating that dorsal stream activity primarily reconstructs motion and low-spatial frequency contours.

Key Takeaways

1
The paper releases STSBENCH, a novel dataset of 2,244 single-neuron recordings from the primate dorsal visual stream (STS/MT/MST), representing a ~50-fold increase over previous datasets.
2
They demonstrate that end-to-end trained 3D CNNs outperform hand-tuned and pretrained models for encoding neuronal responses in the dorsal stream, highlighting the importance of deep nonlinear computations.
3
The study successfully applies a neural-conditional latent diffusion model to reconstruct visual stimuli from dorsal stream neural activity, showing that these reconstructions primarily capture motion and low-spatial frequency information.

Conceptual Flow

HIGH LEVEL

Methodology: Building and Using the Dataset

Scientists collected lots of brain signals from monkeys watching videos, then used these signals to teach computers to understand what the brain sees and even draw pictures from brain thoughts.

Monkey Watches Videos

Brain Activity Recorded

Create Big Dataset

Train AI to Predict Brain

Train AI to Draw from Brain

Results: What the Brain Signals Show

The computer models learned that brain signals from the 'where' part of the brain are good at showing movement and big shapes, while signals from the 'what' part show details like colors and objects.

Brain Signals (Motion Area)

Brain Signals (Object Area)

AI Reconstructs Images

Reconstructed Motion/Shapes

Reconstructed Objects/Colors

This breakdown was generated by SciGrove. Get the same analysis — intuition, storyboard, peer review, a runnable prototype and a glossary — on any paper you upload or paste a DOI for.