SciGroveBeta
Neuroscience

Divisive Normalization Shapes Low-Rank Slow Manifolds for Continuous Working Memory

Zhaotian Gu, Jie Su, Weiwei Wang, Chang Liu, Tianyi Qian, Dahui Wang

Featured August 14, 2026

Original Paper

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

By using a brain-inspired 'division' rule, a new neural network learns to hold onto continuous memories smoothly, avoiding the 'shattering' problem of older models and naturally keeping its internal workings simple and efficient.

In depth
The paper introduces the Recurrent Divisive Normalization Network (RDNN), a biologically inspired model that uses dynamic division to robustly maintain and update continuous variables in working memory. This mechanism prevents the state space from shattering into discrete attractors, a common failure mode of standard RNNs, and intrinsically promotes low-rank dynamics by scaling gradient updates based on neural activity.

Key Takeaways

  • 1
    The RDNN architecture, inspired by divisive normalization, learns robust, high-fidelity continuous slow manifolds, overcoming the fragility of classical continuous attractor networks and the discretization issues of standard RNNs.
  • 2
    Divisive normalization induces activity-dependent gradient scaling during training, leading to an emergent self-compression of the network's effective rank and confining recurrent dynamics to a tight, low-dimensional subspace.
  • 3
    Unlike subtractive inhibition, multiplicative divisive normalization is mathematically essential to prevent continuous memory manifolds from shattering under time-varying external inputs, while also avoiding optimization pathologies of explicit low-rank factorization.

Conceptual Flow

HIGH LEVEL
1
Methodology: How the RDNN Works

The new network uses two groups of brain cells that 'divide' their signals to keep memory stable, unlike old methods that just add or subtract.

Input Signal
Memory Cells
Divide Signals
Stable Memory
2
Results: Stable Continuous Memory

This 'division' trick helps the network learn smooth memory paths, while older networks get stuck in choppy, broken memory states.

Old Method
New Method
Compare Memory
Choppy Memory
Smooth Memory