Machine Learning

Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training

Zijian Zhang, Rizhen Hu, Athanasios Glentis, Dawei Li, Chung-Yiu Yau, Hongzhou Lin, Mingyi Hong

Featured July 5, 2026

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

Surprisingly, adapting big language models with reinforcement learning mostly improves just a few middle layers, not the whole network, allowing smarter training that works even better.

In depth

The paper demonstrates that reinforcement learning (RL) post-training gains in large language models (LLMs) are not uniformly distributed but are highly concentrated in a small subset of transformer layers, typically located in the middle of the network. By introducing layer contribution as a metric, the authors show that training even a single high-contribution layer can match or surpass full-parameter RL, leading to more efficient and effective layer-aware training strategies.

Key Takeaways

1
RL post-training gains in LLMs are highly concentrated in a small subset of transformer layers, often a single layer, rather than being uniformly distributed.
2
High-contribution layers consistently reside in the middle of the transformer stack, a stable structural property observed across various models, tasks, and RL algorithms.
3
This discovered layer structure can be exploited through layer-aware training strategies (e.g., adaptive learning rates, selective training) to consistently outperform standard full-parameter RL.

Conceptual Flow

HIGH LEVEL

Methodology: How They Found It

To see which parts of a big language model learn best from feedback, they tried teaching only one part at a time and measured how much it improved.

Big Language Model

Train One Part

Measure Improvement

Results: What They Found

They found that only the middle parts of the model learned a lot, often doing as well as or better than training the whole model.

Train All Parts

Versus

Train Middle Parts Only

Middle Parts Win

This breakdown was generated by SciGrove. Get the same analysis — intuition, storyboard, peer review, a runnable prototype and a glossary — on any paper you upload or paste a DOI for.