Machine Learning

Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training

Zijian Zhang, Rizhen Hu, Athanasios Glentis, Dawei Li, Chung-Yiu Yau, Hongzhou Lin, Mingyi Hong

Featured July 5, 2026

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

Surprisingly, adapting big language models with reinforcement learning mostly improves just a few middle layers, not the whole network, allowing smarter training that works even better.

In depth

The paper demonstrates that reinforcement learning (RL) post-training gains in large language models (LLMs) are not uniformly distributed but are highly concentrated in a small subset of transformer layers, typically located in the middle of the network. By introducing layer contribution as a metric, the authors show that training even a single high-contribution layer can match or surpass full-parameter RL, leading to more efficient and effective layer-aware training strategies.

Key Takeaways

1
RL post-training gains in LLMs are highly concentrated in a small subset of transformer layers, often a single layer, rather than being uniformly distributed.
2
High-contribution layers consistently reside in the middle of the transformer stack, a stable structural property observed across various models, tasks, and RL algorithms.
3
This discovered layer structure can be exploited through layer-aware training strategies (e.g., adaptive learning rates, selective training) to consistently outperform standard full-parameter RL.

Conceptual Flow

HIGH LEVEL

Methodology: How They Found It

To see which parts of a big language model learn best from feedback, they tried teaching only one part at a time and measured how much it improved.

Big Language Model

Train One Part

Measure Improvement

Results: What They Found

They found that only the middle parts of the model learned a lot, often doing as well as or better than training the whole model.

Train All Parts

Versus

Train Middle Parts Only

Middle Parts Win