Machine Learning

Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs. SFT Fine-Tuned Models

Antyabha Rahman, Akshaj Gurugubelli, Omar Ankit, Kevin Zhu, Aishwarya Balwani

Featured August 7, 2026

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

Smart math models trained with reinforcement learning learn to "see" correct answers much more clearly inside their brains, making deeper layers super important for solving problems.

In depth

The study reveals that Reinforcement Learning (RL) models for mathematical reasoning develop clearer, more linearly separable internal representations of correctness, which emerge earlier in the network. This is coupled with a hierarchical computational architecture where deeper layers are progressively more critical, unlike Supervised Fine-Tuned (SFT) models that distribute importance uniformly.

Key Takeaways

1
RL models encode correctness more clearly: They develop linearly separable representations of correct answers earlier in the network, detectable by linear probes.
2
RL training creates hierarchical architectures: Deeper layers in RL models become progressively more critical for reasoning, contrasting with the uniform importance distribution in SFT models.
3
Token allocation varies by model: While some RL models show higher token variability, others maintain consistent low variability, suggesting pipeline-specific influences beyond just RL vs. SFT.

Conceptual Flow

HIGH LEVEL

Investigating Model 'Brain' Differences

The researchers looked inside the models using three main tools to see how they think differently.

Model's Internal Thoughts

Analyze with 3 Tools

Clarity of Ideas

Important Thinking Steps

Consistency of Output

Key Discoveries About RL Models

They found that models trained with rewards have clearer internal ideas and use their deeper thinking parts more.

RL Model Training

SFT Model Training

Leads To

Clearer Internal Ideas (RL)

Deeper Layers More Critical (RL)

Output Consistency Varies

This breakdown was generated by SciGrove. Get the same analysis — intuition, storyboard, peer review, a runnable prototype and a glossary — on any paper you upload or paste a DOI for.