SciGroveBeta
Machine Learning

Do Models Read What They Write? Causal Registers in Scratchpad Reasoning

Benjamin Shih, John Winnicki, Eric Darve

Featured July 8, 2026

Original Paper

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

Training models to write down their thinking step-by-step can make them actually *use* those written thoughts to figure out the next step, like a calculator using its display to compute.

In depth
The paper demonstrates that fine-tuning language models with running-state supervision can make their internal scratchpad representations causally effective. By performing counterfactual state editing on a controlled task, the authors show that changing a model's internal representation of an intermediate state, while keeping the visible text fixed, redirects its subsequent computation according to a known transition rule. This indicates that the model is not just generating plausible text but actively computing from these internal 'causal registers'.

Key Takeaways

  • 1
    Running-state supervision enables language models to use written scratchpad states as causal inputs for subsequent computation, rather than merely generating correlated text.
  • 2
    A counterfactual state-editing test, which modifies internal state representations while keeping visible text fixed, can rigorously verify if a model computes from its intermediate variables.
  • 3
    The causal pathways for state updates can be compact (e.g., a single attention edge) or distributed across multiple components, even when the state representation itself is low-rank.

Conceptual Flow

HIGH LEVEL
1
Methodology: Testing Causal Use

The researchers gave the model a puzzle, then secretly changed one of its internal 'thoughts' to see if its next answer changed in the way the puzzle rules said it should.

Model's Input
Model's Internal Thought
Secretly Change Thought
Model's Next Answer
2
Results: Models Can Use Thoughts

They found that models trained to write down their steps actually used their changed internal thoughts to get the correct new answer, unlike models that just gave a final answer.

Model Trained to Think Aloud
Model Only Giving Final Answer
Internal Thought Change
Next Step Changes Correctly
Next Step Stays Same