SciGroveBeta
Machine Learning

Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking

Devin Pereira, Willem Zuidema

Featured August 13, 2026

Original Paper

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

Big AI models can 'see' the whole puzzle in their mind, but then they forget parts of it while trying to solve it, making them fail; fixing their memory helps them succeed.

In depth
The paper reveals that large language models (LLMs) develop an emergent world model of the Tower of Hanoi puzzle, representing its state space as a Sierpiński triangle. However, this internal representation degrades during the model's reasoning process, leading to planning failures. The authors demonstrate that restoring this representation via activation steering can significantly recover performance, indicating that the models 'forget' what they knew rather than never knowing it.

Key Takeaways

  • 1
    Large Reasoning Models (LRMs) develop an emergent world model of the Tower of Hanoi puzzle, encoding its state space as a Sierpiński triangle with high fidelity at the prompt.
  • 2
    The internal world model degrades significantly during the LRM's chain-of-thought generation, causing a collapse in planning performance for complex instances.
  • 3
    Performance can be substantially recovered by continuously injecting the clean, prompt-time world model representation during inference, causally linking degradation to failure.

Conceptual Flow

HIGH LEVEL
1
Methodology: How AI's Internal 'Mind' Was Explored

The researchers looked inside AI models to see how they 'think' about a puzzle, checking if they build a mental map and if that map helps them solve it.

Small AI Models
Big AI Models
Analyze Internal States
Mental Map Found?
Map Used to Solve?
Map Changes Over Time?
2
Results: AI Builds a Mental Map, Then Forgets It

They found that AI models create a perfect mental map of the puzzle at the start, but then lose parts of it as they try to plan, which makes them fail, but fixing the map helps them win.

Puzzle Start
AI's Perfect Mental Map
AI Plans Moves
Mental Map Degrades
AI Fails to Solve