Machine Learning

Why Large Language Models Fail at Tabular Prediction

Marta Garnelo, Wojciech M. Czarnecki

Featured August 5, 2026

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

Big AI models that write text really well surprisingly struggle with simple table predictions, mainly because they get confused when there are too many columns, even if the information is still clear.

In depth

The paper systematically investigates why Large Language Models (LLMs) struggle with tabular data prediction. Through controlled experiments, the authors falsify several common assumptions, such as issues with data separability, CSV formatting, or numeric tokenization. They conclusively identify input dimensionality as the primary culprit: LLM performance uniquely degrades as the number of features increases, unlike classical models. In low dimensions, LLMs behave like distance-based classifiers, but their high-dimensional failure mechanism remains distinct and unexplained by standard models.

Key Takeaways

1
LLMs consistently underperform classical methods on tabular prediction, a gap not explained by data separability, CSV format, numeric tokenization, or test batch size.
2
Input dimensionality is the decisive factor: LLM accuracy uniquely collapses as the number of features grows, even when information content is preserved.
3
In low-dimensional settings, LLMs behave like local, distance-based classifiers, but their high-dimensional failure is not reproducible by any standard classical model, even with added noise.

Conceptual Flow

HIGH LEVEL

Methodology: Identifying the Root Cause of LLM Failure

The study tested why big AI models fail on tables by checking five common ideas, finding that only having too many data columns was the real problem.

Big AI Model

Table Data

Test 5 Ideas

Only 'Too Many Columns' is Problem

Results: LLM Behavior in Low vs. High Dimensions

They found that big AI models work like simple 'find-the-neighbor' rules in small tables, but completely break down in big tables, unlike old-school methods.

Small Table

Big Table

AI Model Behavior

Simple Rules (Small Table)

Breaks Down (Big Table)