SciGroveBeta
Machine Learning

DeepWeb-Bench: A Deep Research Benchmark Demanding Massive Cross-Source Evidence and Long-Horizon Derivation

Sixiong Xie, Zhuofan Shi, Haiyang Shen, Jiuzheng Wang, Siqi Zhong, Mugeng Liu, Chongyang Pan, Peilun Jia, Baoqing Sun, Xiang Jing, Yun Ma

Featured June 3, 2026

Original Paper

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

Evaluating AI research agents requires moving beyond simple fact-finding to testing their ability to synthesize complex information and admit when evidence is missing, which this new benchmark achieves.

In depth
The paper introduces DEEPWEB-BENCH, a benchmark designed to evaluate the capabilities of frontier language models in complex, multi-step research tasks. Unlike previous benchmarks that focus on simple retrieval, this framework requires agents to perform massive evidence collection, cross-source reconciliation, and long-horizon multi-step derivation across a structured matrix of entities and analytical dimensions.

Key Takeaways

  • 1
    Retrieval is not the primary bottleneck, as Derivation and Calibration failures account for over 70% of errors.
  • 2
    Strong and weak models exhibit distinct failure modes, with strong models struggling with incomplete derivation and weak models prone to hallucinated precision.
  • 3
    Models demonstrate significant domain specialization, with cross-model agreement limited to .

Conceptual Flow

HIGH LEVEL
1
Methodology: The Matrix Approach

The researchers organized research tasks into a grid where agents must answer specific questions for different companies.

Companies

Research Questions

Fill in the grid

Completed Research Matrix

2
Results: The Performance Gap

The study found that AI models are good at finding information but struggle to combine it correctly to reach a conclusion.

Finding Facts
Combining Facts
Compare success rates
High Success
Low Success