SciGroveBeta
Machine Learning

Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

Lei Bai, Zongsheng Cao

Featured July 1, 2026

Original Paper

This analysis was generated by SciGrove. Upload your own PDFs or enter a DOI — and get the same AI breakdown on any paper.

Get started

AI-generated analysis — This is SciGrove's AI interpretation of the paper, not peer-reviewed content. Always refer to the original paper.

Simply

Instead of making models huge, this paper teaches a smaller 35B AI agent to think many steps ahead and use tools smartly, combining specialized skills from different "teacher" AIs into one powerful agent.

In depth
The paper introduces Agents-A1, a 35B Mixture-of-Experts (MoE) agentic model that achieves performance comparable to trillion-parameter models by scaling the agent horizon rather than model parameters. This is accomplished through a novel knowledge-action infrastructure and a three-stage training recipe, including a multi-teacher domain-routed on-policy distillation with salient vocabulary alignment, which unifies diverse specialized agent abilities into a single model.

Key Takeaways

  • 1
    Agents-A1, a 35B MoE model, matches or outperforms 1T-parameter models on long-horizon agent benchmarks by focusing on agent-horizon scaling.
  • 2
    The authors developed a Long-Horizon Knowledge-Action Infrastructure to provide verifiable, process-level supervision for multi-turn interactions with external knowledge and tools.
  • 3
    A Domain-Routed On-Policy Distillation with Salient Vocabulary Alignment method is proposed to efficiently consolidate specialized teacher models into a unified student agent, mitigating conflicts from heterogeneous reasoning patterns.

Conceptual Flow

HIGH LEVEL
1
Methodology (The "Logic")

The AI learns in three steps: first, general skills; then, specialized skills from different "teacher" AIs; finally, it combines all these skills into one smart agent.

General Data
Train Broad Skills
General Agent
2
Results (The "Impact")

This new smart agent, even though it's smaller, can solve hard, multi-step problems just as well as or better than much bigger, more expensive AIs.

Huge AI Model
New Smart Agent
Compare Performance
Hard Problem 1 Solved
Hard Problem 2 Solved
Hard Problem 3 Solved