ToolSpace: MCP Agent Training From Real Execution Traces
ToolSpace builds training environments for long-horizon agents out of what real tools actually did, not out of an invented database. The framework, proposed by Guanting Dong and 24 co-authors in a paper submitted to arXiv on November 3, 2026 (arXiv:2611.05036), mines execution traces from real MCP services and open agent corpora, then derives environments, tasks and checks from those traces.
The reported effect is large. Across seven long-horizon tool-use benchmarks, training with ToolSpace data raised Qwen3.6-35B-A3B from an average score of 23.0 to 41.0, and to 43.2 when reinforcement learning was added. GLM-5.2 rose from 44.6 to 56.6, a level the authors describe as on par with Claude Opus 5. Run in the other direction, the same pipeline turned two existing benchmarks into harder Pro versions on which seven frontier models dropped by up to 38.2 points.
The claim worth testing is not any single score. It is the ordering decision underneath them: dependencies that were witnessed in execution beat dependencies an LLM merely judged plausible, and the authors report that gap as clear in their ablations. If that holds up, a good share of synthetic agent training data being produced today is built on the weaker of the two signals.
The Reversed Synthesis Order
Existing MCP environment synthesis is data-centric, in the paper's framing. A database and a scenario are synthesized first, and the tools and tasks are derived from them afterwards. The authors name three consequences: tool interfaces may not match the real ones, tool compositions are not grounded in execution, and tasks rarely span more than one environment. Each consequence follows from the same root. When the invented layer comes first, everything downstream inherits its guesses.
ToolSpace reverses that order. The starting material is execution traces of real MCP services and open agent corpora, meaning records of tools being called in real sequences, with real inputs and real returns. The environment is then whatever those traces support. Interfaces match because they were observed in use, and compositions are grounded because the sequence actually happened.
Dependency Mining With Deterministic Rules
The mining step looks at adjacent calls inside a trace and extracts three kinds of dependency: data dependencies, where one call consumes what another produced; state dependencies, where a call only makes sense after an earlier call changed something; and retry dependencies, where a repeated call is part of the workflow rather than noise. The extraction uses deterministic rules rather than LLM judgment, a choice the paper later tests directly.
The mined dependencies form a weighted tool-composition graph, and clustering that graph produces the environments. This is where the reversal pays off a second time. Because the graph edges come from real traces, a cluster can span tools that were actually composed together, instead of staying inside one hand-designed scenario.
State Synthesis and Replay-Verified Tasks
State is synthesized selectively. Databases are built only for state-mutating environments, the ones where calls change stored state and later calls depend on it. Query and stateless tools do not get a synthetic database at all; they replay their recorded returns from the traces. That split keeps the synthesis budget on the part of the environment where state actually matters, and it avoids inventing database contents for tools whose behaviour is already fully recorded.
Task generation then runs state-aware random walks with rollback over the environment, producing chains of calls that respect the current state at each step. Each chain is decompiled into a natural-language task with assertion-based checks. The filter at the end is strict: only samples that replay successfully from the initial state are kept. A generated task that cannot be executed end to end is discarded rather than repaired, which keeps the training set limited to tasks with a witnessed solution path.
Harder Pro Versions of Two Existing Benchmarks
Applied to evaluation instead of training, the pipeline produced Pro versions of AutomationBench and tau3-Bench Banking. On these harder versions, seven frontier models dropped by up to 38.2 points. The abstract does not name the seven models or give the per-model breakdown, so the 38.2 figure is a maximum, not a typical drop.
Even read conservatively, the result is a useful stress signal. If environments rebuilt from real execution traces make frontier models lose that much ground, part of their score on the original benchmarks was measuring how well the synthetic environment matched the model, rather than how well the model handles real tool behaviour. That reading is an interpretation of the result, and the paper itself frames the Pro versions as harder benchmarks, not as a verdict on the originals.
Training Gains Across Seven Benchmarks
The training results are averages across the seven benchmarks. Qwen3.6-35B-A3B nearly doubles, from 23.0 to 41.0, with reinforcement learning on top of the ToolSpace data taking it to 43.2. GLM-5.2 gains twelve points, from 44.6 to 56.6, reaching the level the authors report for Claude Opus 5. The smaller model gaining more is consistent with the method supplying something it lacked: realistic multi-step tool behaviour to learn from, rather than additional raw capability.
Two ablation findings matter more than the headline numbers for teams designing their own pipelines. First, execution-witnessed edges clearly outperformed LLM-judged edges in the composition graph, which is direct evidence for the deterministic mining choice. Second, performance kept improving with data and environment scale, with no plateau reported in the abstract. Both findings are author-reported results from the paper's own experiments, and independent reproduction is not yet available.
Limits and Unconfirmed Points
The MCP scope is the practical limit to keep in view. ToolSpace starts from traces of real MCP services and open agent corpora, so its evidence base is the MCP ecosystem. A team running agents over proprietary internal APIs has no guarantee that the same gains transfer, because their traces, tool shapes and failure modes may differ from the corpora used here. The method itself, mining adjacent calls with rules instead of asking a model to guess dependencies, is portable in principle; the measured numbers are not automatically portable with it.
Related Reading
For a different angle on agent evaluation, see the LifeVal report, where GPT-6 Astra was scored on daily tasks. For agents that keep working in the background with human decision points, see the agent-autoloop report.
FAQ
What is ToolSpace?
ToolSpace is a tool-centric framework proposed by Guanting Dong and 24 co-authors in a paper submitted to arXiv on November 3, 2026 (arXiv:2611.05036). It synthesizes training environments for long-horizon agents from execution traces of real MCP services and open agent corpora, reversing the usual data-centric order.
How does ToolSpace create its tasks?
It mines data, state and retry dependencies from adjacent calls in execution traces using deterministic rules, clusters the weighted tool-composition graph into environments, then runs state-aware random walks with rollback. Each chain is decompiled into a natural-language task with assertion-based checks, and only samples that replay successfully from the initial state are kept.
What score gains does the paper report?
Across seven long-horizon tool-use benchmarks, ToolSpace raises Qwen3.6-35B-A3B from 23.0 to 41.0 on average, and to 43.2 with reinforcement learning. GLM-5.2 rises from 44.6 to 56.6, which the authors describe as on par with Claude Opus 5.
What are the Pro versions of AutomationBench and tau3-Bench Banking?
They are harder benchmark versions produced by applying the ToolSpace pipeline to AutomationBench and tau3-Bench Banking. On these Pro versions, seven frontier models dropped by up to 38.2 points.
Is the ToolSpace code or data publicly available?
The abstract does not confirm a release. Whether the code, the synthesized environments or the training data will be published is not stated in the material the paper summary provides.