Executable Environments Scale Agentic Tool-Use Supervision
ORIGINAL / Davinci-Tool: Scaling Agentic Tool-Use Supervision with Verifiable Executable Environments
The work introduces a dataset and model that scale environment coverage and outcome verification together, unifying restorable states, execution traces, outcomes, and verifiers as supervision substrate, and reusing states and verifiers to build preference data. It is directly relevant to researchers and engineering teams training reliable multi-step tool-use agents.
01 ABSTRACT
The paper argues that reliable tool use requires pairing diverse multi-step workflows with executable evidence, and that scaling trajectories or verification alone is insufficient because new environments introduce states, rules, and dependencies that must be reflected in success criteria. The authors build Davinci-ToolData with two pipelines, Execution-Grounded Task Synthesis and Executable Environment Synthesis, yielding restorable states, execution traces, outcomes, and verifiers, then use supervised fine-tuning followed by outcome-grounded preference construction for direct preference optimization. Verifiable facts: the data contains 30,395 verified trajectories spanning 2,260 invoked tool names, the model has 30B parameters, and reported gains are from 52.4 to 88.2 on τ²-bench, 30.4 to 62.8 on BFCL v4 multi-turn, and 16.9 to 56.2 on MCP-Atlas; the authors argue this route outperforms a larger model with far fewer parameters and release code.
02 KEY FINDINGS
- Davinci-ToolData unifies restorable states, execution traces, outcomes, and verifiers as a supervision substrate.
- Two pipelines, Execution-Grounded Task Synthesis and Executable Environment Synthesis, yield 30,395 verified trajectories spanning 2,260 tool names.
- Training first uses supervised fine-tuning, then reuses EES states and verifiers to compare alternative actions and construct outcome-grounded preferences for DPO.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.CL · ARXIV CS.LG