SynAct: Decision-Level Active Feedback Learning for LLM Agents
ORIGINAL / SynAct: Synergistic Active Feedback Learning for LLM Agents
This work introduces SynAct, a framework that allocates expert feedback at the individual decision level rather than at the task level, using mutual information to identify consequential decisions and improve multi-step LLM agent accuracy under scarce expert attention.
01 ABSTRACT
LLM agents can fail through recoverable slips or knowledge gaps, with only the latter requiring external feedback. SynAct treats expert attention as a binding constraint, ranks decisions by mutual information between action and terminal outcome, queries experts at high-value decisions, and distills corrections into reusable rules. Across four benchmarks and four models, it improves held-out accuracy in all 16 settings, by an average of 11.7 percentage points.
02 KEY FINDINGS
- SynAct allocates expert feedback at the level of individual decisions rather than whole tasks, addressing scarce expert attention.
- It uses repeated rollouts to compute mutual information between actions and outcomes, identifying decisions that separate terminal results, with theoretical separation conditions.
- Evaluations across multiple benchmarks and models show an average improvement of 11.7 percentage points, and a single targeted correction outperforms seven unaimed retries.
- The same signal improves inference-time routing, outperforming budget-matched cascade, uncertainty, and expert-judge routers in most settings.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.LG