Conservative Meta-Agent Search: Architecture Adaptation under Non-Stationarity
This study formalizes meta-agent architecture search as sequential incumbent replacement with switching costs and hard risk constraints, introduces Conservative Meta-Agent Search (CMAS), and releases MetaInvest-Bench, a regime-switching simulator for evaluating meta-level generalization. Its value lies in an auditable, rollback-capable deployment procedure for non-stationary, delayed-feedback settings, with quantified risk metrics such as false-promotion rate.
01 ABSTRACT
The paper formulates meta-agent architecture search as sequential incumbent replacement with switching costs and hard risk constraints, and proposes CMAS, which evaluates candidates via paired shadow replay with interventional component credit. Experiments on the author-designed MetaInvest-Bench simulator over 96 streams report surrogate dynamic-oracle regret and held-out utility. The authors explicitly claim no alpha, and the calibration concerns only an update-level replay statistic on that benchmark, guaranteeing neither full-utility improvement nor sequence-level error control.
02 KEY FINDINGS
- CMAS models architecture search as sequential replacement: a candidate must exceed a pre-specified margin on a variance-penalized simultaneous score, then pass a simulated canary stage with pre-specified rollback before promotion.
- Across 96 streams, CMAS reduces normalized mean surrogate dynamic-oracle regret from 0.116 to 0.104 and raises held-out utility from 0.44 to 0.48 over the strongest adaptive baseline.
- When extra workers add no value, CMAS contracts to near-single-agent teams; its deployment inference cost is ~6% lower, though search-inclusive token cost is the highest among compared methods.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.LG