OOM-RL: Aligning Multi-Agent Systems via Real Capital Loss Constraints
ORIGINAL / OOM-RL: Out-of-Money Reinforcement Learning Market-Driven Alignment for LLM-Based Multi-Agent Systems
Proposes using capital depletion in live financial markets as an externally imposed negative gradient for multi-agent system development, instead of human feedback that may induce sycophancy, and documents the resulting STDAW workflow and RO-Lock state lock.
01 ABSTRACT
The paper argues that current human-feedback and AI-feedback alignment can induce sycophancy, while execution-based environments can be bypassed by unconstrained agents through test evasion. It introduces OOM-RL, which deploys agents in non-stationary, high-friction live markets and uses real execution costs, slippage, and capital depletion as externally imposed negative signals, then reports a 20-month longitudinal evolution from a high-turnover sycophantic baseline to a liquidity-aware architecture formalized as STDAW and RO-Lock. The authors state the main contribution is treating externally imposed economic consequences as an epistemic constraint, not demonstrating investment capability.
02 KEY FINDINGS
- OOM-RL uses live trading losses as an externally imposed negative gradient instead of human feedback signals
- The system evolved over about 20 months and was formalized as STDAW and a Byzantine-inspired uni-directional state lock, RO-Lock
- During the final 94-trading-day window, the annualized Sharpe ratio was approximately 2.06, an observational result
AI GENERATED SUMMARY / DISCOVERED BY ARXIV Q-FIN