OOM-RL II: Reality Is an Oracle, Not a Debugger – Provenance-Constrained Diagnosis in Continually Evolving Agent-Engineered Systems

01 ABSTRACT

Reality can establish that an outcome occurred without identifying which evolving procedure produced it or why. Using a one-year quantitative trading case, the authors distinguish frozen instances, pre-specified adaptive procedures, and ad-hoc development, and propose archive-relative claim identifiability, an evidence hierarchy, and a prospective production-binding protocol. They argue diagnosis requires provenance constraints, but the protocol is proposed rather than prospectively validated.

02 KEY FINDINGS

1. In continually evolving agent-engineered systems, outcome evidence, evaluated-object identity, and causal explanation support distinct claims and cannot substitute for one another.
2. The trading system outperformed a broad market index over one year, but annual alpha was not statistically distinguishable from zero under the main retrospective specification, with adverse relative performance in some subperiods.
3. The authors propose a prospective production-binding protocol to resolve recommendation-to-runtime provenance, and an illustrative compatible-history example shows factual binding can fix a recommendation's referent without supplying its counterfactual effect.