Policy Memory and Belief Revision for Language-Model Agents in Non-Stationary Pair Trading
ORIGINAL / MOIRA: Policy Memory and Belief Revision for Language-Model Agents in Non-Stationary Pair Trading
Proposes MOIRA, a framework that formulates pair trading as hierarchical reinforcement learning and uses large language models to parameterize both high-level pair selection and low-level execution, optimized via prompt updates and textual feedback without gradient fine-tuning.
01 ABSTRACT
The paper studies credit assignment in hierarchical sequential decision-making, using pair trading where high-level semantic choices constrain downstream execution under delayed and ambiguous feedback. The authors propose a language-driven optimization framework in which both policies are LLM-parameterized and updated exclusively through prompts using trajectory- and episode-level textual feedback to adapt to non-stationary markets. Experiments on real market data show improvements over traditional and LLM-based baselines.
02 KEY FINDINGS
- Formulates pair trading as a hierarchical reinforcement learning problem, separating high-level pair selection from low-level execution.
- Both high-level and low-level policies are parameterized by large language models and optimized solely through prompt updates, without gradient fine-tuning.
- Uses trajectory- and episode-level textual feedback for credit assignment, reducing non-stationarity across hierarchy levels and enabling targeted adaptation.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.CL