State-Space Foundation Model for Limit Order Books Encodes Decodable Order-Flow Imbalance
ORIGINAL / Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books
This study examines an 8.2M-parameter Mamba-3 state-space model trained autoregressively on limit order book message streams from eight US equities, finding that order-flow imbalance is linearly decodable from its internal state at R²≈0.59 and that this representation transfers to unseen stocks and extends to traded volume and volatility. It offers empirical clues for interpretability and controllability of market-simulation foundation models.
01 ABSTRACT
The authors train an 8.2M-parameter Mamba-3 state-space model to autoregressively generate message streams for eight large-cap US equities. They report that order-flow imbalance is linearly decodable from the model's internal state at R²≈0.59, that this representation transfers to unseen stocks, and that it holds across traded volume and volatility. Pushing the state along a feature's axis is said to shift next-message beliefs monotonically, which the authors interpret as evidence of a world model.
02 KEY FINDINGS
- Order-flow imbalance is linearly decodable from the model's internal state at R²≈0.59, above controls and baselines.
- The representation transfers to stocks unseen during training and holds across traded volume and volatility.
- Intervening along a feature axis monotonically shifts next-message beliefs, which the authors say is consistent with a world model.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.LG · ARXIV Q-FIN