Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books

01 ABSTRACT

The study reconstructs Polymarket L2 data into order-level LOBSTER format and trains market-making agents with JaxMARL-HFT on 417 real markets. The authors propose a cooperative decomposition into a quoter and a risk manager, and find that reward shaping (reward blending and post-fill markout) substantially improves the decomposed policy. However, the redesigned single-agent market maker quotes more frequently with smaller held-out losses, yet no configuration achieves positive average portfolio value in periodic held-out evaluation, and each configuration is trained with one seed.

02 KEY FINDINGS

1. Reconstructs Polymarket order-level data for simulation training and designs a cooperative multi-agent decomposition (quoter + risk manager) trained with recurrent IPPO.
2. Reward shaping (reward blending and post-fill markout) mitigates credit assignment and substantially improves the decomposed policy, yielding the best result.
3. The redesigned single-agent market maker quotes more frequently and incurs smaller held-out losses, but contributions of jointly changed observations, actions, and risk constraints are not separately identified.
4. No configuration establishes positive average portfolio value in periodic held-out evaluation, and each configuration is trained with one seed.