Deep Reinforcement Learning for Optimal Trading with Partial Information

01 ABSTRACT

The paper studies optimal trading when the trading signal follows an Ornstein-Uhlenbeck process with regime-switching parameters, formulating it as a partially observable Markov decision problem in which the trader must infer latent information from the history of observable processes. The authors propose three reinforcement-learning-based approaches with recurrent neural networks for filtering and trading: a one-step method directly encoding GRU hidden states into the RL trader, and two two-step methods using posterior regime probability estimates for mean-reverting regimes or forecasts of the next signal value. Simulations and an equity pairs-trading application indicate that providing posterior probability estimates achieves superior cumulative rewards and more interpretable strategies, leading the authors to conclude that the quality and structure of information supplied to the agent matter for trading performance and policy interpretability.

02 KEY FINDINGS

1. The problem assumes the trading signal follows an Ornstein-Uhlenbeck process with regime-switching parameters, with the trader inferring latent information only from the history of observable processes, forming a partially observable Markov decision problem.
2. Three RNN-plus-RL methods are compared: a one-step approach directly encoding GRU hidden states, and two two-step methods using posterior regime probability estimates for mean-reverting regimes or next-signal-value forecasts.
3. Simulation and equity pairs-trading results show that the posterior-probability method achieves superior cumulative rewards and more interpretable strategies.