Solving realistic multi-stage stochastic programs is among the most challenging domains in optimization. At least four barriers exist: 1) the model size grows exponentially with number of time steps and branches (the curse of dimensionality); 2) the dynamic/behavioral nature of financial markets causes complexity when building representative generative stochastic models; 3) the imbalanced incentive structure (buy and sell side) and the lack of education of many investors can give rise to overly simplistic recommendations; and 4) the rise of the AI and Reinforcement Learning (RL) technologies makes it difficult to predict areas of major changes and advancements.

Still, the usefulness of reinforcement learning has become well understood in specific environments. For example, RL can mitigate the curse of dimensionality by replacing exact Bellman expectation calculations with sample-based approximations and using neural networks to generalize across high-dimensional state spaces. Although RL has been extensively studied in robotics, gaming, and automated driving, among others, its application to multi-period financial planning remains at an early stage. This talk discusses three areas for potential applications of RL for improving investment performance, along with a case study. The presentation concludes with the author’s view of current limitations of RL and related algorithms and possible future research.