Risk-sensitive Decision-making

A novel approach to long-term risk-sensitive control in stochastic systems through the so-called ergodic-risk criteria

In stochastic systems, balancing performance with resilience to rare events is critical. Traditional approaches often focus on finite-horizon risk, overlooking long-term cumulative risks unless by imposing strong assumptions on process noise (missing reference). My recent work (Talebi & Li, 2024) addresses this by introducing ergodic-risk criteria through probabilistic limiting theorems that can account for noise with unbounded moments. Then through primal-dual optimization techniques, an optimal policy is obtained that balances average performance with ergodic-risk constraints. This extends risk-sensitive methods to a more versatile framework for long-term risk-sensitive control in stochastic systems capable of handling extreme events. Central to my analysis is a tailored Functional Central Limit Theorem (FCLT), where I can establish convergence even in non-stationary environments with heavy-tailed noise.

This is achieved by utilizing ergodic theory (Meyn et al., 2009), ensuring uniform ergodicity of controlled processes where stability is maintained even with heavy-tailed noise, provided it has a finite fourth moment.

Key references:

  1. Talebi, S., & Li, N. (2024). Uniform Ergodicity and Ergodic-Risk Constrained Policy Optimization. ArXiv Preprint ArXiv:2409.10767.
  2. Meyn, S., Tweedie, R. L., & Glynn, P. W. (2009). Markov Chains and Stochastic Stability (2nd ed.). Cambridge University Press.