Menu

    Blog Stats

    • 139,695 hits

Function Approximation

The Asymptotic Convergence-Rate of Q-learning

July 23, 2019

The Asymptotic Convergence-Rate of Q-learning the-asymptotic-convergence-rate-of-q-learning The asymptotic rate of convergence of Q-learning is Ο( 1/tR(1-γ) ), if R(1-γ)<0.5, where R=Pmin/Pmax, P is state-action occupation frequency. |Qt (x,a) − Q*(x,a)| < B/tR(1-γ) Convergence-rate is the difference between True value and Optimum value, i.e., the smaller it is, the faster convergence Q-learning is. We hope the Ο( 1/tR(1-γ) ) should… read more »

Policy Gradient Methods for Reinforcement Learning with Function Approximation

January 15, 2025

  Policy Gradient Methods for Reinforcement Learning with Function Approximation Math Analysis Markov Decision Processes and Policy Gradient So far in this book almost all the methods have been action-value methods; they learned the values of actions and then selected actions based on their estimated action values; their policies would not even exist without the… read more »

Sidebar