# The Asymptotic Convergence-Rate of Q-learning

### The Asymptotic Convergence-Rate of Q-learning

the-asymptotic-convergence-rate-of-q-learning

The asymptotic rate of convergence of Q-learning is Ο( 1/t^{R(1-γ) }), if R(1-γ)<0.5, where R=P_{min}/P_{max}, P is state-action occupation frequency.

|Q_{t} (x,a) − Q*(x,a)| < B/t^{R(1-γ)}

Convergence-rate is the difference between True value and Optimum value, i.e., the smaller it is, the faster convergence Q-learning is. We hope the Ο( 1/t^{R(1-γ) }) should be as small as possible, which means the R is bigger, i.e., the on-policy distribution is higher, the state space should be smaller.