Monte Carlo Reinforcement Learning

Monte Carlo Reinforcement Learning

• Generate random rollouts (path through state space)
• Sample and average the returns from state-action pairs
• As number of visits to states increases, $V^{\pi }(s)$ will converge on optimal values

Like Value Iteration, Monte Carlo can use discounted rewards from a given state (Return): $G=R_{t+1}+\gamma R_{t+2}+{\gamma }^2{R}_{t+3}+\dots +{\gamma }^{T-1}{R}_T$ Value function is the expected return of a given state: $v_{\pi }(s)=E_{\pi }[G_t|S_t=s]$