Transition Model

We can write a transition model to describe these actions. Since the actions are stochastic, the model looks like:

$$P(s^{\prime }|s,a)$$

Where $a$ is the action that takes the agent from $s$ to $s^{\prime }$.

Transitions are assumed to be first order Markovian, that is, they only depend on the current and next states. So we can write a large set of probability tables that describe all possible actions executed in all possible states.