1-ε

ε

1-ε+ε/A

ε/A

The number of states

The number of states visited during the episode

The number of possible state-action value pairs

The number of possible actions in each state

To penalize long episodes

To prioritize short-term rewards over long-term rewards

To balance exploration and exploitation

To weigh the importance of future rewards

Model of the environment

Value Function

Policy

Reward Signal

Always take the action that gives you the best reward at that point.

That all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal (called reward)

Ignore rewards and find other signals.

That all of what we mean by goals and purposes can be well thought of as the minimization of the expected value of the cumulative sum of a received scalar signal (called reward)

The agent takes the same action at each step during an episode.

There is a predefined max size of the steps in the episode and the number of steps can not exceed that number

The number of steps in an episode is always the same.

Each episode can have a different number of steps.