What is the main idea behind multi-step bootstrapping in Reinforcement Learning?

TD(0) can be used for the solution of

Which one of these is a key feature of TD Learning?

Consider the undiscounted, episodic MDP below. There are four actions possible in each state, A = {up, down, right, left}, which deterministically cause the corresponding state transitions, except that actions that would take the agent off the grid in fact leave the state unchanged. The right half of the figure shows the value of each state under the equiprobable random policy. If π is the equiprobable random policy, what is q(5, down)?

Consider the undiscounted, episodic MDP below. There are four actions possible in each state, A = {up, down, right, left}, which deterministically cause the corresponding state transitions, except that actions that would take the agent off the grid in fact leave the state unchanged. The right half of the figure shows the value of each state under the equiprobable random policy. If π is the equiprobable random policy, what is v(15)?

When it is not possible to determine a policy that is greedy with respect to the value functions v_π, q_π (Select all that apply).

Which of the following is a requirement on the behavior policy b for using off-policy Monte Carlo policy evaluation?  This is called the assumption of coverage.

When does Monte Carlo prediction perform its first update?

After 99 episodes, the estimated value of the state s is 5.8. For the next episode, for the state s the agent receives a Return G₁₀₀=7. What will be the new estimate of the state value of the state s?

Which approach can not find an optimal deterministic policy? (Select all that apply)