Set the following training parameters:

• alpha: 0.1
• gamma: 0.95
• epsilon strategy: decay (eps start=1, eps end=0.05)
• episodes: 5000
• seed: 2026

Save the policy as Slippery Decay. Look at the q-table heatmap (attach it here also).

Elaborate on the differences between the Slippery Decay heatmap to the:

• Slippery Greedy heatmap
• Non-Slippery Decay heatmap

Now run the agent experiment twice (usual settings - 2000 episodes/1000 max steps/seed2026):

• use the (improved in the previous question) Slippery Decay policy on the slippery environment
• use the Non-Slippery Decay policy on the slippery environment

How does the non-slippery policy perform in the slippery environment? Write how often the slippery agent manages to reach the goal compared the the non-slippery one. What might be the reason behind the difference in performance between the two policies? How does the non-slippery agent try to tackle the environment compared to the slippery one?

Run the agent experiment (again 2000 episodes, 1000 steps max, seed 2026) with the slippery decay agent. How many successful runs are there now?

Now keep the same training settings, except switch the epsilon strategy to Decay. Keep start and minimum epsilon at 1 and 0.05 respectively. Save the policy as Non-Slippery Decay.

When you now train the agent, what is the difference between the greedy heatmap and the decay heatmap here? 

Also, how does the agent behave when you let him tackle the environment (agent playback)? Compare this to the behavior of the greedy agent when you do the agent playback.

Attach the q-table heatmap plot of both agents here as well.

Set the following training parameters:

• alpha: 0.1
• gamma: 0.95
• epsilon strategy: off (greedy)
• episodes: 5000
• seed: 2026

Save the policy as Slippery Greedy. Attach the heatmap plot here.

What do you notice about the heatmap and how does it differ to the non-slippery greedy heatmap?

Tweak your training hyperparameters. How can you ensure that about 2/3 (~1300) runs of the slippery decay agent are successful? Include a screenshot with the results and the plot here.

Run the agent experiment here with the Non-Slippery Decay policy (2000 runs, 1000 max steps, seed 2026).

How do you interpret the result? Attach the plot here.

Run the agent experiment (again 2000 episodes, 1000 steps max, seed 2026) with the slippery greedy agent. How many successful runs are there?

Now check 'Is Slippery' on Tab 1 and repeat the experiment with the same settings (runs, max steps, seed). What do you observe? Also include the plot of the experiment again here.

Can a random agent handle this environment reasonably well? Attach the plot of the experiment you just did.