After 12 mornings you have taken the tram 6 times (Q = −19), the bus 3 times (Q = −20.3) and cycled 3 times (Q = −18). You use UCB with c = 2 to choose on morning t = 13:

A_t = argmax_a [ Q_t(a) + c · √( ln t / N_t(a) ) ],   ln 13 ≈ 2.565

What is the UCB value Q_t(a) + c · √( ln t / N_t(a) ) of the option that UCB chooses? Give your answer to two decimal places.

Your estimates are Q(Tram) = −25, Q(Bus) = −19 and Q(Cycling) = −25. You use ε-greedy with ε = 0.3, choosing uniformly among all three options when exploring. What is the probability that you take the bus tomorrow?

You track Q(Cycling) with a constant step size α = 0.2 instead of sample averages. Which statement is true?

You learn preferences with a gradient bandit. Your average reward so far is R̄ = −19. This morning you cycle on a snowy morning and arrive in 30 minutes (reward −30). What happens to the preferences?

In Reinforcement learning, the algorithm passively observes labeled examples without interacting with an environment.

Supervised learning does not require labeled data.

Labeling handwritten digits for recognition models is a use case for Supervised Learning.

Learning to recognize faces from labeled photos is Reinforcement Learning.

The policy tells the agent how valuable a state is.