Worse

Better

Value iteration is always faster than policy iteration.

Value iteration computes policies directly, while policy iteration computes values directly.

Value iteration updates both policies and values simultaneously, while policy iteration alternates between evaluation and improvement steps.

Hill Climbing search

Breadth-First search

Simulated Annealing search

Child 1: 1 4 6 2 6 7 6 1, Child 2: 8 5 3 4 5 7 2 3

Child 1: 1 4 6 2 4 6 7 6, Child 2: 8 5 3 4 5 7 2 3

Child 1: 1 4 6 5 7 6 1, Child 2: 8 5 3 2 5 7 2 3

Child 1: 1 4 6 2 6 7 2 3, Child 2: 8 5 3 4 5 7 6 1

determines how much the value of a state considers future states. A higher places more emphasis on distant future rewards, while helps algorithms converge by ensuring values do not diverge. 

controls the randomness of the agent's actions during policy search. A smaller results in more exploratory behavior, while a larger leads to more deterministic policies.  

determines the immediate reward only and ignores future states. A higher improves convergence by focusing solely on the present.

balances the trade-off between exploration and exploitation. A smaller focuses on exploring unknown states, while a larger exploits known rewards.  

Model-based learning involves building or using an approximate model of the environment's transition probabilities, while model-free learning directly estimates values and policies without using the transition probability distribution. 

Model-based learning relies on trial-and-error to learn optimal actions, while model-free learning computes exact solutions based on a known model of the environment. 

Model-based learning assumes the environment's transition dynamics are deterministic, while model-free learning assumes the environment's transitions are probabilistic.