We only have observations. But we usually have a lot of them.

We have observations, our system makes predictions, and receives occasional rewards.

We have observations and labels, labels are in the same format as the ones our system will have to predict. Our learning process is driven by the difference between each expected and predicted outputs.

Training set in the form:

x (observation)

Training set in the form:

x (observation) -> continuous y (target)

Training set in the form:

x (observation) -> discrete y (target)

Gap between the best possible solution in our search space and the optimal solution, caused by approximation error.

Sensibility of the optimal solution found (in our search space) to the training set.

Small errors in the data which causes inexact search space exploration.