We define the loss function (error between current predicted value and reality):

Replacing the predicted value with the formula of the estimation  we obtain

Now we define a Learning rate . It should be a small number, e.g. 0,01.

We start with two values for and . Suppose and .

At each step:

We calculate the partial derivative of the loss function with respect to , and replace the current values of to obtain the derivative value :

Similarly, we calculate the partial derivative with respect to , :

Now we update the current value of and using the equations

We repeat this process until the loss function is a very small value, or ideally 0. The final values of and are the optimum values.

Solving a minimization problem

from which we found the estimated values