Gradient descent
- A heuristic method that starts at a random point and iteratively moves in the direction that decreases a function you want to minimize
- In ANNs this is the loss function
- The step size that is used to decrease the result of the function is called the learning rate