ANN AI calculus

Gradient descent

  • A heuristic method that starts at a random point and iteratively moves in the direction that decreases a function you want to minimize
  • In ANNs this is the loss function
    • The step size that is used to decrease the result of the function is called the learning rate