Adagrad

Domaine

Vocabulary
Apprentissage profond

Anglais

Adagrad

Adagrad is an adaptive learning rate algorithms that keeps track of the squared gradients over time and automatically adapts the learning rate per-parameter. It can be used instead of vanilla SGD and is particularly helpful for sparse data, where it assigns a higher learning rate to infrequently updated parameters. • Adaptive Subgradient Methods for Online Learning and Stochastic Optimization • Stanford CS231n: Optimization Algorithms • An overview of gradient descent optimization algorithms