Stochastic gradient descent

De wiki.datafranca.org
Aller à : navigation, rechercher

Domaine

Vocabulary
Claude
Apprentissage profond


Définition



Termes privilégiés

descente de gradient stochastique

Anglais

Stochastic gradient descent

SGD

Stochastic Gradient Descent (Wikipedia) is a gradient-based optimization algorithm that is used to learn network parameters during the training phase. The gradients are typically calculated using the backpropagation algorithm. In practice, people use the minibatch version of SGD, where the parameter updates are performed based on a batch instead of a single example, increasing computational efficiency. Many extensions to vanilla SGD exist, including Momentum, Adagrad, rmsprop, Adadelta or Adam. • Adaptive Subgradient Methods for Online Learning and Stochastic Optimization