Modèle sans MatMul

Définition

xxxxx

Français

modèles sans multiplication matricielle

modèles sans MatMul

modèles sans multiplication de matrices

Anglais

MatMul free models

MatMul-free language models

Matmul-free LM

MatMul free models are large language models in wich MatMul operations are completly eliminated. These models are both effective and ressource-efficient, almost like biological neural networks, since the replacement of multiplication-heavy operations with simpler additions and substraction enabled by ternary weight.

Sources

Source : arxiv

Source : datacamp