MatMul

Définition

XXXXXXXXX

Français

XXXXXXXXX

Anglais

Matrix multiplication

Scalable MatMul-free Language Modeling

 Matrix multiplication (MatMul) typically dominates the overall computational cost of large language models (LLMs). This cost only grows as LLMs scale to larger embedding dimensions and context lengths. In this work, we show that MatMul operations can be completely eliminated from LLMs while maintaining strong performance at billion-parameter scales. 

Source

Source : huggingface