What is a Transformer?

Transformer Neural Network Architecture — Self-attention mechanism and transformer model topology.Source: Research Archive

The Transformer(Technology) is a neural-network architecture introduced in 2017.

Self-attention

Each token looks at every other token in the input and decides how much to weight it. That single mechanism replaced the recurrent networks that came before.

Why it scales

Because self-attention is parallel, transformers train efficiently on modern GPU(Hardware)s. That is what made Large Language Model(Technology)s possible.

Explore Further

Entities mentioned

Neural Network(Technology)Transformer(Technology)GPU(Technology)Large Language Models(Concept)