Encoder–Decoder and Cross-Attention

BERT reads both ways, GPT writes forwards, and T5 does both, joined by cross-attention: decoder queries against encoder keys. Real cross-attention weights from a trained English-to-French translator, and what happens when it may only read in step.