GPT

Language Models are Few-Shot Learners (GPT-3)

Notes

175B parameters. In-context learning emerges at scale. Changed the field.

Language Models are Unsupervised Multitask Learners (GPT-2)

Notes

Showed large LMs can perform tasks zero-shot. Introduced the scaling intuition.

Neural Language Models: From RNNs to Transformers

The evolution of neural sequence prediction, and how it connects to classical methods