RLHF

Deep Reinforcement Learning from Human Preferences

Notes

Foundational RLHF paper. Learning reward models from human comparisons.

Training language models to follow instructions with human feedback (InstructGPT)

Notes

RLHF applied to GPT-3. The bridge from raw LM to useful assistant.

Instrumental Goals and Hidden Codes in RLHF'd Language Models

How RLHF-trained language models may develop instrumental goals, and the information-theoretic limits on detecting them.