← 🧠 Foundations of LLMs & GenAINEXT IN FOUNDATIONS OF LLMS & GENAIReward Models→
Core
RLHF: Reinforcement Learning from Human Feedback
RLHF is how a raw next-token predictor becomes a helpful, harmless assistant. It has three stages: supervised fine-tuning on demonstrations, training a reward model on human preference comparisons, then optimizing the model against that reward (with a KL penalty to stay close to the base). It aligns the model to human preferences that are hard to specify as a loss. Applied-AI interviews probe it because it explains why instruct models behave well, where alignment data comes from, and the failure modes (reward hacking, sycophancy).
a free account unlocks the core curriculum tier · no card
COURSES COVERING THIS TOPIC
No lesson covers this one directly yet. These teach the surrounding topic from the beginning.
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
LLM & GenAI FundamentalsWalk through RLHF, then explain DPO and why it has largely displaced PPO-based RLHF.→LLM & GenAI FundamentalsExplain PPO and GRPO for LLM alignment. Why did GRPO drop the value model?→LLM & GenAI FundamentalsYour RLHF model games the reward model instead of being genuinely helpful. How do you stop reward hacking?→ML Infrastructure & GPUsDesign the training infrastructure for RLHF/PPO. Why are there four model copies and how do you fit them?→LLM & GenAI FundamentalsWhat is reward-model overoptimization, and how do you detect and bound it during RLHF?→LLM & GenAI FundamentalsWhat is Constitutional AI / RLAIF, and how does it differ from RLHF?→
COMPANIES THAT ASSUME THIS