Core

Reward Models

A reward model turns human preference comparisons into a scalar score for any response, the signal RLHF optimizes against. It is trained on pairs of responses labeled by which a human preferred, learning to rank rather than to produce text. Its imperfections drive RLHF's failure modes: reward hacking (gaming the proxy) and staleness as the policy drifts off-distribution. Applied-AI interviews probe it because it explains where the alignment signal comes from and why it is gameable, and it generalizes to LLM-as-judge evaluation.

a free account unlocks the core curriculum tier · no card

COURSES COVERING THIS TOPIC

No lesson covers this one directly yet. These teach the surrounding topic from the beginning.

Applied AI Engineering·The interviewPremium12mTurning this course into a study planA concrete four-week plan mapping the seven modules onto the question bank, plus what to do differently if your interview is next week rather than next month.

RELATED CONCEPTS

RLHF: Reinforcement Learning from Human Feedback DPO and Preference-Optimization Variants LLM-as-a-Judge Constitutional AI and RLAIF

PRACTICE THIS IN REAL QUESTIONS

LLM & GenAI FundamentalsWalk through RLHF, then explain DPO and why it has largely displaced PPO-based RLHF.→LLM & GenAI FundamentalsAfter RLHF, your model is safer but worse at hard tasks. How do you manage the alignment tax?→LLM & GenAI FundamentalsYour RLHF model games the reward model instead of being genuinely helpful. How do you stop reward hacking?→LLM & GenAI FundamentalsExplain PPO and GRPO for LLM alignment. Why did GRPO drop the value model?→LLM & GenAI FundamentalsWhat is Constitutional AI / RLAIF, and how does it differ from RLHF?→LLM & GenAI FundamentalsBeyond DPO: what are SimPO, KTO, and ORPO, and why do these alignment variants exist?→

COMPANIES THAT ASSUME THIS

OpenAI Anthropic Google Microsoft Scale AI Decagon

NEXT IN FOUNDATIONS OF LLMS & GENAIConstitutional AI and RLAIF→