← 🧠 Foundations of LLMs & GenAI
Core
Context Rot and Long-Context Failure Modes
Context rot is the practical degradation of model quality as the input window fills up, even when the official window is a million tokens. Information in the middle gets ignored, attention concentrates on the first and last tokens, and reasoning that needs several scattered facts at once falls apart. Applied AI interviews probe it because candidates routinely assume a large window is a substitute for retrieval, and it is not.
a free account unlocks the core curriculum tier · no card
TAUGHT IN THESE COURSES
This concept is a reference. These lessons put it in sequence, with what comes before and after it.
Applied AI Engineering·How language models work, enough to build onFree13mA big context window does not mean you should fill itContext windows keep growing, and the instinct is to stop thinking about what goes in them. This lesson closes the module by showing why capacity is not recall, what actually degrades as the window fills, and how to decide between more context and better context.Applied AI Engineering·Prompting as an engineering disciplineFree13mThe prompt failures you will meet, and what each one meansSix failures account for most of what goes wrong with a prompt in production. Each has a distinct cause and a distinct fix, and recognising which one you are looking at is faster than any amount of rewording.Agent Engineering·Memory and contextSign in14mWhat to drop when the context fillsEvery long run eventually has more history than budget. The order in which you discard things decides whether the agent stays coherent or quietly loses the thread, and the default of dropping the oldest messages is close to the worst option available.
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
LLM & GenAI FundamentalsHow do you evaluate long-context models (needle-in-a-haystack and beyond)?→RAG & Agent System DesignContext windows are now huge. When do you just stuff everything in context instead of building RAG?→LLM & GenAI FundamentalsWhat is an attention sink, and how does StreamingLLM use it for endless generation?→System Design for AI in ProductionDesign a text-to-SQL feature: let users ask questions in natural language over a real database.→ML Infrastructure & GPUsHow do you quantize or compress the KV cache, and why does it matter for long-context serving?→System Design for AI in ProductionDesign an AI meeting summarizer that handles thousands of meetings a day.→
COMPANIES THAT ASSUME THIS