How Models Behave
The model landscape, tokens and context windows, sampling, embeddings, reasoning models, the cost-quality-latency triangle, and just enough ML to make good system decisions.
- Chapters
- 7
- Hours
- 1
- Difficulty
- Beginner
- 1.0beginner
The model landscape
Place any model into a tier, and know what each tier is for: frontier, mid, fast, and local, plus open-weight vs hosted and chat vs reasoning vs multimodal.
- 1.1beginner
Tokens and context windows
Tokens are the unit every LLM bill, limit, and prompt is measured in. How to count them in Python before you send, and why non-Latin text costs more.
- 1.2beginner
Sampling and temperature
How temperature and top-p reshape a model's probability distribution, how to pick them per task, and why temperature 0 is not a determinism guarantee.
- 1.3beginner
Embeddings
What embeddings are, why cosine similarity and dot product agree on normalized vectors, and the four engineering patterns built on them: search, memory, routing, clustering.
- 1.4beginner
Reasoning models
What test-time compute is, the effort and budget knobs across providers, why you pay for hidden thinking tokens, and when reasoning earns its cost.
- 1.5intermediate
The cost-quality-latency triangle
The book's recurring mental model: cost, quality, and latency are three axes you can't max at once. Name the one you sacrifice, then learn the levers that cheat it.
- 1.6beginner
Just enough ML
The three ML ideas an AI engineer who never trains a model still needs: the data split, overfitting, and distribution shift, framed as eval decisions.