Working with Model APIs
Calling models reliably: streaming, structured outputs, tool calling, errors and fallbacks, model selection, gateways and routing, and prompt caching economics.
- Chapters
- 8
- Hours
- 1
- Difficulty
- Beginner to Intermediate
- 2.0beginner
Calling models
What happens when you POST a chat completion: messages, roles, the system prompt, and the three provider shapes you need to know side by side.
- 2.1beginner
Streaming
How token-by-token streaming wins on perceived latency, why it breaks output validation, and the four ways to get both.
- 2.2beginner
Structured outputs
Why schema-enforced JSON, not free text, is the contract between an LLM and the code that consumes its output, and how to wire it up with Pydantic.
- 2.3intermediate
Function and tool calling
The model proposes a tool call, you execute it, the model continues: how to define tool schemas, run the loop, parallelize, and return errors the model can act on.
- 2.4intermediate
Errors, retries, fallbacks
How to handle the timeouts, rate limits, and overloaded responses every LLM API throws at you, with the right backoff, fallback, and idempotency rules.
- 2.5beginner
Choosing models and reading benchmarks
Public leaderboards don't predict your product's quality. Shortlist by tier, test on your own eval set, decide. That's the whole workflow.
- 2.6intermediate
Gateways and routing
One API in front of many providers, cheap-then-expensive cascades, provider failover, and the rule for when a gateway earns its operational cost.
- 2.7beginner
Prompt caching
The single biggest cost lever in API-native LLM products. A 50k-token agent over 10 turns drops from $1.53 to $0.47 with one config change.