LLMOps glossary
Key terms from this course, in plain English.
- A/B test
- Comparing two versions on real users to measure which performs better.
- AI gateway
- A central component through which all model calls pass, handling keys, retries, routing, limits and logging.
- Canary release
- Releasing a new version to a small share of users before everyone.
- Drift
- Changes in inputs, outputs or model behaviour over time that degrade performance.
- Evaluation harness
- Code that runs test cases through an AI system and grades the outputs.
- Exponential back-off
- Retrying with progressively longer waits, usually with random jitter.
- Fallback
- An alternative model or response used when the primary one fails.
- Feature flag
- A switch to turn features or configurations on or off without redeploying.
- Guardrail
- A check or control around an AI system's inputs, outputs or actions.
- LLM judge
- A language model used with a rubric to grade outputs.
- LLMOps
- Practices for building, deploying, monitoring and maintaining LLM applications.
- MLOps
- Practices for reliably building, deploying and maintaining machine learning models.
- Offline evaluation
- Testing on a fixed evaluation set before release.
- Online evaluation
- Measuring quality on live traffic and user signals.
- p95 latency
- The response time that 95% of requests are faster than.
- PII redaction
- Removing or masking personally identifiable information.
- Regression
- A drop in quality compared with a previous version.
- Release gate
- An automated check that must pass before a change can be deployed.
- Rollback
- Returning to a previous known-good version.
- Shadow testing
- Running a new version on real traffic in parallel without showing its outputs.
- Streaming
- Sending a model's response token by token as it's generated.
- Time to first token
- How long a user waits before the first part of a response appears.
- Trace
- A record of every step of one request through an AI system.