What makes AI products different
AI products are right most of the time, not all of the time. That one fact changes how product managers specify, design, test and launch.
Learning objectives
- Contrast deterministic software with probabilistic AI products
- List the extra responsibilities of an AI PM
- Explain the quality–cost–latency trade-off
- Recognise common AI feature patterns
1.1 Why AI products are different
Traditional software is deterministic: the same input gives the same output, and a bug is reproducible. AI products are probabilistic: they're right most of the time, wrong some of the time, and the output can vary. That changes how you specify, test, design and launch.
| Traditional software | AI product | |
|---|---|---|
| Behaviour | Defined by code | Learned from data or steered by prompts |
| Correctness | Pass or fail | A distribution: accuracy, error types, edge cases |
| Specification | Requirements and acceptance tests | Examples, evaluation sets and quality thresholds |
| After launch | Stable until changed | Can drift as data, users or models change |
| Cost | Mostly fixed | Often per request (tokens, compute) |
1.2 The AI product manager's job
An AI PM does everything a product manager does — understand users, define the problem, prioritise, align the team, measure outcomes — plus:
- Decide whether AI is the right solution at all
- Define what "good enough" means, with examples and evaluation sets
- Design for errors: what happens when the AI is wrong?
- Balance quality, latency and cost per request
- Own the risks: fairness, privacy, safety and regulation, with the right experts
- Plan for continuous improvement after launch
1.3 The quality–cost–latency triangle
Bigger models are usually more capable but slower and more expensive per request. Techniques like retrieval, better prompts, caching or smaller specialised models can shift the balance. A PM must decide which corner matters most for each feature: a real-time autocomplete needs speed; an overnight contract review needs quality.
1.4 Types of AI features
| Pattern | Example |
|---|---|
| Generate | Draft an email, write a product description |
| Transform | Summarise, translate, rewrite in a different tone |
| Classify & extract | Route tickets, pull fields from invoices |
| Answer | Q&A over help articles or company documents (RAG) |
| Predict & recommend | Churn risk, next-best product |
| Act | Agents that complete multi-step tasks with tools |
Technical background for each pattern is in Path 3: RAG, AI Agents, Fine-tuning and LLMOps.
Take a feature you know (in your product or one you use). Describe how it would change if AI powered it: what could go wrong, how you'd specify 'good', and what each request would cost.
See a worked example
Example — help-centre search: Today: keyword search, predictable. With AI answers: can answer in natural language but may be wrong or invent policy. Spec: 200 real questions with approved answers; target ≥ 90% correct and grounded, 0 invented policies. Cost: about 1–3p per question at expected volume; cache common questions.
Key takeaways
- AI outputs are a distribution, not pass/fail.
- AI PMs define 'good enough' with examples and evals.
- Every feature trades quality, cost and latency.
- Generate, transform, classify, answer, predict, act.