What should an AI startup give a product design agency before the work starts?
Give the agency real model outputs, failures included, the top tasks, success criteria, speed and cost limits, hard limits, data rules and a model owner.
An AI startup should give its design agency real model outputs, including the bad ones, the list of tasks the product must handle, the success criteria the team already tests against, latency and cost limits, what the model must never do, the data and privacy rules, and one engineer who can answer model questions. Without these, designers design for an ideal model, and the screens break the first time the real one is slow, vague or wrong. Humbleteam designs AI products for clients including Synthesia, the text-to-video platform, Cluely, MyRadar on Apple Vision Pro and Ultralytics, the company behind the YOLO computer-vision models.
Humbleteam, a digital product design agency, publishes this guide. It covers the inputs an AI startup should hand over in the first week, a one-page brief template, an invented worked example and the mistakes that make a brief useless. Sources are described as of November 15, 2026.
Why is a brief for an AI product different from a normal design brief?
A normal brief describes users, goals, scope and constraints. All of that still applies. What changes is that the most important material in the product is not designed by anyone. The model writes the content, and it does not write the same thing twice.
Designers need to see that content before they draw a single screen. Microsoft’s guidelines for human-AI interaction start with two rules: make clear what the system can do, and make clear how well it can do it. A design team cannot follow either rule if nobody has told it what the system can do or how well.
What should the startup hand over in the first week?
Seven inputs cover most AI products.
- Real outputs. Twenty to fifty actual responses from the current model for real tasks, not cherry-picked demos. Include long ones, short ones, wrong ones and refusals.
- The task list. The jobs users come to the product to do, ranked by how often they happen and how much they matter.
- Success criteria and evals. Whatever the team already uses to judge answer quality. Anthropic’s documentation advises teams to define success criteria and build evaluations that are specific and measurable. If those exist, the designers should read them. If they do not, writing them is a model task, not a design task, but the brief should say so.
- Speed and cost. Typical time to the first visible output and to a full answer, and the cost per request if it shapes the product, for example a limit on retries or a credit system. Model providers track time to first token as a separate measure for streamed answers, and designers need both numbers to design the wait.
- Hard limits. What the model must never do, which topics it refuses and which actions always need a human to confirm.
- Data rules. What user data the model sees, what it stores, what users can delete and what may never leave a region or a customer’s account.
- A model owner. One engineer who answers model questions within a working day during the project.
Which failure cases do designers need to see?
The ones users will actually meet. Google’s People + AI Guidebook groups AI failures into user errors, system errors and a third type, context errors, where the system makes a wrong assumption about what the user wants. Give the agency at least one real example of each.
Add the cases that are specific to your model: answers that sound confident and are wrong, answers that stop halfway, answers in the wrong language or format, and slow answers on large inputs. Each one becomes a screen state the designers must plan for. A brief that hides the failures gets a design that cannot handle them.
How much should the brief say about how the model works?
Enough for designers to explain it to users, not enough to retrain it. The People + AI Guidebook notes that people are more likely to have unachievable expectations for products they assume have human-like capabilities, and that the product should communicate its limits. Designers can only write that explanation if they understand the limits themselves.
A one-paragraph description usually works: what the model is good at, what it is weak at, what it can see when it answers, and what parts of the product are fixed logic rather than generated output. Mark the difference clearly. A price, a balance or a legal notice should usually come from fixed logic, and the design should show that it is not a model guess.
What does a one-page brief look like?
Use this template. Each line should be filled in or marked as unknown, with a date for when it will be known.
| Section | What to write |
|---|---|
| Product and user | Who uses it, for which job, and how they do that job today |
| Top tasks | The 5 to 10 tasks the product must handle, ranked |
| Real outputs | Link to the folder of real responses, including failures |
| Success criteria | How the team judges a good answer today, and the eval results if any |
| Speed and cost | Time to first output, time to full answer, cost limits that affect the interface |
| Hard limits | What the model must never do; actions that always need confirmation |
| Data rules | What the model sees, stores and forgets; what users control |
| Fixed versus generated | Which parts of each screen are generated and which are fixed logic |
| Metric | The product metric the design should move, and its current value |
| Model owner | Name, role and response time for model questions |
What does a filled-in brief look like?
The example below is invented to show the shape. An illustrative seed-stage startup is building an assistant that drafts replies to customer support tickets.
| Section | Example entry |
|---|---|
| Product and user | Support agents at mid-size online stores; today they write replies by hand from a macro library |
| Top tasks | Draft a reply to a refund request; draft a reply to a delivery question; summarize a long ticket thread |
| Real outputs | 40 real drafts, 9 of them marked as wrong tone, wrong policy or incomplete |
| Success criteria | A draft is good if the agent sends it with light edits; the team reviews 100 drafts a week against this |
| Speed and cost | First words appear in about 2 seconds; full draft in about 8; no limit on regenerate |
| Hard limits | Never promise a refund amount; always ask the agent before sending |
| Data rules | The model sees the ticket thread and the store’s policy pages; it stores nothing after the session |
| Fixed versus generated | Order number, amounts and dates come from the order system, not the model |
| Metric | Median time to first reply, currently 4 hours |
| Model owner | The ML engineer, answers in the shared channel the same day |
Two lines in this example change the design directly. “Never promise a refund amount” means the draft must show amounts as fixed data from the order system, visibly different from generated text. “About 8 seconds” for a full draft means the interface needs a waiting state and probably streamed text, not a spinner.
What makes a brief useless?
- Only demo outputs. The designers design for the best case and the product meets the average case.
- No owner for model questions. Every question waits for a weekly call and the design stalls.
- Changing the model mid-project without telling the agency. Outputs change and the designed states no longer match.
- Mixing up a model problem and a design problem. A design cannot fix an answer that is often wrong. It can make that answer easy to check and correct.
How does the brief fit with choosing the agency?
The brief comes after you have chosen. If you are still comparing agencies, our comparison of AI-native design agencies for startup products lays out the options, and our guide on giving an AI design agency a task it cannot rehearse shows how to test them. The same test task can open the brief, so the agency starts from a problem it has already seen.
What can you produce now?
- A folder of 20 to 50 real model outputs, with the failures labelled.
- The one-page brief template above, filled in or marked unknown with dates.
- A short note on what the model is good at, weak at and able to see.
- A list of hard limits and actions that always need confirmation.
- The name of your model owner and the response time you can promise.
