How do you usability-test a copilot or chat interface when its answers change every time?
Fix the tasks, control the answers for part of each session, log what every participant saw, and sort each finding into a design fix or a model fix.
Usability-test a copilot by fixing what you can and recording what you cannot. Give every participant the same tasks with written success criteria, run part of each session on a controlled set of answers so you can judge the interface on its own, run the rest on the live model, and log the exact prompt and answer each person saw. Then sort every problem into a design fix or a model fix before anyone changes anything. Humbleteam designs AI products, including copilots and chat interfaces, for clients such as Synthesia, the text-to-video platform, Cluely and MyRadar.
Humbleteam, a digital product design agency, publishes this guide. It covers why chat interfaces break ordinary usability tests, a session plan with controlled and live parts, what to log, how many people to test with, an invented worked example and a sorting table for findings. Sources are described as of November 15, 2026.
Why is an ordinary usability test not enough for a chat interface?
In a classic moderated test, you watch participants try to complete specific tasks and ask them to think aloud. Every participant sees the same screens, so when five people stumble at the same step, the step is the problem.
A chat interface breaks that logic. Two participants who type the same question can get two different answers. If one of them fails the task, you cannot tell whether the layout misled them or the answer did. A test that mixes the two produces findings that nobody can act on: designers redesign a screen that was fine, and the real model problem stays.
Google’s People + AI Guidebook adds a third kind of failure. Besides user errors and system errors, AI products produce context errors, where the system makes wrong assumptions about what the user wants. A good test plan has a place to record all three.
How should the session be structured?
Split each session into two parts with the same tasks.
- Controlled part. The copilot returns answers you prepared in advance, so every participant sees the same output for the same task. This tests the interface: whether people find the entry point, understand what the copilot can see, read the answer, check its sources and act on it.
- Live part. The same or similar tasks run on the real model. This tests whether the answers themselves hold up and how people react when an answer is wrong, vague or slow.
If the model is not ready, the controlled part can be a Wizard of Oz test. NN/g describes it as a method where a user interacts with a mock interface controlled, to some degree, by a person. Google’s guidebook also suggests letting people try a “Wizard of Oz” version to see how they feel about the results before investing in automation. For a copilot, the “wizard” pastes prepared answers or writes them live within agreed rules.
Keep think-aloud in both parts. In a thinking aloud test, participants use the system while continuously thinking out loud, which shows what they expected an answer to say before they read it.
What should each task include?
Free chat tells you little. Write tasks the way you would for any product test, with a real goal and a clear end. Each task needs four parts:
- The goal, in the participant’s words: “Find out which of your three suppliers raised prices last quarter.”
- The success criterion: what a correct finish looks like and what counts as a partial one.
- The prepared answer for the controlled part, including at least one task where the answer is deliberately wrong or incomplete.
- What to watch: did they check the source, edit the prompt, undo, or trust the answer without checking?
The deliberately wrong answer matters, because it shows whether people check before they trust. If nobody notices a wrong figure in the controlled part, the interface is not giving them enough to check it, and that is a design problem you can fix.
What should you log in every session?
Log enough to replay the session later and to separate the interface from the model:
- The exact prompt each participant typed, including edits and retries.
- The exact answer they received, with a note of whether it was prepared or live.
- The time from sending to the first visible output, and to the full answer. Anthropic’s latency guide treats time to first token as its own measure, separate from overall response time.
- Whether they opened sources, used regenerate, undo or copy, and whether they finished the task.
- What they said they expected before reading the answer.
How many participants do you need?
For finding interface problems, small rounds work. Jakob Nielsen’s well-known analysis found that testing with 5 users finds about 85% of usability problems, and recommends several small rounds with fixes in between over one large study. That reasoning fits the controlled part, where the answers do not change.
The live part needs a different view. Five sessions show how people react to model mistakes, but not how often the model makes them. Measure answer quality with an evaluation set run by the engineering team, not with a usability study. Anthropic’s documentation, for example, recommends defining success criteria and building evaluations that are specific and measurable. The usability test tells you what a wrong answer costs a user. The evaluation tells you how often it happens.
What does a test plan look like for one copilot task?
The example below is invented to show the shape. An illustrative procurement tool has a copilot in a side panel that answers questions about supplier contracts.
| Part | What happens | Success criterion | What to log |
|---|---|---|---|
| Task | “Find which supplier raised prices last quarter and by how much.” | Names the right supplier and the increase, and opens the contract to confirm | Prompt, answer, sources opened, time to finish |
| Controlled, correct answer | Prepared answer names the supplier, the increase and links the contract | Participant confirms the figure in the contract | Did they open the link; did they read the answer correctly |
| Controlled, wrong answer | Prepared answer gives a wrong percentage with a source link | Participant spots the mismatch when checking the source | Whether anyone noticed; what made them check or not |
| Live | Same task on the real model | Same criterion | The exact live answer; whether it was correct; reaction to errors |
| Debrief | “What do you think the copilot could see when it answered?” | Participant describes the scope correctly | Their words, quoted in notes |
How do you sort what you found?
Before fixing anything, put each problem in one of three columns. The findings below are invented to show the sorting.
| Finding | Type | Who fixes it |
|---|---|---|
| People could not find where to ask the copilot | Design | Product design |
| Nobody checked the source on the wrong answer | Design | Product design: make sources easier to see and compare |
| The live answer was wrong for 2 of 5 people | Model | Engineering: add the task to the evaluation set |
| The copilot assumed the wrong date range | Context | Both: show the assumption in the answer and let people change it |
| People waited without knowing if anything was happening | Design | Product design: show progress early |
Microsoft’s guidelines for human-AI interaction offer a useful check here. The first two are to make clear what the system can do, and how well it can do it. Many design findings from copilot tests trace back to one of those two.
If your copilot has already shipped and adoption is low, start with usage data before a new study. Our guide to auditing an AI copilot that users don’t adopt walks through that. If the open question is still where the copilot should sit in the product, our guide on side panel, inline or command bar placement covers that decision, and the test plan above can compare two placements directly.
What can you produce now?
- Five tasks with written success criteria and one deliberately wrong prepared answer.
- A session script with a controlled part and a live part.
- A logging sheet: prompt, answer, prepared or live, timing, sources opened, outcome.
- The three-column sorting table, ready for the debrief.
- A short list of tasks to hand to engineering for the evaluation set.
