Should a startup pay for a trial project before hiring a design agency?
A paid trial can settle a close call between two design agencies, but only if it is paid, bounded to one to two weeks, and scored against criteria written beforehand.
Two agencies make your final round. Both have strong portfolios, both sent sensible proposals, and both references were warm. You still cannot tell which team will work well with yours, so someone suggests a test: give each agency a small piece of real work first.
For two finalists and one remaining doubt, that is often a good idea, with three conditions. Pay for it. Keep it small enough to finish in one to two weeks. Write down how you will judge it before it starts. Run that way, a trial shows how a team asks questions, records assumptions, takes feedback and hands work over, none of which a pitch can show. Without those conditions, it mostly measures who had the most free time.
Humbleteam publishes this guide and is a design agency, so apply the same sheet to us. Proposals and reference calls come before the trial stage, and the guide to evaluating a design agency when every pitch looks good covers them. This guide starts when two finalists are left.
When does a paid trial help, and when does it mislead?
A trial answers one question: what is it like to work with this team on a real problem? It does not answer every question.
| Situation | Use a trial? | Why |
|---|---|---|
| Two finalists, close proposals | Yes | The open doubt is working style, which a document cannot show |
| Five agencies still on the list | Not yet | Each trial costs you money and founder time, and costs the agency a team’s week |
| The problem itself is unclear | Run paid discovery first | A trial on a vague problem rewards the team that guesses best |
| You expect a year or more together | Partly | Two weeks cannot show how a team behaves in month seven, so lean on references too |
| Nobody on your side can answer questions within a day | No | You would be scoring your own delays |
| The agency staffs the trial with its strongest people only | Only if the same people are named for the real work | Otherwise you are testing a team you will not get |
Hiring research gives a frame, with a caveat: it studies employees, not agencies. A classic review of 85 years of selection research (Schmidt and Hunter, 1998) found that a work sample test, a hands-on simulation of part of the job, paired with a general mental ability test was one of the three best combinations for predicting job performance. A 2022 reanalysis (Sackett and coauthors) lowered the mean validity of the top-ranked selection methods by .10 to .20 points and ranked structured interviews first. Read together, a trial is strong evidence and not proof. It works best next to a structured conversation and references.
Two limits apply in every case. A trial shows effort on one bounded problem, and it cannot show business results, because a design needs a release and time to produce them. And once you have paid a team and liked its work, saying no feels harder. Write your pass mark before the read-out.
Why pay, and how to pay fairly
Speculative work is work done for free in the hope of getting paid. AIGA, the largest professional membership organization for design in the US, lists three risks. Clients risk compromised quality, because little time goes into research, alternatives and prototype testing. Designers risk being taken advantage of. Both sides carry legal risk over intellectual property. AIGA does not forbid the practice. It asks both sides to understand those risks before they start.
Payment also changes behavior. A team paid for a diagnosis acts like the team you would hire. A team working for free acts like a bidder, and you see the pitch again. Fair terms are simple:
- Pay every finalist, and keep finalists to two or three. NCVO, the membership body for charities and voluntary organizations in England, suggests paying two shortlisted agencies for a day and telling agencies how many others are in the process. Do the same.
- Price it from the proposal. Multiply the named people, their days and the day rate each agency quoted for the real project. Do not ask for a discount because it is a test.
- Ask whether the fee is credited if you sign. One agency that sells paid discovery credits the fee against the build. Ask, but do not make it a condition.
- Put ownership of the output in writing, both for the case where you hire the agency and the case where you do not. Have a lawyer check the wording.
- Pay whatever the result. If the fee depends on a yes, you have rebuilt the free pitch.
How to scope a trial that is real but bounded
Founders who run paid work trials to hire employees say the goal is to simulate normal working conditions as closely as possible (Lenny Rachitsky’s 2024 interviews with six founders). The same holds for agencies. Give the team the access and the people the real project would have.
A good trial problem has four traits:
- Real: it comes from your roadmap, not from a case study.
- Bounded: one flow, one decision or one screen family, finished in one to two weeks of calendar time.
- Judgeable: you hold something to compare against, such as funnel numbers, support tickets or interview notes, so you can tell a good answer from a confident one.
- Self-contained: it does not wait on a decision you have not made.
An illustrative example: new admins in a B2B SaaS product stop at the “connect your data” step and ask support for help. The trial asks each agency to say what is going wrong, propose a change to that step and name what to test first. It does not ask for a rebrand, a pricing change or the whole onboarding flow.
Hand over what the real project would get on its first day: data with names removed, files, access, the people who can answer questions, and the constraints you already know. Name one decision owner. Agencies lose days when data arrives late or when three people give conflicting feedback. If that happens in the trial, it will happen in the project, so record it as your finding.
A one-page trial brief you can copy
| Field | What to write | Illustrative example |
|---|---|---|
| Problem | One sentence: who is stuck, where, and what it costs | New admins stop at the “connect your data” step and contact support |
| Decision this informs | The choice you will make after the read-out | Fix the step in the interface, or first change what the product accepts |
| What we provide | Data, files, access and people, each with a date | Funnel numbers for the step, 20 support tickets without names, the Figma file, an engineer on call |
| Constraints | Technical, legal, brand and timeline limits | Existing component library, no new data sources this quarter |
| Out of scope | What the trial will not cover | Rebrand, pricing, other onboarding steps |
| Named team | Names, roles and share of time | Lead designer and researcher, both named in the real proposal |
| Schedule | Kickoff, midpoint feedback, read-out | Day 1, day 5, day 9 |
| Fee | Amount, what it covers, invoice date, any credit | Fixed fee invoiced at kickoff, paid whatever the result |
| Ownership | Who owns the output if you hire them and if you do not | Startup owns the output in both cases, wording checked by a lawyer |
| Decision owner | One person, plus a reply time | Head of product, answers within one business day |
| Pass mark | What you will accept as a pass, written before kickoff | No “Not met” on the deal-breaker rows of the scoring sheet |
If you cannot write the Problem row in one sentence, you do not need a trial yet. You need discovery.
How to judge the output and the way the team works
Judge the output and the way of working separately. For the output, ask whether the work solves the problem inside your constraints. For the way of working, ask whether you want this team in your week. In our view, an average deliverable from a team that showed its reasoning is a better sign than a polished one from a team that hid its assumptions.
Run one feedback test. At the midpoint, send a written round with a mix: two points you are sure of and one you are not. Ask for a written reply to each, which can be accept, push back with a reason, or ask a question. A team that accepts every point and a team that defends every point have each told you something.
| Area | What to look for | Your evidence | Result |
|---|---|---|---|
| Questions asked | Asked for data, users and constraints in the first two days, and asked something you had not thought of | Write what you saw | Met / Partly / Not met |
| Assumptions written down | A list of what they assumed, which assumptions would change the design, and how they would check them | Write what you saw | Met / Partly / Not met |
| Problem framing | Restated the problem in their own words and said what the trial cannot settle | Write what you saw | Met / Partly / Not met |
| Output fit | Solves the stated problem inside your constraints, and your engineers can say what they would build | Write what you saw | Met / Partly / Not met |
| Response to feedback | Answered each point in writing and pushed back at least once with a reason | Write what you saw | Met / Partly / Not met |
| Same people (deal-breaker) | The named people did the work, and any change was announced | Write what you saw | Met / Partly / Not met |
| Handover (deal-breaker) | A colleague who missed the work can follow the decisions, files are organized, open questions have owners | Write what you saw | Met / Partly / Not met |
| Honest limits (deal-breaker) | Said what the work does not prove and what to test next, without being asked | Write what you saw | Met / Partly / Not met |
Fill the sheet in during the trial, not afterward. Do not add the rows into one total. A single number lets screens that look good offset a missing handover. A “Not met” on a deal-breaker row ends the evaluation unless the agency gives a reason you can verify.
What to use instead when a trial is the wrong tool
Match the option to your largest doubt.
| Option | Use it when | Typical shape | What you learn |
|---|---|---|---|
| Paid discovery | The problem is unclear, or the scope would change the price | A fixed question, fixed inputs and a written output, for example one to three weeks in one agency’s published terms | How they diagnose, plus a scope you can take elsewhere |
| Short design sprint | One product decision needs customer reactions | The GV sprint runs five days and ends with customer interviews on Friday | How they run a room and read users |
| Narrow audit | You want to see how they analyze before they design | An expert reviews one flow and rates the findings, as in this expert review description | The quality of their diagnosis and writing, not their craft |
| Paid workshop | You mainly doubt chemistry | A two-hour kickoff workshop, as in the NCVO guidance above | How they start, not how they deliver |
If the doubt is craft on a real problem, run the trial. If it is scope, buy discovery. If it is a product decision, use a sprint. If it is analysis, order an audit. If it is chemistry, book a workshop.
Whichever you pick, fill in the brief before you contact anyone, and decide the pass mark before the read-out. After the read-out, write a short decision note: the agency you chose, the strongest evidence, the open risk, and who owns each dependency.
