How to design turn taking and interruptions in voice AI

A voice assistant should make it clear when it is listening, speaking, waiting, or doing work, and what happens when the user interrupts. Turn taking is part of the product task, not simply a speech setting. A natural-sounding voice can still leave someone unsure whether a request was heard or whether an action continued after they spoke over it.

ElevenLabs documents configurable interruption behavior for its agents, including allowing interruption by default and restricting it around particular tool calls. This is a product example of an available control, not a Humbleteam project or a rule every voice product should copy.

Which speaking states should a user recognize?

Map the task before choosing an animation or audio cue. A user needs to distinguish at least listening, processing, speaking, paused, and unable to continue. The exact set depends on the product. A tool call that changes a record may need an additional state that says the operation is still running after the assistant stops talking.

Use concise spoken acknowledgments when they prevent uncertainty, and a visible state for users who can look at the screen. “I’m checking the order” can be appropriate when a lookup takes time; repeating filler during every short pause is distracting. For screenless use, test whether a tone or brief phrase communicates the state without being mistaken for a completed answer.

Keep the visible transcript aligned with what was actually heard and said. If the assistant is cut off halfway through an explanation, a transcript that shows the unspoken remainder as delivered can mislead the user. The product should distinguish generated text from speech already played where that difference matters.

When should the user be able to interrupt?

Ordinary conversation often needs barge-in: the user corrects a name, changes direction, or says they have enough information. An interruption should stop or duck speech quickly, acknowledge the new input, and retain the task context needed to proceed. It should not silently treat a background sound as approval for an action.

Some operations cannot safely stop mid-call. ElevenLabs’ documentation describes modes that suppress interruption during a tool call or through the following agent turn. The product team must decide when that restriction is justified and make the wait understandable. A user who says “stop” while an external operation runs needs an accurate outcome, not a comforting claim that nothing happened.

Separate stopping speech from canceling work. The assistant can stop reading a long answer while a lookup continues, or it can request cancellation of an action. Those are different commands with different possible results. If an operation may already have completed, check its status before offering a retry.

How should a pause differ from an abandoned task?

Silence may mean the user is thinking, searching for a number, or finished. Avoid treating every pause as a command to continue. ElevenLabs also documents a skip-turn tool that lets an agent wait after a request such as “Give me a second.” That illustrates one way a product can respect an explicit pause.

Define what the assistant keeps during that wait: the current intent, confirmed details, and unresolved questions. Give the user a clear way to resume. If a session expires, explain what was saved and what must be repeated. Do not promise continuity across devices or sessions unless the product actually preserves it.

In an illustrative support flow, a customer asks about an order and pauses to find the order number. The assistant should wait without inventing an answer. When the customer returns, it should connect the number to the same support task, while confirming any detail whose scope may have changed.

What should happen after an interruption?

Resume from the user’s current goal, not from the start of a memorized script. If the user corrects the destination address while the assistant is reading a delivery estimate, update the address-dependent part of the task and make any recalculation explicit. If the new utterance begins a different task, confirm that switch when abandoning the first task would lose work.

For tasks with external effects, show or speak a concise action recap before execution. The AI agent action-flow guide covers approval, partial completion, and recovery in detail. Voice adds the need to confirm that the recap was actually heard, especially after an interruption.

What should a turn-taking test matrix include?

Test on the devices and in the environments the product actually targets. Use task completion and user understanding as the measures. The following proposed matrix is illustrative:

After each scenario, ask the participant what the system is doing and what will happen next. Tune the speech timing and state cues against those answers. For a broader AI product experience, the test is whether the person can finish the original task after conversation inevitably becomes untidy.