Key points
- ElevenLabs published a guide to the 2026 voice AI stack, covering nine layers from speech recognition to monitoring, using its ElevenAgents platform as the example.
- ElevenAgents offers Custom mode (pick your own LLM, TTS and ASR settings) and Realtime mode, where ElevenLabs manages them; Realtime v1 Mini is the default.
- ElevenLabs reports Scribe v2 Realtime transcribes in about 150 ms and that an interruption signal stops the agent in under 200 ms.
- MCP is off by default per workspace; admins opt in, then set an approval mode for each server.
Why it matters for marketers: Teams evaluating voice agents get a checklist of layers, latency figures and controls to question vendors on before committing to a build or purchase.
ElevenLabs has published a guide that explains how a production voice agent is assembled in 2026. It splits the system into nine layers: speech recognition, turn taking, language model, orchestration, tools, knowledge retrieval, speech synthesis, telephony, and guardrails with monitoring. The company uses its ElevenAgents product to show how each layer works.
The guide stresses that the flow is not a straight line. According to ElevenLabs, speech can start playing before the language model has finished its reply, and tool calls and retrieval can run mid-turn. It also describes two setup options: a Custom mode where teams choose their own models, and a Realtime mode where ElevenLabs handles those choices.
For marketers and content teams, the useful part is the vocabulary and the numbers. ElevenLabs reports roughly 150 ms for Scribe v2 Realtime transcription and under 200 ms for stopping the agent when a caller interrupts. These are vendor-reported figures, so teams should test them on their own calls. The guide also notes that the choice of language model is both a latency and a cost decision, and that ElevenLabs passes third-party model costs through at the provider's published rate.
Control features matter for brand-facing deployments. Workflows let teams route a caller between subagents on different models and hand off to a human. MCP connections are disabled by default, and admins pick an approval mode per server.
When comparing voice agent vendors, ask how each handles turn taking, model fallback, tool approvals and post-launch analytics. This guide offers one vendor's answers to those questions.
Sources
This brief is our own summary. Read the full reporting at the source.