ElevenLabs Agents Explained: What the Platform Actually Is

ElevenLabs Agents is the company's platform for building AI agents that can hold real conversations over voice or chat. It's separate from the text-to-speech tools ElevenLabs is best known for. The product was previously called Conversational AI, while the documentation now refers to it as ElevenAgents.

That name change can be confusing. Buyers may land on a text-to-speech pricing page, see character-based credits, and assume agents are priced the same way. They aren't. Agents are billed by conversation minute, with language model and phone line costs charged separately.

This page offers a plain-English guide to the product: what an agent is, how it handles each turn in a conversation, how its billing differs from the creative platform, and what you can, and can't, connect one to today.

For the text-to-speech side, read our guide to the text-to-speech platform explained. For a broader look at the vendor, see the business overview guide.

What an ElevenLabs Agent Is

An ElevenLabs Agent is a hosted AI assistant that listens to a person, decides what to say, and speaks back in near real time. You build it by filling in settings, not by writing a server.

The settings are the product. You give the agent a system prompt, a voice, a language model, and optional knowledge and tools. The vendor runs the speech pipeline behind it.

One agent can serve several front doors. The same configuration answers a phone number, a chat widget on your site, an iOS or Android app, or a raw WebSocket connection from your own code.

That is the useful mental model: an agent is a configuration plus a set of connected surfaces. Everything else on this page is about what you can put into that configuration.

• A system prompt that sets the job, tone, and rules.
• A voice and a language model, both swappable.
• A knowledge base for facts the model does not know.
• Tools that let the agent do things, not just talk.
• Channels: phone, web widget, mobile SDK, or WebSocket.

How the STT, LLM, and TTS Loop Works

Every turn runs through four parts in order: speech-to-text, a language model, text-to-speech, and a turn-taking model that decides when the agent should speak. The vendor documents these as the four coordinated pieces of the platform.

Speech-to-text (STT) turns the caller's audio into words. The language model (LLM) reads those words plus your prompt and any retrieved knowledge, then writes a reply. Text-to-speech (TTS) renders that reply as audio.

The turn-taking model sits across all of it. It reads conversational cues, including filler sounds like um and ah, to judge whether the caller has finished or is just thinking.

Here is the part most write-ups skip. Each stage adds delay, and the stage you control most is the language model. Swapping a large frontier model for a smaller one usually cuts more lag than any voice setting will.

The second non-obvious point: a fast pipeline with poor turn-taking still feels wrong. An agent that answers in 400 milliseconds but cuts people off mid-sentence gets rated worse than a slower one that waits properly.

• STT: a fine-tuned speech recognition model transcribes the caller.
• LLM: models from several providers, or your own endpoint.
• TTS: the low-latency voice models the vendor is known for.
• Turn-taking: a proprietary model that decides when to speak.

Where Speech-to-Speech Models Differ

A speech-to-speech model takes audio in and gives audio out, with no text step in the middle. That is a different design from the four-stage loop above, which the industry calls a cascaded pipeline.

Skipping the text step removes handoffs, so it can feel quicker. It also carries tone, laughter, and hesitation through to the reply, because none of that is thrown away in a transcript.

The cost is control and inspection. With a cascaded pipeline you can read exactly what the model heard and exactly what it planned to say, then log it, test it, and block it. Audio-native models make that harder.

Watch the name collision here. ElevenLabs also sells a Speech to Speech voice changer, which converts one recorded voice into another. That is an audio editing tool, not a conversational agent, and the two are billed differently.

For a side-by-side on the audio-native approach, see our GPT live comparison.

Agents vs the Creative Platform: Minutes vs Characters

The Agents platform is billed in conversation minutes, while the creative text-to-speech products are billed in characters and credits. That single difference changes how you forecast cost.

With text-to-speech you can count characters in a script before you generate anything. With an agent you cannot. The length of the call is set by the caller, not by you.

The vendor publishes a per-minute overage rate of $0.080 across its paid Agents tiers, and states that the language model and any telephony are billed separately on top. Text messages are listed at $0.003 each. Prices change without notice, so confirm on the vendor page before you budget.

Minutes track connected conversation time. A caller who reads out a long account number slowly costs the same as one who gets a useful answer, so a chatty agent is not the only way to burn a bundle.

Plan tiers, bundled minutes, and concurrency ceilings live on our pricing guide and the limits guide. To model an all-in per-call figure, use the voice agent cost calculator.

• Creative platform: characters and credits, forecastable up front.
• Agents platform: conversation minutes, set by the caller.
• LLM tokens and telephony are extra line items, not included.
• Chat-only messages are priced per message, not per minute.

What a Business Can Point an Agent At

An agent can handle any job where the answer lives in a document you can upload or a system you can call over an API. That is the honest boundary, and it is wider than most people expect.

Knowledge comes from a knowledge base built from files, URLs, or pasted text. Larger sources are retrieved by RAG, which means only the passages relevant to the caller's question get pulled into the prompt.

Actions come from tools. The platform ships system tools for ending a call, detecting language, transferring to another agent, transferring to a phone number, skipping a turn, playing keypad tones, and detecting voicemail. Your own tools call your APIs.

The vendor's own product page names integrations including Zapier, Salesforce, HubSpot, Zendesk, Shopify, Stripe, Genesys, Twilio, and Amazon Connect. It also lists jobs such as appointment scheduling, order tracking, ticket triage, and lead qualification.

Which of those a business can actually run unattended is a separate question. We answer it job by job in the use-case readiness guide, and cover the wiring in the integrations guide.

• Knowledge base from files, URLs, or text, retrieved with RAG.
• System tools for transfers, keypad tones, and voicemail.
• Custom tools that call your own APIs during the call.
• Batch calling for outbound lists, with per-row variables.

What an Agent Cannot Do on Its Own

An agent cannot touch any system you have not wired to it, and it cannot know any fact you have not given it. Everything else it says is the language model guessing.

That is the root of most bad deployments. A team uploads a marketing PDF, skips the tools, then wonders why the agent invents an order status. It had nothing real to read.

Regulated work needs paperwork, not just prompts. The vendor supports HIPAA configurations with Zero Retention Mode and a signed business associate agreement, and states that agreement is available on Enterprise plans only.

Concurrency is a hard ceiling too. Calls above your plan's simultaneous-call limit are handled differently, which matters the moment a marketing email goes out. We cover those ceilings in the limits guide, and real language coverage in the languages guide.

Disclosure is the last one. Tell callers they are speaking with an AI, and keep a fast route to a person for anything sensitive.

• No tool wired means no action taken, only talk.
• No knowledge source means confident guessing.
• HIPAA work needs Zero Retention Mode and an Enterprise agreement.
• Simultaneous calls are capped by plan tier.
• Sensitive topics still need a human path.

How the Agents Platform Fits the Wider Voice-Agent Market

ElevenLabs enters the voice-agent market from the audio side, while most rivals entered from the orchestration side. That history still shows in the products.

Vapi, Retell AI, Bland, and Synthflow were built as call-orchestration layers first. They mix and match voice vendors, and several of them can use ElevenLabs voices underneath.

The trade is control against convenience. An orchestration platform gives you more choice of voice and model, while a single-vendor stack gives you fewer moving parts and one bill for the speech side.

There is also the build-it-yourself route, using open toolkits to assemble the same four stages. At Layer3Labs, we map that in the self-hosted voice agent stack, and compare model choices in best LLM for voice agents.

For head-to-head reading, see the head-to-head against Vapi and the alternatives roundup.

• Audio-first vendor, strongest on voice quality and turn-taking.
• Orchestration-first rivals offer more vendor swapping.
• Open toolkits trade setup effort for full control.
• Several rivals resell the same voices you would get direct.

Where to Start If You Are Evaluating It

Start with one narrow job, one knowledge source, and a test number nobody has published. That is the cheapest way to learn whether the platform fits your calls.

Build the agent in the console, then run simulated conversations before a real caller ever hears it. The vendor ships simulation, next-reply, and tool-call testing for exactly this.

Our step-by-step build walkthrough is the build walkthrough. If you want the vendor-neutral version of the same decision, read AI voice agents for small business.

For a verdict on how the platform performs in practice, see our our platform review.