How to Build AI Voice Agents in 2026: Vapi vs Retell AI vs Amazon Bedrock Nova Sonic vs ElevenLabs (with Real Case Studies and Deployment Guides)

The voice-agent market in 2026 looks nothing like it did eighteen months ago. Sub-500ms turn latency is table stakes, three vendors have credible speech-to-speech models in general availability, and a wave of "voice agent as a service" platforms — Vapi, Retell AI, ElevenLabs Conversational AI, LiveKit Agents — have collapsed weeks of orchestration work into a config file. If you're picking a stack this quarter, the interesting question is no longer "can it work?" — it's "which of these should I actually pay for, and where does it break?"

This guide is the field notes we wish we'd had. We'll walk through every serious platform in the voice-agent space, the trade-offs that only show up after you go live, the pricing that actually applies at 10k concurrent calls, real deployments from enterprises like Bank of America, DoorDash, Klarna, JPMorgan Chase and Air India, and a deployment cookbook per platform. If you want the underlying theory of how voice agents work — cascaded vs speech-to-speech, endpointing, barge-in, latency budgets — start with our companion post Building Voice Agents: Everything You Know About Chat Agents Still Applies. Everything here builds on that.

The five architectures you'll pick between

Before the vendors, the shape. Every 2026 voice agent falls into one of five buckets, and every platform below is really an opinionated packaging of one of them:

1. 1Cascaded, self-hosted (LiveKit Agents, Pipecat, custom). You wire STT + LLM + TTS + VAD + telephony yourself. Maximum control, maximum work.
2. 2Cascaded, managed (Vapi, Retell AI). Same STT+LLM+TTS pipeline, but the vendor handles orchestration, telephony, endpointing, barge-in and observability. You bring the prompt and tools.
3. 3Speech-to-speech (Amazon Nova Sonic, OpenAI Realtime, Google Gemini Live). One multimodal model handles audio in, reasoning, tool calls and audio out. No text checkpoint. Lowest latency, hardest to debug.
4. 4Conversational bundles (ElevenLabs Conversational AI, Deepgram Voice Agent API). A best-of-breed vendor (TTS or STT) wraps the whole loop so their strong component sets the tone.
5. 5Telco-first (Twilio ConversationRelay, Amazon Connect + Lex). The contact-center vendor exposes the LLM plug-in point and owns the phone number, routing and compliance.

Vapi — the developer-first cascaded platform

Vapi (YC W23) is arguably the most-adopted "batteries-included" voice agent platform among startups. It exposes a single REST/WebSocket API where you configure a model (OpenAI, Anthropic, Groq, your own), a voice (ElevenLabs, PlayHT, Cartesia, Deepgram Aura, Rime), a transcriber (Deepgram, Assembly, Whisper), tools/function calls, and a phone number — and Vapi handles the WebRTC/SIP media, VAD, endpointing, barge-in and streaming glue. Time from zero to a phone number that rings your agent: about 15 minutes.

What Vapi is genuinely good at

• Model & voice freedom. Swap the LLM or TTS in one JSON field. No lock-in to a specific model family.
• Sub-800ms end-to-end when paired with Cartesia Sonic or Deepgram Aura on the TTS side and Groq/Cerebras for the LLM.
• Tools and "squads" — multi-agent handoff within a single call is a first-class primitive.
• BYO telephony. Bring your own Twilio/Telnyx SIP trunk, or use Vapi's numbers.
• Server-side webhooks for every event (call started, transcript chunk, tool call, ended, analysis) — the observability story is strong.

Where it hurts

• Cost stacks. You pay Vapi's per-minute orchestration fee (~$0.05/min at time of writing) on top of your LLM, STT and TTS. High-volume phone deployments quickly exceed $0.20–0.35/minute all-in.
• Debugging cascades is still cascades. When the agent mishears "fifteen" as "fifty," it's your STT choice, not Vapi's fault — but you own the fix.
• Cold-start jitter on rarely-used voices; pin a warm voice for latency-sensitive flows.

curl -X POST https://api.vapi.ai/assistant \
  -H "Authorization: Bearer $VAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Support Triage",
    "firstMessage": "Hi, this is Aria from support. What can I help you with?",
    "model": {
      "provider": "openai",
      "model": "gpt-4.1-mini",
      "temperature": 0.4,
      "systemPrompt": "You are a warm, concise support agent. One question at a time. No markdown."
    },
    "voice": { "provider": "cartesia", "voiceId": "sonic-english" },
    "transcriber": { "provider": "deepgram", "model": "nova-3" },
    "serverUrl": "https://api.mycompany.com/vapi/events"
  }'

Retell AI — the enterprise cousin

Retell AI (also YC) targets the same cascaded-managed niche as Vapi but leans harder into the contact-center use case. Its differentiators in 2026 are its Retell LLM (a proprietary conversational model tuned specifically for turn-taking and interruption handling), a warm transfer primitive (agent → human without a redial), and native batch-outbound campaign tooling. If your voice agent is really a phone-heavy operations tool — appointment reminders, collections, insurance verifications — Retell was built for you.

Where Retell wins

• Turn-taking quality. The Retell LLM is fine-tuned to not interrupt mid-thought and to handle "uh, actually…" corrections gracefully.
• Warm transfer to a human is a single flag — genuinely rare among cloud voice platforms.
• Batch dial campaigns with concurrency caps, retry logic and per-lead outcomes exposed as webhooks.
• Compliance posture — HIPAA BAAs, SOC 2 Type II, PCI redaction filters on transcripts.

Vapi vs Retell — how to pick

• Vapi is more model-agnostic. Retell nudges you toward its own LLM; Vapi treats every LLM as swappable config.
• Retell is stronger on outbound. Vapi does outbound too, but Retell's campaign UI + reporting is a level above.
• Latency parity. Both land ~700–900ms end-to-end with good component choices. Neither will beat a true speech-to-speech model.
• Pricing. Retell is ~$0.07–0.31/min all-in depending on model; Vapi is similar range with more per-component granularity.

Amazon Bedrock Nova Sonic — the speech-to-speech contender

Amazon Nova Sonic (GA on Bedrock since 2025) is Amazon's unified speech-to-speech model. Unlike cascaded stacks, Nova Sonic takes audio in and emits audio out over a single bidirectional stream — it hears tone and emits matched prosody, and never surfaces a text intermediate. The trade is exactly what you'd expect: sub-300ms feel and genuinely lifelike voice, in exchange for a black box in the middle and Bedrock lock-in.

Nova Sonic supports tool use inside the stream (yes — you can call Lambdas mid-conversation), integrates with Bedrock Knowledge Bases for RAG, and inherits Bedrock's IAM, VPC endpoints, KMS-encrypted logs, and Guardrails. For AWS-native shops it slots in cleanly alongside AgentCore, Step Functions and Connect.

Deploying Nova Sonic — reference architecture

1. 1Media layer: Amazon Connect (for PSTN) or LiveKit/WebRTC (for app/web). Both can bridge 16kHz PCM into Bedrock's bidirectional stream.
2. 2Bedrock stream: invoke_model_with_bidirectional_stream on amazon.nova-sonic-v1:0. One long-lived stream per call.
3. 3Tools: Register your tool schema in the session; handle toolUse events by invoking a Lambda and sending back toolResult.
4. 4RAG: Attach a Bedrock Knowledge Base to enrich system prompt context per call, or use a retrieve tool the model calls on demand.
5. 5Guardrails: Enable Bedrock Guardrails on the session for PII redaction, denied topics and prompt-injection filtering.
6. 6Observability: Stream event metadata (not audio) to CloudWatch + OpenSearch for cost and latency dashboards.

ElevenLabs Conversational AI — voice-first, model-flexible

ElevenLabs got here by having the best-sounding TTS voices on the market. Their Conversational AI platform wraps that voice quality with an integrated STT (Scribe), a turn-detection model, and pluggable LLMs (their own Flash model plus GPT/Claude/Gemini). If "our brand voice must sound like a real, warm human" is a hard requirement — media, hospitality, retail — this is the one to try first.

Distinctive features

• Voice cloning + IVC — clone a specific speaker (with consent), including your CEO for internal comms or your on-brand talent.
• Multi-lingual out of the box in 30+ languages with the same voice.
• WebRTC React SDK (@elevenlabs/react / useConversation) that handles mic, playback, VAD, barge-in and events in a hook.
• Client tools — the agent can call JavaScript functions in your frontend directly (navigate, show a modal, submit a form), not just server-side APIs.
• Data residency options — US, EU, India, global.

Also on the table (and when to pick them)

LiveKit Agents

The LiveKit Agents framework (Python + Node) is what you reach for when you want cascaded-managed primitives but without a hosted vendor tax. LiveKit runs the WebRTC media plane; you wire in STT/LLM/TTS plugins (all major vendors supported), and their VoicePipelineAgent handles VAD, endpointing, streaming and barge-in. Self-host or use LiveKit Cloud. This is what powered OpenAI's ChatGPT Voice for years and is a very common building block underneath other products.

Pipecat

Pipecat (Daily.co, open-source) is a Python framework for real-time multimodal pipelines. Similar niche to LiveKit Agents but framework-first, with excellent support for Deepgram, ElevenLabs, Cartesia and OpenAI Realtime. Great for research, custom flows, and teams that want the pipeline as code they can rewire.

Deepgram Voice Agent API

Deepgram — long the leader in low-latency STT — now offers an end-to-end Voice Agent API that packages Nova-3 (STT) + their orchestrator + Aura-2 (TTS) with your LLM of choice. If Deepgram is already your STT, this is the shortest jump to a full agent.

OpenAI Realtime API

OpenAI's Realtime API (GPT-4o Realtime family and Realtime mini) is the other credible speech-to-speech option alongside Nova Sonic. Similar bidirectional WebSocket / WebRTC shape, expressive prosody, tool calls in-stream. Latency is excellent; pricing is per-input-audio-minute and per-output-audio-minute. Use it when you're OpenAI-native and want the fastest possible feel.

Google Gemini Live

Gemini Live API covers the same speech-to-speech territory with tight Google Cloud integration and multimodal (video-in) support. Good when you're on GCP or need vision + voice in one session.

Twilio ConversationRelay

Twilio ConversationRelay exposes a WebSocket where you receive user transcripts and stream text back — Twilio handles the phone, VAD and TTS. Best when telephony compliance, phone numbers and global reach are the hard part and you just want to plug your LLM in.

Side-by-side comparison (2026)

All-in pricing assumes typical component choices, US region, ~1M minutes/month. Latency numbers are median end-to-end (user stops speaking → first audio out) measured with a warm session and good component choices. All figures are illustrative — verify with each vendor before contracting.

• Vapi — Cascaded managed · ~700–900ms · $0.15–0.35/min all-in · Best for: fast startup builds, any-model freedom, tool-heavy flows.
• Retell AI — Cascaded managed · ~700–900ms · $0.07–0.31/min · Best for: outbound campaigns, contact-center replacement, warm transfer.
• Amazon Nova Sonic (Bedrock) — Speech-to-speech · ~300–500ms · ~$0.30–0.60/min (audio in + audio out + tools) · Best for: AWS-native shops, IVR replacement, natural prosody.
• ElevenLabs Conversational AI — Cascaded managed, voice-first · ~600–800ms · $0.10–0.30/min + voice fees · Best for: brand-critical voice, multilingual, in-browser widgets.
• LiveKit Agents — Cascaded framework · ~600–900ms · component pass-through + infra · Best for: teams that need control and are OK owning the pipeline.
• Pipecat — Cascaded framework, OSS · varies · component pass-through · Best for: research, custom pipelines, self-hosted experiments.
• Deepgram Voice Agent API — Cascaded managed · ~500–800ms · $0.08–0.20/min + LLM · Best for: Deepgram-STT shops, latency-sensitive flows.
• OpenAI Realtime — Speech-to-speech · ~300–500ms · usage-based per audio minute (mini is materially cheaper) · Best for: OpenAI-native, consumer chat, fastest feel.
• Google Gemini Live — Speech-to-speech · ~400–600ms · usage-based · Best for: GCP, vision + voice combined.
• Twilio ConversationRelay — Telco-first · ~800ms + your LLM · Twilio media rates + LLM · Best for: global PSTN, compliance-heavy telephony.

Real case studies — who's actually running voice agents

Bank of America — Erica (2 billion+ interactions)

Bank of America's Erica virtual assistant surpassed 2 billion interactions with 42M+ users, and in 2024–2025 rolled out generative-AI voice enhancements to handle nuanced spoken queries about transactions, disputes and card controls. Erica is a bespoke stack (not an off-the-shelf platform), but the pattern — narrow-scope banking agent, tight guardrails, human handoff on any friction — is the enterprise playbook every other bank now copies.

DoorDash — self-serve support voice agent

DoorDash publicly discussed (Google Cloud Next 2024/2025) a self-serve voice-agent that resolves a large share of Dasher support calls without a human, built on Gemini + Vertex AI. Key design choices: strict scope, live sentiment monitoring, and instant escalation to a human when the caller shows frustration. It's the canonical case for "cascaded, with an aggressive human-handoff policy."

Klarna — the AI assistant that replaced ~700 agent-equivalents

Klarna's AI assistant (chat-first but voice-adjacent, built on OpenAI) handled the workload of ~700 full-time agents within one month of launch in early 2024, resolving 2/3 of customer service chats. It's one of the most cited datapoints in the industry for LLM-driven customer-service ROI — and a cautionary tale, since Klarna publicly reversed some of that automation in 2025 after quality regressions on complex cases. The lesson: measure containment and CSAT, not just deflection.

JPMorgan Chase — LLM Suite and voice-enabled workflows

JPMorgan has publicly discussed its LLM Suite (rolled out to 200k+ employees in 2024–2025), including voice-enabled research assistants for advisors. It's not a customer-facing agent, but it's the largest documented deployment of voice-enabled LLM tooling inside a bank. Their disclosures on prompt logging, red-teaming and access control are a useful template.

Air India — agentic voice IVR on Azure

Air India publicly discussed at Microsoft events its migration of the customer-service IVR to an agentic voice assistant on Azure OpenAI + Azure AI Speech, handling flight status, rebookings and refunds across multiple languages. Reports have cited it handles the majority of contact-center calls end-to-end — a good example of the cascaded, telco-first pattern using Azure Communication Services for the media plane.

ElevenLabs enterprise deployments

ElevenLabs publishes case studies with Sonos, H&M, Deutsche Telekom, LiveKit and others — mostly around brand-voice IVR and multilingual support. Common pattern: ElevenLabs voice + a hosted LLM + the useConversation React SDK embedded in the brand's app.

Startups on Vapi and Retell

Vapi and Retell case study pages list dozens of production customers — from Bland-style outbound sales (real-estate, insurance, home services), to restaurant reservation agents, to medical intake and appointment platforms. The recurring theme in every honest case study: the wins are in narrow, high-volume, structured flows — "book a table," "confirm an insurance claim," "pre-qualify a lead." Open-ended flagship deployments ("our voice agent handles anything") universally underperform.

Deployment guide by platform

Deploying on Vapi (fastest path to a phone number)

1. 1Create a Vapi account, get your API key.
2. 2POST /assistant with your model, voice, transcriber, firstMessage, systemPrompt, tools.
3. 3Buy a Vapi phone number or attach a Twilio/Telnyx SIP trunk.
4. 4Attach the assistant to the number in the dashboard or via PATCH /phone-number.
5. 5Set serverUrl on the assistant to your HTTPS endpoint — you'll receive status-update, function-call, end-of-call-report events.
6. 6Log transcripts + latencies. Iterate on the prompt by listening to the recordings.

Deploying on Retell AI

1. 1Create a Retell agent via dashboard or API — pick the Retell LLM or a plugged-in OpenAI/Anthropic model.
2. 2Define your prompt, tools (with JSON schema), and optional knowledge base.
3. 3For inbound: import a phone number and route it to the agent.
4. 4For outbound campaigns: use the batch dial UI or POST /create-batch-call with a CSV of leads.
5. 5Set the webhook URL to receive call_started, call_ended, call_analyzed events + full transcripts.
6. 6Enable warm transfer with a target SIP endpoint / phone number for escalations.

Deploying Nova Sonic on AWS (production reference)

1. 1Enable model access to amazon.nova-sonic-v1:0 in the Bedrock console for your region.
2. 2Provision a media entry point: Amazon Connect (PSTN) or LiveKit/Chime SDK (WebRTC).
3. 3Write a small orchestrator (Lambda + Fargate, or ECS) that opens invoke_model_with_bidirectional_stream per call.
4. 4Bridge audio: pipe 16kHz PCM from the media source into the Bedrock stream; play audioOutput events back.
5. 5Register tools in the session; handle toolUse events by calling Lambdas and returning toolResult.
6. 6Attach a Bedrock Knowledge Base for RAG; enable Bedrock Guardrails for PII + denied topics.
7. 7Stream event metadata to CloudWatch; dashboards for TTFB, tool latencies and cost per call.

Deploying ElevenLabs Conversational AI in a web app

1. 1Create an agent in the ElevenLabs Conversational AI dashboard — pick your voice, LLM, first message, prompt.
2. 2Configure tools (server-side and/or client-side) in the agent settings.
3. 3In your React frontend, npm i @elevenlabs/react and use useConversation.
4. 4For authenticated agents, add a server endpoint that mints a conversation token via POST /v1/convai/conversation/token with your xi-api-key.
5. 5Call conversation.startSession({ conversationToken, connectionType: 'webrtc' }) after getUserMedia({ audio: true }).
6. 6Handle onMessage for transcripts + agent replies, onError for graceful UX.

import { useConversation } from "@elevenlabs/react";

export function VoiceWidget() {
  const convo = useConversation({
    onMessage: (m) => console.log(m),
    onError: (e) => console.error(e),
  });

  async function start() {
    await navigator.mediaDevices.getUserMedia({ audio: true });
    const { token } = await fetch("/api/eleven/token").then(r => r.json());
    await convo.startSession({ conversationToken: token, connectionType: "webrtc" });
  }

  return (
    <button onClick={convo.status === "connected" ? convo.endSession : start}>
      {convo.status === "connected" ? "End" : "Talk"}
    </button>
  );
}

The hard-won operational lessons

• Latency is a full-stack concern. Fast model + slow TTS = slow agent. Measure TTFB per stage, not just end-to-end.
• Stream everything.* Partial STT into the LLM, first-sentence LLM output into TTS. Do not batch.
• Endpointing is worth an engineer-week. Bad endpointing produces every complaint your users will ever write.
• Barge-in is non-negotiable. If the user starts talking, your agent stops. Instantly.
• Prompt for the ear. No markdown, ≤3 sentences per turn, one question at a time, spell out or skip long strings.
• Confirm the risky bits. Names, numbers, actions. "Just to confirm — you said fifteen, one-five?"
• Design the human handoff on day one. "Let me get a specialist" beats "I don't understand" every time.
• Log full turns. Transcript + audio + per-stage latency + barge-in points. "It felt slow" is unfixable without this.
• Cost meters differently. Voice is per-minute and per-concurrent-call, not per-token. Model your infra bill accordingly.
• Compliance is real. Call recording, biometric voice data, PII in transcripts — talk to legal before you go live.

Learn by building — voice agents in AgentSwarms

The cheapest way to build intuition for any of the platforms above is to build the same thing on a stack where you can see every knob. AgentSwarms runs the cascaded loop on Lovable AI's STT and TTS models so you can prototype the whole flow in the browser, then port the learnings into Vapi, Retell, Nova Sonic or ElevenLabs.

• Build a voice agent in the [Agent Builder](/agents). Click New Voice Agent — you get the standard builder plus a Voice tab where you pick the STT model (openai/gpt-4o-mini-transcribe), the TTS model (openai/gpt-4o-mini-tts), the voice, and a greeting. Every tool, RAG source, guardrail and memory your chat agent has works unchanged.
• Talk to it in the [Voice Playground](/voice-playground). Hold-to-talk or auto turn-detect, listen to replies, tune the prompt live. Ship-ready sample agents (support triage, B2B discovery, language tutor, standup coach) are one-click forkable.
• Read the [Voice Agents deep-dive lesson](/learn#dd-voice-agents) for the architecture theory in full — cascaded vs speech-to-speech, endpointing, barge-in, latency budgets.
• Consult the [voice docs](/docs/voice) for the API shape and integration cookbook.
• Cross-reference [Building Voice Agents: chat agents with ears and a mouth](/blog/building-voice-agents-tts-stt-vs-speech-models-nova-sonic) for the underlying mental model this whole post assumes.

Prototype in AgentSwarms, port to your production platform of choice. The vendor-specific bits are surprisingly thin once your agent logic, tools and prompt are right — which is exactly the point of this post's opening claim: a voice agent is the agent you already have, wearing a microphone.