Build Realtime Voice Agents with GPT-Live 1 in AgentSwarms
Voice agents now have a second engine: GPT-Live 1, a realtime speech-to-speech model you can interrupt mid-sentence. Plus the latest Claude, Gemini and GPT models across agents and swarms.
Until now, AgentSwarms voice agents worked in turns: you spoke, we transcribed, a chat model answered, and text-to-speech read it back. That pipeline is great for tool-heavy agents, but it never felt like a conversation. Today voice agents get a realtime engine: GPT-Live 1.
What GPT-Live 1 changes
- Speech-to-speech in real time — audio streams directly between your browser and the model, so replies start almost immediately.
- Interruptions — talk over the agent and it stops and listens, like a person would.
- The agent opens the call — it greets you with your agent's opening line and asks a first question.
- Background reasoning — when a question needs careful thought, GPT-Live hands it to GPT-6 Astra behind the scenes and speaks the verified answer.
- Live captions and usage — both sides are transcribed on screen, with billed seconds shown during the call.
Classic engine vs GPT-Live engine
- Classic (speech-to-text → chat model → text-to-speech) — supports tools (web search, knowledge bases, SQL, calculator), guardrails and any chat model. Best for task-heavy agents.
- GPT-Live 1 — natural, low-latency, interruptible conversation using your system prompt and greeting. Tools and knowledge bases are not connected to the live engine yet.
Build a GPT-Live voice agent
- 1Open Agent Builder → New voice agent (or edit an existing one).
- 2In the Voice tab set Voice engine to GPT-Live 1 — realtime speech-to-speech.
- 3Pick a GPT-Live voice — for example Marin (default), Meridian (North American), Vesper (British) or Quartz (Australian).
- 4Write the persona in the system prompt and set the greeting — the agent uses it to open the call.
- 5Save, then open it in the Voice Playground and press Start live call. Allow the microphone when asked.
Want to try it first? Two new samples are in the Voice Playground for everyone: Marin — GPT-Live Hotel Concierge (demo data only — nothing is actually booked) and Meridian — GPT-Live Agentic AI Interview Coach.
Live voice is billed per second while the call is open, with a 15-second minimum at the start. Muted time still counts, so end the call when you're done. Background reasoning requests are billed as normal AI usage. You need to be signed in to start a call.
Also new: the latest models across AgentSwarms
The Agent Builder, swarm nodes, Playground, Prompt Compare and the model fallback dialog now include the latest models on the built-in gateway:
- OpenAI — GPT-6 Astra, GPT-6 Luna, GPT-5.6 Sol, GPT-5.6 Terra and GPT-5.6 Luna (tools supported).
- Google — Gemini 3.8 Flash and Gemini 3.1 Flash Lite alongside Gemini 3.1 Pro.
- Anthropic — Claude Opus 5.5, Claude Fable 5.1, Claude Opus 5, Claude Sonnet 5 and Claude Haiku 4.5. Claude agents chat and run in swarms. Update: built-in Claude now supports enabled tools through native Messages tool rounds, capped at five rounds. Replies currently appear when complete rather than token by token.
- TypeSafe — Jev for typed decisions (see our Jev post).
The new Jev Decision Triage template uses three families in one run: Jev decides, GPT-6 Luna writes, Claude Sonnet 5 reviews. Different families make different mistakes — that's useful in a reviewer step.
Further reading & references
Was this useful?
Comments
Loading comments…