ElevenLabs Eleven Realtime v1: Speech-to-Speech Voice AI
On October 20, ElevenLabs released Eleven Realtime, a speech-to-speech model family for real-time voice agents. v1 Mini is a lightweight conversation-optimized model, and v1 Max is a reasoning-capable model (the-decoder, October 20). Speech recognition, turn-taking, voice activity detection, and voice generation are integrated into one system. Both models support more than 90 languages and a voice library of more than 17,000 voices (analyticsindiamag, October 20).
One-line summary for developers: voice AI is shifting from the fragmented cascade (ASR → LLM → TTS) to a single speech-to-speech system. ElevenLabs claims Max is the "fastest and most accurate" combination on the Artificial Analysis leaderboard. Max's 98.8% placed only third in accuracy, a tension worth keeping in view. And one design note: since the speech-to-speech structure ships no transcripts at all, enterprise agent safety, compliance, and app-provenance tooling have to be designed separately.
The Two Models: Mini for Natural Conversation, Max for Reasoning Work
- v1 Mini: optimized for natural real-time conversation. Target uses: AI assistants, teachers, interactive characters.
- v1 Max: reasoning-capable. Target uses: customer support, lead qualification, multi-step tool use.
- Speech recognition and voice activity detection are integrated into the same system, optimized from ASR through generation for low-latency conversation.
Accuracy: Big Bench Audio Max 98.8%, Third of 43 Models
- Big Bench Audio: Max 98.8%, ranking 3rd among 43 evaluated models; Mini 75.9%.
- Ahead of Max: StepAudio 3 Realtime 99.7%, Qwen Audio 3.0 Realtime Plus 99.2%.
- Speech Agent Arena (blind comparison): Max 998 points, Mini 988 points.
The lead claim heard at release, "there is no model faster and more accurate than Max on the Artificial Analysis leaderboard", and the accuracy ranking of third place sit in tension. Reading the numbers means reading these two facts together.
Latency and Full-Duplex Behavior: First Audio Under One Second, Mini Beats Max
- Average first audio response: Max 0.57s, Mini 0.62s.
- Full-Duplex-Bench: Mini 95.8% ranks above Max 94.1%. The direction of the result differs from the accuracy benchmark, and no explanation for it was given.
The API: OpenAI-Compatible, Two-Line Migration
- Delivery: the ElevenAgents platform or an OpenAI-compatible Realtime API. WebRTC is not supported at launch; WebSocket is the connection method.
- Migration is a two-line change: swap the endpoint, swap the model name, and the OpenAI SDK works as-is (journaldunet, October 20).
- Pricing combines per-minute audio connection time with token usage. Max costs more than Mini. Specific unit prices are undisclosed (the-decoder, journaldunet).
// Example (conceptual)
endpoint: https://api.elevenlabs.io/v1/realtime
model: eleven-realtime-v1
// unchanged — OpenAI SDK client code and parameter shapes stay the same
Background: A Week of Voice AI Releases, and the OpenAI-Compatible Migration Design
The same day brought Mistral's Voxtral 3 release (#3376), and voice AI releases were dense throughout the third week of October. ElevenLabs has also been pushing the "OpenAI-compatibility first" pattern, complementing the platform with items like 'Architect' (#3115) and VoiceStudio (#3250). The migration design that lets existing OpenAI Realtime API stacks swap in Eleven Realtime by changing only the endpoint and the model name is what stands out here. The pattern is aimed at lowering the switching cost for voice-agent teams who already have OpenAI wired in.
Still unconfirmed: the 98.8% and 0.57s figures are Artificial Analysis independent measurements under ElevenLabs-provided conditions, but perceived production latency is unconfirmed. With token unit prices undisclosed, operating costs cannot be predicted. The structure ships no transcript either, so friction with enterprise compliance tooling is possible (techtimes, October 20).
Practical Takeaways: Signals for Voice Agent Builders
- Transcript-less structure is a design decision: speech-to-speech ships no transcripts, so audit logs, safety review, and provenance design need separate thinking.
- The "fastest and most accurate" claim does not mean unconditional number one: Max is third on accuracy. Build the habit of reading the leaderboard itself rather than marketing copy.
- Why Mini wins on full-duplex while Max wins on accuracy: a benchmark split inherent to the architecture. Minimum call latency requirements and improvement direction have to be measured in-house.
Sources
- Original issue #3379
- the-decoder: ElevenLabs' new realtime voice models (October 20)
- Analytics India Magazine: ElevenLabs launches Eleven Realtime v1 (October 20)
- Related issue #3376: Mistral Voxtral 3 (the same week's voice AI release)
- Related issue #3115: ElevenLabs 'Architect'
- Related issue #3250: VoiceStudio
FAQ
What is Eleven Realtime?
Eleven Realtime is a speech-to-speech model family from ElevenLabs for real-time voice agents, released on October 20. v1 Mini is a lightweight conversation-optimized model, and v1 Max is a reasoning-capable model. Speech recognition, turn-taking, voice activity detection, and voice generation are integrated into one system, and both models support more than 90 languages and a voice library of more than 17,000 voices.
How accurate is Eleven Realtime Max?
On Big Bench Audio, Max scored 98.8%, ranking third among 43 evaluated models, while Mini scored 75.9%. StepAudio 3 Realtime at 99.7% and Qwen Audio 3.0 Realtime Plus at 99.2% ranked ahead of Max. ElevenLabs' claim that Max is the fastest and most accurate combination sits in tension with the third-place accuracy ranking.
How fast does Eleven Realtime respond?
Average first audio response is 0.57 seconds for Max and 0.62 seconds for Mini. The figures are Artificial Analysis independent measurements under ElevenLabs-provided conditions. Perceived production latency is unconfirmed.
How do you migrate from the OpenAI Realtime API?
Migration is a two-line change: swap the endpoint, swap the model name, and the OpenAI SDK works as-is. Delivery is through the ElevenAgents platform or an OpenAI-compatible Realtime API. WebSocket is the connection method, and WebRTC is not supported at launch.