Przejdź do nawigacji

Multi-Context WebSocket

Stream expressive dialogue audio for multiple independent streams (contexts) multiplexed over a single WebSocket connection.

Each context, identified by a client-chosen context_id, behaves like an independent Text to Dialogue WebSocket session: it registers its own voices and settings, buffers its own text, and produces its own audio stream. This is useful for scenarios requiring concurrent or interleaved dialogue generations, such as conversational AI applications that need to handle interruptions.

The connection uses Eleven v3 dialogue models only (model_id must start with eleven_v3). The default model is eleven_v3_conversational.

Context setup

  • Every message must include a context_id. A message containing only close_socket is the exception.
  • The first message for a new context_id creates that context and must include voices (voice IDs to register for the context). Optional voice_settings and pronunciation_dictionary_locators are only accepted on this first message.
  • For eleven_v3_conversational, only one voice ID may be registered per context. For eleven_v3, you may register up to 10 voices per context.
  • A connection can hold at most 5 simultaneous contexts; close a context to free a slot.

Streaming text

  • Send inputs: an array of { "text", "voice_id", "new_turn"? }. Each voice_id must be registered for that context. Text for the same turn is buffered per context until the server has enough context, then partial audio chunks tagged with the context_id are emitted.
  • Set new_turn to true (or switch voice_id) to finalize the current prosody segment and start a new speaker turn.

Control messages

  • flush: force generation of the context’s buffered text.
  • close_context: flush the context’s remaining audio, emit its is_final message, and close it. Other contexts stay open.
  • close_socket: flush and close all contexts, then close the connection.
  • keep_alive: reset the context’s 20 second inactivity timeout (no generation). A context idle for longer is automatically flushed and closed (its is_final message is sent); other contexts are unaffected.

Protocol errors — a missing context_id, an unregistered voice, messaging a context that is closing, or exceeding the context limit — send an error payload and close the whole connection.

Authentication

Authentication is connection-level, not per context: use the xi-api-key or Authorization header, single_use_token query parameter, or include xi_api_key, authorization, or single_use_token in the first message of the connection. Anonymous sessions are rejected.

For a single dialogue stream per connection, see the Text to Dialogue WebSocket. For non-streaming dialogue over HTTP, see Create dialogue and Stream dialogue.

Uzgadnianie

WSS
/v1/text-to-dialogue/multi-stream-input

Nagłówki

xi-api-keystringOpcjonalny

Parametry zapytania

model_idstringOpcjonalnyDomyślnie eleven_v3_conversational

Identifier of the model that will be used, you can query them using GET /v1/models. Must be a v3 model.

output_formatanyOpcjonalny

Output format of the generated audio. Formatted as codec_sample_rate_bitrate. So an mp3 with 22.05kHz sample rate at 32kbs is represented as mp3_22050_32.

language_codestringOpcjonalny

Language code (ISO 639-1) used to enforce a language for the model and text normalization. If the model does not support the provided language code, it will be ignored. This parameter is not supported for multilingual_v2 models.

sync_alignmentbooleanOpcjonalnyDomyślnie false

When true, character timing from the model may be attached to audio chunks as alignment (snake_case field names).

apply_text_normalizationanyOpcjonalny
A string parameter to control text normalization.
seedintegerOpcjonalny1-4294967295
If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed.
enable_loggingbooleanOpcjonalnyDomyślnie true

When enable_logging is set to false zero retention mode will be used for the request. This will mean history features are unavailable for this request, including request stitching. Zero retention mode may only be used by enterprise customers.

Wyślij

text_to_dialogue_websocket_client_multiobjectWymagany

JSON text frame from client. Every frame carries a context_id; the first frame for a context must register voices, later frames stream inputs and control flags.

Odbierz

text_to_dialogue_websocket_audio_chunk_multiobjectWymagany

Partial audio for a context's current generation, base64-encoded using output_format, tagged with the context_id it belongs to.

OR
text_to_dialogue_websocket_final_audio_for_turn_multiobjectWymagany

Sent after the final audio for a given turn of a context has been sent, tagged with the context_id. For streaming codecs (MP3, Opus, etc.), the persistent encoder buffers a small amount of audio across turn boundaries, so a few bytes belonging to the marked turn may arrive interleaved with the next turn's first audio chunk. Clients that need exact per-turn boundaries should request a PCM output format, which has no buffering.

OR
text_to_dialogue_websocket_final_multiobjectWymagany

Sent for a context once it is finalized (via close_context, close_socket, or inactivity timeout) and all its audio has been flushed.

OR
text_to_dialogue_websocket_errorobjectWymagany
Error payload sent before the socket is closed with a WebSocket close code.