Realtime

Overview

GPT Live uses WebRTC for microphone input and synthesized audio output. Grok Voice 3.0, Grok TTS, Grok Transcribe 2.0, and Muse transcription use WebSocket sessions. Select the transport for your model below.

Realtime model

Model	Transport	Input	Output
GPT Live amanai/gpt-live	WebRTC	Microphone + events	Audio + transcripts
Grok Voice 3.0 amanai/grok-voice-3.0	WebSocket	Audio + text	Audio + transcripts
Grok Voice TTS amanai/grok-voice-tts	WebSocket	Text	Streaming audio
Grok Voice Transcribe 2.0 amanai/grok-voice-transcribe-2.0	WebSocket	Microphone audio	Partial/final transcripts
Muse Voice Transcribe 1.0 amanai/muse-voice-transcribe-1.0	WebSocket	Microphone audio	Partial/final transcripts

For WebSocket models, create a session with POST /v1/realtime/sessions and send model, max_seconds, and max_credits. Connect to the returned websocket_path with session_id and client_secret.value. Audio input uses mono PCM signed 16-bit samples at 24 kHz. Grok TTS accepts text.delta followed by text.done; transcription clients finish input with input_audio_buffer.end. Close with DELETE /v1/realtime/sessions/{id}.

Session object

const session = {
  type: "realtime",
  model: "amanai/gpt-live",
  language: "id-ID",
  instructions: "Speak naturally and keep answers concise.",
  audio: { output: { voice: "cove" } },
  delegation: { type: "client", ack_filler: false }
};

Language presets: English (US) (en-US), English (UK) (en-GB), Indonesian (id-ID), Malay (ms-MY), Arabic (ar-SA), Czech (cs-CZ), Danish (da-DK), Dutch (nl-NL), Filipino (fil-PH), Chinese (Simplified) (zh-CN), Japanese (ja-JP), Korean (ko-KR), Spanish (es-ES), French (fr-FR), German (de-DE), Italian (it-IT), Macedonian (mk-MK), Persian (fa-IR), Polish (pl-PL), Portuguese (Brazil) (pt-BR), Romanian (ro-RO), Russian (ru-RU), Swedish (sv-SE), Thai (th-TH), Turkish (tr-TR), Vietnamese (vi-VN), Hindi (hi-IN). Any other valid bounded BCP-47 tag can be sent directly.

The Playground exposes instructions, voice, and acknowledgement. API clients may also send language directly; the edge compiles it into the session language policy.

GPT Live voices: arbor, breeze, cove, ember, juniper, maple, sol, spruce, vale.

WebRTC flow

const peer = new RTCPeerConnection();
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
for (const track of stream.getTracks()) peer.addTrack(track, stream);

peer.ontrack = (event) => {
  remoteAudio.srcObject = event.streams[0];
};

const offer = await peer.createOffer();
await peer.setLocalDescription(offer);

const response = await fetch("https://api.amanai.sh/v1/realtime/calls", {
  method: "POST",
  headers: {
    Authorization: "Bearer " + AMANAI_API_KEY,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    sdp: peer.localDescription.sdp,
    session
  })
});

const established = await response.json();
await peer.setRemoteDescription({ type: "answer", sdp: established.sdp });

The response includes the SDP answer, a private session_id, and a one-use session_control_token. Do not expose the control token in logs or URLs.

Conversation loop

1. Acquire microphone permission and add the audio track to the peer connection.
2. Negotiate the session once with POST /v1/realtime/calls.
3. Play the remote audio track and consume transcript or state events from the data channel.
4. Keep the same connection open for every turn so context is preserved.
5. Reconnect as a new session only after terminal failure or expiry.

Connect a Chat model to Live

Your application owns its conversation history and selects the Chat model. For a request that must use Chat, send recent turns to POST /v1/chat/completions from your backend, then append the verified answer to the open Live data channel as speakable context. This uses the existing endpoints and needs no Amanai conversation database.

// Application backend: use your own Amanai key and selected Chat model.
const chat = await fetch("https://api.amanai.sh/v1/chat/completions", {
  method: "POST",
  headers: { Authorization: "Bearer " + process.env.AMANAI_API_KEY,
             "Content-Type": "application/json" },
  body: JSON.stringify({ model: selectedChatModel, messages: recentTurns })
});
const answer = (await chat.json()).choices[0].message.content;

// WebRTC client: send the verified Chat answer to the open Live data channel.
events.send(JSON.stringify({
  type: "session.context.append", channel: "speakable",
  content: [{ type: "input_text", text: answer }]
}));

Live may also emit delegation.created for a client task; your application can call Chat and return that result with delegation.context.append. The model decides when to delegate, so an application that requires Chat for a specific request should call it explicitly. Keep API keys and Live control tokens on your backend. The current Amanai data-channel event format is distinct from the newer official OpenAI Live SDK event format.

Download the complete Live + Chat example. It includes a localhost demo and a private backend mode. Each successful Live setup and each Chat request is billed under your own Amanai key.

Cleanup

curl -X DELETE https://api.amanai.sh/v1/realtime/sessions/$SESSION_ID \
  -H "Authorization: Bearer $AMANAI_API_KEY" \
  -H "X-Amanai-Realtime-Control-Token: $SESSION_CONTROL_TOKEN"

Stop local microphone tracks, close the peer connection, and release the server session when the user ends the call.

Billing

GPT Live is billed per successfully established WebRTC session. WebSocket models are metered by their model-specific usage, within the session credit limit. Failed setup without service is not billed. See Billing & pricing for the current rate.