Streaming

With stream: true the model returns output as it generates it. This suits chat interfaces and avoids long waits on lengthy answers. You can stream through Chat Completions; models that support the Claude-native format can also be streamed through the Anthropic Messages format (see the last section of this page).

Replace your-model-id in the samples with a model ID from the model square (or call GET /v1/models).

Chat Completions streamingCopy Anchor Link

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MYAIEXP_API_KEY"],
    base_url="https://ai.myaiexp.com/v1",
)

stream = client.chat.completions.create(
    model="your-model-id",  # replace with a model ID from the model square
    messages=[{"role": "user", "content": "Count from 1 to 5, one number per line."}],
    stream=True,
)

for chunk in stream:
    # some chunks have no choices or content (e.g. first/last chunk); check first
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
print()

Response formatCopy Anchor Link

The server pushes Server-Sent Events (SSE). Each line starts with data: followed by a JSON chunk; the incremental text is in choices[0].delta.content, and the stream ends with data: [DONE]:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","model":"your-model-id","choices":[{"index":0,"delta":{"content":"1"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","model":"your-model-id","choices":[{"index":0,"delta":{"content":"\n2"},"finish_reason":null}]}

data: [DONE]

With an SDK you don't parse this yourself; just iterate over the stream.

To get token usage while streaming, add "stream_options": {"include_usage": true} to the request. Usage then appears in the usage field of the final chunk (whose choices is an empty array).

Anthropic Messages streamingCopy Anchor Link

When calling /v1/messages, also add "stream": true. The response is the standard Anthropic SSE event stream; incremental text is in delta.text of content_block_delta events. The Anthropic SDK offers a messages.stream() helper:

import os

import anthropic

client = anthropic.Anthropic(
    api_key=os.environ["MYAIEXP_API_KEY"],
    base_url="https://ai.myaiexp.com",  # note: no /v1
)

with client.messages.stream(
    model="your-model-id",  # replace with a model ID from the model square
    max_tokens=512,
    messages=[{"role": "user", "content": "Count from 1 to 5, one number per line."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
print()

Events arrive roughly in the order message_start → content_block_start → several content_block_delta → content_block_stop → message_delta (with stop_reason and output usage) → message_stop:

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"1\n2"}}

event: message_stop
data: {"type":"message_stop"}

See Anthropic SDK / Messages format for more.

NotesCopy Anchor Link

• The first chunk's content may be an empty string and some chunks have no choices; check before reading (the examples do).
• Add -N to curl, otherwise output is buffered and looks like it arrives all at once.
• Behind a reverse proxy such as Nginx, disable proxy buffering or the stream is collected into a single block.
• Thinking and reasoning models take longer to emit the first token, so streaming suits them better.
• Streaming is also a good way to avoid timeouts; see Errors: Timeouts.