Runnable Python example: streaming chat completions with usage
"""Runnable example for openai-python v2.11.0
Streams a chat completion and requests a final usage chunk via
stream_options={"include_usage": True}. It prints streamed text as it
arrives, captures the final usage snapshot if delivered, and reports
whether the stream ended cleanly or was interrupted.
"""
import os
from openai import OpenAI
def main() -> None:
# Expect the model name in the environment, e.g.:
# export OPENAI_MODEL="gpt-4.1-mini"
model = os.environ["OPENAI_MODEL"]
client = OpenAI()
# Request a streaming chat completion with usage included.
stream = client.chat.completions.create(
model=model,
messages=[
{"role": "user", "content": "Explain streaming chat completions in one paragraph."},
],
stream=True,
stream_options={"include_usage": True},
)
final_usage = None # Will hold the CompletionUsage object if it arrives
interrupted = False
print("Assistant:", end=" ", flush=True)
try:
for chunk in stream:
# Emit text only from non-empty choices with a non-None delta.content.
for choice in chunk.choices:
content = choice.delta.content
if content is not None:
print(content, end="", flush=True)
# Capture the first non-null usage object as the total request usage.
if chunk.usage is not None and final_usage is None:
final_usage = chunk.usage
except Exception as exc: # An interruption (network error, manual cancel, etc.)
interrupted = True
print("\n[stream interrupted: {}]".format(exc))
# End-of-stream handling.
if not interrupted:
print("\n[stream ended cleanly]")
else:
print("[no further chunks due to interruption]")
# Usage reporting: only trust usage if a non-null usage object arrived.
if final_usage is not None:
print("\nToken usage (entire request):")
print(f" prompt_tokens: {final_usage.prompt_tokens}")
print(f" completion_tokens: {final_usage.completion_tokens}")
print(f" total_tokens: {final_usage.total_tokens}")
else:
print("\nToken usage: unknown (no final usage chunk was received)")
if __name__ == "__main__":
main()Interpreting streamed chunks: empty choices, nullable usage, and EOF
Treat each streamed ChatCompletionChunk as either delivering incremental text or bookkeeping metadata, and only trust token counts once a full usage object appears.
First, never assume choices contains text. The SDK type defines choices as a list that “can also be empty for the last chunk if you set stream_options: {"include_usage": true}”[3]. When include_usage is enabled, that final usage-only chunk has an empty choices list and exists only to deliver usage [2][3]. In your loop, emit delta text only when choices is nonempty and the first choice’s delta has content; if choices is empty, skip text handling.
Second, interpret usage as a nullable, streaming-specific field. On ChatCompletionChunk, usage is documented as “an optional field that will only be present when you set stream_options: {"include_usage": true} in your request” and, when present, “contains a null value except for the last chunk”[3]. The stream options contract adds that with include_usage: true all chunks include a usage field, but it is null on ordinary chunks and populated only on the extra final chunk before the wire terminator [2]. That final chunk’s usage is a CompletionUsage object whose prompt_tokens, completion_tokens, and total_tokens are required integers [4]. Usage numbers are therefore reliable only once you see a non-null chunk.usage and can read those integer fields. Before that point, treat usage as “not yet available,” even though the field is present with a null value.
Third, expect usage to arrive on a dedicated final chunk, not spread across all chunks. With include_usage set, the SDK specifies that it will stream “an additional chunk … before the data: [DONE] message,” where the usage field shows token statistics and choices is always an empty array; “all other chunks will also include a usage field, but with a null value”[2]. At the wire level, [DONE] is a special terminator line; the SDK’s streaming iterator explicitly stops reading when sse.data.startswith("[DONE]") and then closes the response [7]. Your Python loop never sees a chunk for [DONE], only the sequence of ChatCompletionChunk objects leading up to it.
End-of-stream behavior for usage falls into three practical patterns:
- Normal completion with usage: You iterate over all chunks without error, eventually see a chunk where
choicesis empty andusageis a non-nullCompletionUsageobject, and then the iterator ends. In this case, those integer counters (prompt_tokens,completion_tokens,total_tokens) are the totals for the entire request [2][3][4]. - Clean EOF without usage: The iterator finishes but you never saw a non-null
chunk.usage. The SDK notes that “if the stream is interrupted, you may not receive the final usage chunk which contains the total token usage for the request”[2][3]. When the loop exits without any non-null usage object, treat token usage as unknown rather than inferring it from partial data. - Interrupted stream: Exceptions or manual breaks stop iteration before the final usage chunk is processed. The
ChatCompletionChunkand stream options docs both warn that if the stream is “interrupted or cancelled, you may not receive the final usage chunk”[2][3]. Your calling code should treat any exception during iteration, or an intentionalbreakfrom the loop, as an interruption and avoid trusting usage—at best you will have only nullusagevalues and no reliable total.
In practice, robust logic is: only append text from chunks where choices is nonempty; track a final_usage variable that you assign once when you encounter chunk.usage is not None; after the loop, if final_usage is still missing, classify the request’s token usage as unavailable, whether the loop ended quietly without a usage chunk or was cut short by an error or manual stop.[2][3][4][7]
SDK compatibility for stream_options across specific versions
| SDK version | create signature | stream_options named argument | notes |
|---|---|---|---|
| "v1.25.2" | "create(..., stream=..., ...)" | "no" | "The synchronous client.chat.completions.create signature in v1.25.2 does not include a named stream_options parameter and has no **kwargs; only stream is present among the streaming-related arguments.[9]" |
| "v1.26.0" | "create(..., stream=..., stream_options=..., ...)" | "yes" | "In v1.26.0, the synchronous client.chat.completions.create signature adds a named stream_options parameter alongside stream, and the streaming overload returns Stream[ChatCompletionChunk].[8]" |
| "v2.11.0" | "create(..., stream=..., stream_options=..., ...)" | "yes" | "In v2.11.0, client.chat.completions.create continues to accept stream and stream_options as named keyword arguments in the streaming overload that returns Stream[ChatCompletionChunk]; the runnable example in this guide targets this signature. This comparison is limited to these three tagged versions and does not imply behavior outside their documented signatures.[1]" |