API

Streaming

Server-sent events, with one extra frame type you will not have seen before: the router moving you to another machine mid-answer.

Turning it on

Set stream: true. The transport is server-sent events, so any client that already streams from an OpenAI-compatible endpoint works unchanged.

const stream = await client.chat.completions.create({  model: "qwen2.5-32b-instruct",  messages: [{ role: "user", content: "Draft the weekly summary." }],  stream: true,}); for await (const chunk of stream) {  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");}

The frames

Deltas arrive as chat.completion.chunk objects and the stream ends with [DONE]. The final content frame carries usage and the vacuum block — so the receipt ID arrives at the end of a stream, not the start.

sse

data: {"id":"cmpl_8Qd2","object":"chat.completion.chunk","model":"qwen2.5-32b-instruct",       "choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]} data: {"id":"cmpl_8Qd2","object":"chat.completion.chunk",       "choices":[{"index":0,"delta":{"content":"The week"},"finish_reason":null}]} data: {"id":"cmpl_8Qd2","object":"chat.completion.chunk",       "choices":[{"index":0,"delta":{},"finish_reason":"stop"}],       "usage":{"prompt_tokens":38,"completion_tokens":212,"total_tokens":250},       "vacuum":{"receipt":"0x9f3c1e7a2b8d40f6","node":"vac1q7f4mk2xw9d3hs8p",                 "epoch":2926,"cost_usdc":"0.0000628"}} data: [DONE]

If you need the receipt ID before the stream finishes, read the x-vacuum-receipt response header, which is set on the initial response. It is reserved when the request is accepted, so it is stable even if the request is later rerouted.

When a node drops

Nodes are consumer machines. Someone launches a game, a laptop sleeps, a connection flickers. When that happens mid-stream the router moves the request to another node and tells you so with a frame no other provider sends.

sse

data: {"vacuum":{"event":"rerouted","from":"vac1q7f4mk2xw9d3hs8p",                 "to":"vac1qp82ne6tz0k4wxv","reason":"node_disconnected"}}

What this means for your code

Content already delivered stands. The new node continues from the tokens you have, it does not replay them.
The seam may be visible. Two different machines produced the two halves, so style can shift slightly across the join. For most prose this is invisible; for strict formats, prefer a non-streaming call.
Ignoring the frame is fine. A client that does not know about it sees an uninterrupted stream. Handle it if you want to log reroutes or surface them.

If no replacement node can be found, the stream ends with an error frame of type stream_interrupted rather than [DONE]. Treat a stream that ends without [DONE] as incomplete.

Billing a stream

You are billed for the tokens delivered to you, once, no matter how many nodes were involved. A node that dropped mid-response earns nothing for that request and takes an uptime penalty for it.

If you disconnect early, generation stops and the reservation is settled against what had been produced at that point. There is no charge for the tokens you would have received. The receipt records the same truncated count.