Streaming replies are attractive for AI social experiences because they reduce the perceived delay before content appears. The hard engineering work is preventing a connection interruption from changing what the user sees or pays for.

SSE is a transport, not an exactly-once contract

Server-Sent Events give a browser a convenient one-way event stream, but a network reconnection can replay or omit application-level events depending on the server’s retention policy. MDN documents the event, data, id and retry fields used in event streams. The id field supplies a last-event identifier that can help a client resume, but it does not promise that a model inference ran only once or that a database mutation was applied only once. For AI chat, distinguish three separate properties: a response was generated, its events were delivered, and its text was rendered. Each needs a durable identifier and a reconciliation rule. Without them a smooth-looking reconnect can produce duplicated sentences or hidden duplicate usage charges.

Version a structured event envelope

Define conversation_id, response_id, schema_version, sequence, event_type and payload. Use explicit event types such as response_started, text_delta, tool_progress, response_completed and response_failed instead of inferring completion from a closed socket. A final completion event should carry an authoritative text length or digest for reconciliation. Avoid storing access tokens, personal data or customer secrets inside the event id field, because identifiers may be logged by proxies. The client should validate schema and message size before applying any event. Treat all streamed model text as untrusted content, rendering plain text safely and never interpolating raw payloads into HTML. A response schema deserves the same compatibility discipline as any other externally consumed API.

Deduplicate according to semantic response IDs

Maintain a highest contiguous applied sequence number for each response and a bounded buffer of out-of-order events. A replay with sequence less than or equal to that checkpoint should not append visible text a second time. An event arriving ahead of a gap should wait briefly, then trigger a replay or explicit reconciliation path. Never assume every stream chunk is a token; a chunk can contain multiple characters, a tool status or a completion marker. Keep durable committed messages separate from transient preview text until the response completion rule is satisfied. This prevents the browser from treating a partially visible message as a final assistant answer. Retain only a minimal stream checkpoint according to the product’s storage policy.

Reconnect without launching a second inference

Issue the response identifier before the generator begins. A reconnecting EventSource request should authorize the same user and resume from the event buffer, not create a fresh model run. If the buffer has expired, return a terminal status that directs the client to a conversation-reconciliation endpoint rather than pretending an invisible event never happened. Keep a cancellation token attached to the response identifier so a user can stop generation even after a network break. Distinguish a canceled response from a failed one and avoid charging twice for resumed transport. Set bounded timeouts for inactive streams and clean up server resources when the browser truly leaves the conversation. Browser-native automatic retries must never become an uncontrolled backend cost multiplier.

Test loss at four distinct boundaries

Interrupt transport before the first event, in the middle of text, immediately after a completion event reaches the server, and after the client has rendered completion but before its acknowledgement is recorded. In each case, reconnect with the last known event identifier and compare visible output to the authoritative response. Inject duplicate and out-of-order ids, invalid JSON, oversized payloads and an expired session. The release gate is one visible response with exactly the expected content and one recorded generation expense. Repeat through a reverse proxy with buffering enabled and disabled, because a development server’s immediate flush behavior may conceal production buffering. Record both first-event latency and full-completion latency for meaningful user experience comparison.

Specify error semantics after headers are sent

An HTTP failure can describe a problem before streaming headers have started, but once a stream is underway the application may need an explicit response_failed event with a safe error code. Define separate categories for authentication expiry, model overload, unavailable tools, policy refusal and canceled generation. Do not stream raw exception traces or hidden prompts as diagnostic data. The browser should show a clear retry choice only when a retry is safe. For example, resuming an incomplete reply is different from repeating a state-changing action such as sending a message or creating a payment. Maintain an idempotency key for side effects and a response reconciliation record, allowing frontend and backend teams to debug without retrieving the complete personal conversation.

Publish operational indicators, not just demos

The HTML Living Standard’s server-sent events section and MDN explain transport behavior, while product reliability depends on the surrounding application contract. Track reconnect rate, duplicate suppressions, missing completion events, buffered event age and requests that unexpectedly launch a second inference. Review rates by browser, network and deployment version. Alert on changes to reconciliation error rates rather than declaring success because a stream eventually prints words. A good AI social experience makes interruption understandable: users know whether a response is still composing, has finished, was canceled or needs an explicit restart. Reliable streaming is defined by reproducible accounting and state, not by visual smoothness alone.

Sources