Real-time AI voice and digital-human apps need a reliable connection, but a “connected” socket can be a poor proxy for an actual working conversation. This original engineering runbook isolates idle timeouts, heartbeats and safe reconnects without confusing them with AI inference delay. It draws on MDN’s WebSocket API and server specifications plus AWS load-balancer troubleshooting guidance, consulted on 12 November 2026. All timing thresholds below are illustrative test inputs and must be verified in the deployed environment.

The failure that looks like a model stall

A user sees a live AI answer stop after a quiet interval. The microphone icon still looks active, but the next utterance never receives a reply. Teams often blame inference latency or the speech model without checking the networking path. A reverse proxy or application load balancer can close an idle connection while the app believes its WebSocket is usable. An idle timeout is different from a model deadline: the former is about traffic on a connection, the latter about how long a computation may legitimately run. Amazon’s Application Load Balancer troubleshooting documentation names idle timeouts among potential causes of 504 responses in HTTP request paths. A WebSocket upgrade can have different symptoms, so do not assume that every disconnect produces a 504. Verify the state at each network hop, and retain a bounded trace of timestamps rather than private conversation content.

Draw the whole connection path

Document the phone, cellular or Wi-Fi network, edge CDN, load balancer, gateway process, and downstream AI inference or transcription service. Record which leg is WebSocket, which uses HTTP streaming, and which hop injects a heartbeat. One socket from a browser to the gateway is not the same as a separate gateway-to-provider streaming session. Include the proxy idle timeout, TCP keepalive policy, server process lifecycle, mobile backgrounding rules and token expiration. A diagram with ownership makes investigations actionable: the client team handles reconnect state, operations owns edge timeout, and the AI team owns inference deadlines. Use timestamps from a monotonic clock where possible and correlate logs with a privacy-safe request ID. This prevents the false conclusion that a healthy downstream inference job implies the user’s browser still has an active connection.

Heartbeat frames are not application acknowledgements

WebSocket defines ping and pong control frames, as explained in MDN’s server documentation. Browsers do not expose a general ping-frame sender on the ordinary WebSocket JavaScript interface, so a product may implement an application-level heartbeat message for browser visibility. These serve different purposes. A server receiving a low-level pong knows the transport path replied; it does not prove that the screen rendered a transcript or that the speech engine made progress. Conversely, a heartbeat sent every second may waste battery and bandwidth. Choose an interval shorter than the shortest verified idle timeout but long enough to avoid needless mobile radio wakeups. For illustration only, if an edge closes idle sockets after 60 seconds, a 20–30 second heartbeat is a candidate to test, not a universal setting.

A six-case reconnect experiment

Create a repeatable test matrix rather than watching a successful demo once. First, hold the app foregrounded and silent for longer than the expected idle window. Second, move from Wi-Fi to cellular while an AI response is half streamed. Third, put the app into background and return after the network is suspended. Fourth, restart one gateway process while the model is still working. Fifth, expire an authorization token and verify that a reconnect is not silently accepted with stale privileges. Sixth, deliberately delay a server message without losing transport connectivity. For each condition, record open/close events, a bounded retry budget, visible UI state, message sequence identifier, and whether the user can recover without duplicating their voice command. A disconnected call must not silently continue recording.

Prevent retries from duplicating expensive work

A reconnect design should separate transport restoration from business action replay. Every submitted user turn receives a stable client-generated request identifier scoped to the authenticated conversation. When connectivity returns, the client asks the server whether that identifier was accepted, completed, or canceled. A blind resubmit risks generating two AI answers and double billing. Server operations must have explicit idempotency behavior and expiration boundaries; a remembered identifier is not proof of authorization. Keep a short-lived ledger with state transitions such as accepted, streaming, completed, canceled and failed. If the user interrupted the response before the socket failed, the server should not resume an obsolete turn without asking. MDN also warns that ordinary WebSocket interfaces do not provide automatic backpressure, which is a separate reason to bound in-flight messages during reconnection.

Set the release gate with measured outcomes

Define an acceptance budget appropriate to the product rather than advertising an invented “99.99% voice reliability” claim. In a controlled lab, measure disconnect detection delay, time to visible reconnect, duplicate action count, missing transcript segments, and unwanted microphone capture after cancellation. Segment measurements by mobile network, browser, app state and gateway instance. Include a failure injection where heartbeats still succeed but inference is frozen; an honest user-facing fallback should offer text or retry rather than display a falsely healthy live-video session. Compare the edge logs with client events and explain every disagreement. The resulting runbook is operationally useful because each problem has an owner, a reproducible trigger and a remediation test. It distinguishes reliable conversation continuity from merely keeping a socket open.

Sources

  1. MDN, Writing WebSocket Servers, living technical documentation, accessed 12 November 2026.
  2. MDN, WebSocket interface, living API documentation, accessed 12 November 2026.
  3. AWS, Troubleshoot your Application Load Balancers, living official documentation, accessed 12 November 2026.

Image: Tyler / Unsplash.