OpenAI has detailed a new WebSocket-based approach for its Responses API, designed to cut latency in agentic workflows. According to the company's engineering post, the change replaces repeated HTTP request/response cycles with a persistent, bidirectional connection between client and server.
The Problem
Traditional agent systems make a series of independent HTTP calls to the model, each requiring a new TCP handshake, TLS negotiation, and repeated headers and metadata. For multi-step agent tasks that involve many back-and-forth calls, this overhead accumulates and can dominate perceived latency, separate from actual model inference time.
The WebSocket Approach
A WebSocket connection is opened once and stays open, allowing data to flow in both directions without repeated setup costs. OpenAI says this reduces network overhead, lowers latency for successive messages, and makes it easier to maintain context across a session.
The post also describes connection-scoped caching: frequently used data and metadata can be cached for the life of a WebSocket connection rather than resent with every message, reducing payload size in agent loops that repeatedly reference the same context or instructions.
Why It Matters
OpenAI frames the update as infrastructure aimed at agentic use cases — coding assistants and other applications that make many rapid calls — where communication overhead, not just model speed, has been a bottleneck. The company positions the shift from per-request HTTP calls to persistent WebSocket sessions as a step toward more real-time agent interactions.