Stateless is simpler until the agent starts doing things. Then the server should hold the conversation.

The problem

Every assistant has to pick who owns the transcript.

Stateless: the client keeps the history and sends all of it on every request. The server is a pure function. That’s how most chat demos work, and it’s a good default: easy to scale, easy to test, nothing to store or expire.

It starts to break when the agent does more than talk:

  • The transcript holds things the client never saw. Tool calls, tool results, ids the model used. The client ends up rebuilding a history it only half understands.
  • What happened isn’t in the text. The person pressed Stop, ignored a proposed action, or sent from two tabs at once. A transcript the client assembles has no record of any of it.
  • The client becomes an editor of the past. Whatever the client sends is what the model believes happened, including history that was trimmed, reordered or tampered with.
  • Requests grow every turn. The whole history goes over the wire, every time.

What I concluded

Hold the conversation on the server once the agent has tools. The client sends an id and only what’s new.

  stateless                        stateful
  ─────────                        ────────
  client ── whole history ──▶      client ── id + new text ──▶
           server (pure)                    server
                                            load ▸ run ▸ append
POST /assistant
{ conversationId: "8f2c",
  text: "Which orders are late?",
  results: [{ toolUseId: "c1", outcome: "done" }] }

The demo shows what that buys. Three things that usually get lost become facts the next turn can read:

  • A client call ends the request. If the agent asks the app to apply a filter, the answer ends there. The result comes with the next request, and a call it leaves out is recorded as declined, so the model never sees a call with no answer.
  • Stopping is recorded. The partial answer is saved with a marker saying the person stopped it, so the next turn doesn’t treat a half-sentence as finished.
  • Two tabs can’t interleave. Each append is a conditional write against the version it read. The first wins and the second is refused.

The trade-offs

  Stateless Stateful
Server Pure, nothing stored A store, keys, expiry
Request size Grows every turn Small and flat
What really happened Whatever the client says Exact, including tool calls
Branching and editing Easy on the client Has to be a server feature
Testing Trivial Pass in an in-memory store

When stateless is still right

Single-shot features, like summarizing a page or drafting one paragraph, have no tools to track and no history worth keeping. Stateless is the right call there, and it’s cheaper.

Rules

  • Keyed by the verified person. Nobody shares a conversation by guessing an id.
  • Narrowed access starts fresh. The old transcript holds things read under wider access.
  • It expires. Every stored row carries the same expiry.
  • The loop only knows load and append, so the store is swappable.

What you get

Small requests, a transcript that matches what actually happened, and an agent that remembers what it did, not only what it said. The cost is a store to run and less freedom on the client.

Related: Agent loop, Event stream, Queued messages.