Event stream
Every assistant request answers with the same event stream, and everything a run says goes out through one function you can wrap.
The problem
AI endpoints tend to pick a transport per feature. Chat streams. “Generate a report” is a JSON POST. The slow one becomes a job you poll. Now the client has three parsers and three error paths, a button can’t show progress the way chat does, and a producer that gets slower means changing every caller.
Inside the run there’s a matching mess. Every progress line needs dressing: which step it’s under, which tool call made it, whether this person should see it. Pass that through every function and the signatures fill up with plumbing.
What I concluded
One stream, out
Every request (a chat turn, a command, a make) answers with the same server-sent event stream. Every event has the same frame: an id, the request id, a type, content and metadata.
event: thinking { content: "Checking 3 questions" }
event: content { content: "Most late orders ship from…" }
event: payload { metadata: { kind: "report", v: 1 } }
event: error { metadata: { status: 409, code: "…" } }
A button that wants one thing doesn’t need its own endpoint. It sends a make and reads the stream until its one payload arrives. So it gets live progress and openable steps for free, the same as chat.
One function, in
Inside the run, there’s one way to emit, and it lives on the context every piece of work already gets: ctx.inform(message). Behavior is added by handing work a context whose inform has been wrapped.
function within(ctx, stepId) {
return { ...ctx, inform: (msg, opts) => ctx.inform(msg, {
...opts,
metadata: { parentStepId: stepId, ...opts.metadata },
}) };
}
A step runs its work with within(ctx, id), and the work has no idea it’s nested. The same move tags parallel tool calls, strips debug detail for normal users, and adds a default “open” view when work didn’t attach one. Each returns a context, so they stack.
Rules
- Status lives in the event. Once the stream opens it’s already 200, so a later failure is an
errorevent with a code the client can act on. - Interrupt is not error. One means the assistant needs the person. The other means something broke.
- Clients ignore unknown events. Adding a type never breaks an old client.
- The request id is the trace id. A thumbs-down sends it back and lands on the exact trace.
- Wrappers change what’s emitted, never the work. A debug run and a normal run do exactly the same thing.
- The transport is the last listener. The HTTP layer subscribes and writes events out. Tests subscribe and collect them.
What you get
One parser, one error path, and one place to read events, whatever sent the request. You give up HTTP caching and resuming a dropped stream, which has been cheaper than running three transports. And you can add a new concern to every event without touching the code doing the work.
Related: Agent loop, Stateful or stateless, Streaming, Step tree.