← BlogContact

Where the Tokens Went

I thought I had a prompt-size problem. The measurement showed something else: the workers were repeatedly reading context they did not need, and the coordinator was carrying more history than the next decision required.

The fix was not a shorter sentence at the top of the task. It was a change in what each part of the system was allowed to see.

A dense mass of repeated context funnels through a narrow aperture and emerges as a few compact, separately bounded packets feeding a final gate.
The expensive part of an agent run is rarely the instruction you wrote. It is everything the system keeps re-reading around it.

The first wrong diagnosis

The dashboard showed a worker receiving roughly 58,000 to 65,000 input tokens for a single task. My first reaction was to blame the task brief. That was incomplete. The explicit worker brief was already bounded, but the model could still spend turns inspecting the whole project, loading skills, and repeating tool context.

The local session evidence made the shape visible. In one active project session, there were 1,803 model calls, about 340.2 million cache-read input tokens, about 18.9 million cache-write input tokens, and only about 3.6 thousand fresh input tokens. The expensive thing was not the sentence that started the task. It was the repeated context around thousands of turns.

Input tokens in one session, by kind cache-read 340.2 million cache-write 18.9 million fresh input 3.6 thousand thousand million billion logarithmic scale — every gridline is a factor of ten
← swipe sideways to see the full chart →
The scale is logarithmic, which flatters the smallest bar enormously. On a linear axis the fresh-input bar would be too thin to draw at all: that is the whole finding.
Cache-read tokens may be cheaper than fresh input, but they still consume context, slow work, and hide where the agent is spending its attention.

What I was doing wrong

First, I had a legacy discussion function that sent the complete discussion history on every round. Round two received round one; round three received rounds one and two; and the pattern grew quadratically. The newer DaC discussion path already produced compact summaries, but the old path was still reachable.

Second, worker briefs described a job without always naming the exact files the worker owned. That invited a worker to discover the scope for itself. A worker with three files to change should not need to explore two hundred files to understand its assignment.

Third, review gates were given code and test excerpts without always carrying the authoritative file path. An excerpt is useful for speed, but it cannot prove an absence claim. “This symbol is never set” requires opening the real file.

Finally, I was treating a long worker run as evidence of diligence. It was often just repeated context. More turns are not automatically more quality.

The changes I implemented

BEFORE — history travels with every round round one round two round three amber blocks are earlier rounds re-sent — the worker also reads the whole repository AFTER — six bounded stages Scoped task one job, named files Compact DaC packet decisions, not transcripts Bounded worker run OWNED_PATHS only max 4 turns Verify the real file an excerpt cannot prove an absence One evidence assembly assembled once, not per gate Independent gate can still overturn nothing carries history it cannot act on — and the last stage still gets to disagree
← swipe sideways to see the full diagram →
The savings come from the boundaries, not from asking any stage to think less. The independent gate at the end is the part I was least willing to trade away.

1. Compact discussion summaries

The issue discussion path now uses the compact DaC protocol. Each round receives the prior round’s decisions, disagreements, findings, and open questions—not every previous response. It exits early when consensus is reached, while retaining the evidence needed to understand the decision.

2. Explicit worker ownership

Worker packets now carry an OWNED_PATHS field. The worker is told not to edit outside those paths and to report evidence only for them. This is both a safety boundary and a context boundary: ownership tells the worker what not to read.

3. Bounded worker sessions

Direct model input is capped at 12,000 characters and delegated briefs at 6,000 characters. Command Code workers run with a maximum of 4 turns, skip automatic skill discovery, and do not persist a session. These limits reduce accidental context growth without changing the implementation model or removing verification.

--max-turns 4
--no-skills
--no-session

4. Review excerpts plus authoritative paths

Review gates still receive a compact code and test excerpt. They also receive TARGET_FILE and an explicit instruction: an excerpt cannot prove absence. If a reviewer makes an absence claim, it must open and verify the target file first.

5. Hooks for the parts that can be enforced

Global hooks now block oversized worker briefs, transcript or full-repository dumps, unbounded shell output, unsafe secret handling, unauthorized worker Git operations, and unfinished idle agents. A hook cannot stop a worker from making a poor internal choice about which file to read, so the orchestrator’s ownership packet remains necessary.

What I deliberately did not change

I did not downgrade the quality model for implementation or independent verification. I did not remove the TDD forward pass. Acceptance criteria still inform tests, tests still inform implementation, and later gates still inspect the result. I did not replace verification with a cheaper excerpt-only review.

The goal is not “fewest tokens at any cost.” The goal is useful context: enough evidence to make a reliable decision, without repeatedly transporting history that cannot change it.

The operating rule

Measure before optimising. Find out whether the cost is fresh input, cached context, tool output, retries, parallel overlap, or a long-lived session. Then reduce repetition at the layer that creates it.

Scoped task → compact DaC packet → bounded worker run → authoritative-file verification → one evidence assembly → independent gate.

That preserved the part I care about: later reviewers can still challenge an earlier conclusion. The savings came from removing repetition, not from asking the system to care less.

The lesson: token usage is often an architecture problem disguised as a prompt problem. If every worker can see everything and every round repeats everything, a stronger model only makes the repetition more expensive.

Meharban Singh

Meharban Singh

AI systems / delivery architect. I build software with AI agents governed by rules, hooks, gates and independent review — and watch it in production after go-live.