Your Agent Is Only as Smart as Its Harness

Context management is not housekeeping around an agent. It is part of the capability you get from the model.

OpenAI’s ARC-AGI-3 experiment is unusually clear evidence. With the official harness, GPT-5.6 Sol scored 13.3% on the public task set. When OpenAI retained reasoning and replaced rolling truncation with compaction, the score rose to 38.3% while the model used six times fewer output tokens.

The model did not change. The harness did.

In the official harness, every game action discarded the model’s private reasoning. The next turn still included past moves and brief notes, but not the plans and insights behind them. Once the history exceeded its limit, the harness also removed the oldest messages.

Together, these two features of the harness—discarding reasoning and rolling truncation—helped explain why GPT‑5.6 Sol was struggling to learn over time.

Retaining the reasoning changed not only what the model remembered, but how it behaved:

With reasoning retained, we noticed two big changes. First, GPT‑5.6 Sol spent less time thinking before each action, because it no longer had to interpret the game from scratch every turn. Second, when it was able to remember its past thoughts, GPT‑5.6 Sol was much better at learning over time and employing coherent strategies.

This matches what I have seen in manual agent evaluations. When reasoning is discarded between actions, success rates fall. The agent may still have a transcript of what happened, but it loses the working theory that made those actions coherent. It has to reconstruct that theory on every turn, and the reconstruction is not always the same.

The second change is just as important:

The next improvement came from replacing rolling truncation with compaction, another setting in the Responses API.

The ARC-AGI-3 harness addresses context limits with rolling truncation. When the conversation context exceeds 175,000 characters, the oldest messages are discarded.

Rolling truncation has two drawbacks. First, the model loses earlier observations and actions. Second, it spends much of the tasks operating with a fuller context window, which can slightly impair performance.

Rolling truncation forgets by age. Compaction tries to preserve the state that matters. OpenAI’s compaction item carries prior state and reasoning into the next context window using fewer tokens. That makes compaction more than a cost optimization: it is a continuity mechanism.

OpenAI’s illustration of reasoning and tool use across compaction boundaries.

Selective pruning is a related but different tool. Anthropic exposes separate controls for thinking-block clearing and tool-result clearing. Vercel’s AI SDK offers pruneMessages, which can remove reasoning as well as tool-call and tool-result content. These controls can reduce cost and preserve room for new work, especially when tool outputs are large and no longer useful.

But context editing is not free. Every deletion changes the information available to the model and can break the continuity of its reasoning. It can also invalidate cached prefixes. My default is therefore to edit as little as the task allows: retain reasoning and compact before resorting to blind truncation. If the context must shrink, prefer compaction over pruning or deletion rules, because it is often difficult for the harness to predict reliably what information the agent will need later.

Another useful tactic is to keep large tool results out of the active context in the first place. A harness can save them to temporary, task-scoped files and return a path with a short description instead. The agent can then read only the relevant section—or search the file with grep—when it needs more detail. The full result remains available without consuming context on every subsequent turn, reducing the need to edit history while preserving continuity.

If you are building an agent harness, context management affects cost, latency, and caching. This result shows that it can also determine whether the agent appears capable at all.

The practical hierarchy is becoming clearer: compaction beats rolling truncation, and preserving continuity beats reconstructing it.