AI and agents · Agentic engineering · 8 min
Context for agents: keeping focus through deliberate compaction
A large context window does not guarantee that an agent uses all the information equally well. Selection and summarization must preserve evidence and useful state.
More context does not automatically mean better context
Extending a session can preserve useful decisions, but it also accumulates traces, repeated results, outdated content and changes of goal. The model receives all of these as part of a finite input; choosing what stays is an engineering decision, not just a budget one.
The “Lost in the Middle” study found that some of the evaluated models performed worse on retrieval tasks when the relevant information sat in the middle of a long context. It is evidence for specific tasks and models, not proof that every model fails in the same way or that a single cut-off point exists for each window.
“Smart Zone” as a heuristic, not a universal threshold
In an interview summarized in 2026, Dex Horthy describes the “smart zone” and the “dumb zone” as a practical way to talk about the deterioration he observed when extending certain flows. The report mentions around 100 thousand tokens as a reference from his experience; that figure depends on the model, the task, the order of information, the retrieval system and the tools. It is not a general limit, nor a result established by the “Lost in the Middle” study.
The operational signal matters more than a fixed number: the agent repeats attempts, contradicts verified decisions, loses constraints or cannot tell evidence from hypothesis. Those symptoms justify reducing and reordering the state, even when the token counter is below the maximum advertised by the provider.
Compact while preserving evidence and state
A useful summary is not a shorter version of the whole conversation. It is a concise working state that lets work continue with traceability:
- Current goal and acceptance criterion.
- Verified facts with their source or file of origin.
- Decisions made, assumptions still open and safety limits.
- Tests run, relevant results and pending failures.
- Concrete next step and conditions for stopping or requesting review.
Design a verifiable compaction cycle
Compact when closing a milestone or when you detect a loss of focus. Separate retrieved content from instructions; discard duplicates and stale data; validate the summary against primary sources before starting a new context. For long tasks, keep short, versioned artifacts in the workspace approved by the team, taking care not to persist secrets or unnecessary data.
Evaluate the strategy with representative cases: does the new session keep decisions, constraints and pending tasks? The goal is not to reach a universal “optimal” size, but to reduce noise without erasing decisive information.
Sources and rigor
References
- Dex Horthy / HumanLayer: 12 Factor Agents, 2025-04-03
- Liu et al.: Lost in the Middle: How Language Models Use Long Contexts, TACL 2023
Evaluates information retrieval at different positions in the context for specific tasks; it does not establish a universal token threshold.
- BigGo: Dex Horthy and the Dark Factory experiment, 2026-07-15
Editorial summary of an interview; the case serves as a reported experience, not as a controlled study or a measurement of the whole industry.
Conclusion
The context window is a resource to be managed: selection, provenance, compaction and continuity tests are more useful than assuming more tokens always solve the problem.
Keep exploring