Context Engineering for AI Agent Systems
1. Definition and thesis
Context Engineering is the systematic design of an agent’s runtime information environment. It includes prompt wording, but its central object is the context window: the finite sequence of tokens the model can attend to during one generation. In modern agent systems, that sequence is assembled from many competing sources: root instructions, developer guidance, tool definitions, schemas, retrieved passages, intermediate tool results, memory summaries, scratchpads, user turns, and output budget.
The shift from “prompt engineering” to “context engineering” is not just a vocabulary change. Prompt Engineering asks, “What should we say to the model?” Context engineering asks, “What should be visible to the model right now, what should be hidden but recoverable, what should be compressed, and what should be made durable outside the transcript?” Anthropic’s 2025 engineering guidance frames this as curating a “limited attention budget” and maximizing “high-signal tokens,” while OpenAI’s 2025–2026 agent guidance emphasizes stable cached prefixes, dynamic suffixes, tool search, compaction, and session state as first-class design concerns. ([Anthropic Engineering: Effective Context Engineering][1]; [OpenAI Guide: GPT-5.5 prompting][9]) Anthropic
The core claim of this article is: agent reliability increasingly depends less on finding a magic prompt and more on building a context compiler. That compiler selects, orders, trims, caches, retrieves, and externalizes information under token, latency, cost, and accuracy constraints.
2. Why prompt engineering became insufficient
A single-turn assistant can often be improved by rewriting instructions. An agentic application is different. It calls tools, reads files, searches databases, accumulates intermediate state, delegates subproblems, and may run across many turns. The “prompt” becomes a changing composite object rather than a fixed text block.
Anthropic’s “effective context engineering” guidance explicitly describes agent systems as using just-in-time context, lightweight identifiers, targeted queries, and progressive disclosure rather than loading entire data objects into context. Claude Code is presented as an example: it stores file paths and lightweight references, uses targeted shell queries, and loads only the needed slices of data instead of whole logs or files. ([Anthropic Engineering: Effective Context Engineering][1]) Anthropic
OpenAI’s 2026 guidance around “Shell + Skills + Compaction” similarly treats long-running agents as systems that need persistent procedures, external execution, and continuity across context limits. Skills are described as procedures the model loads on demand via metadata, while compaction is used to keep long agent runs moving when the conversation outgrows the available window. ([OpenAI Blog: Shell + Skills + Compaction][10]) OpenAI Developers
Google’s Gemini and ADK guidance points in the same direction from a different angle. Gemini’s long-context documentation calls the context window a form of short-term memory, while Google’s ADK material describes sessions and event streams as the definitive application state from which model context is selected, transformed, and injected. ([Google Gemini: Long context][13]; [Google ADK: Context-aware multi-agent systems][16]) Google AI for Developers
The pattern across vendors is clear: the prompt is one component in a larger Agent Runtime.
3. Context window as working memory, not storage
A context window is best understood as Working Memory, not as a database. Anthropic’s documentation defines the context window as the text the model can reference while generating a response, distinguishes it from training data, and warns that larger windows are not automatically better because recall and accuracy can degrade as the token count grows — a phenomenon Anthropic calls “context rot.” ([Anthropic Docs: Context windows][2]) Claude Platform
Google makes the same distinction in its Gemini documentation: the context window is the model’s short-term memory, and only a limited amount of information can be kept there at once, even when the model family supports very large context windows. Google also notes that smaller contexts often require dropping older messages, summarization, retrieval, or filtering, while long context can reduce but not eliminate the need for those techniques. ([Google Gemini: Long context][13]) Google AI for Developers
OpenAI’s session-memory cookbook defines context as the total input-plus-output token window the model can attend to, and argues that even large windows can become overwhelmed by uncurated histories, verbose tool results, and noisy retrieval packs. It treats trimming, summarization, and state management as necessary for sustained coherence, tool-call accuracy, cost control, hallucination containment, debugging, and handoff resilience. ([OpenAI Cookbook: Session memory][7]) OpenAI Developers
This distinction matters architecturally:
| Misconception | Better framing | Design implication |
|---|---|---|
| “The model remembers everything in the chat.” | The model attends to the current rendered context. | Store durable state outside the transcript and reinsert only what is relevant. |
| “A bigger context window solves memory.” | Larger windows increase capacity but do not guarantee retrieval, salience, or correctness. | Curate, rank, and structure context even with long-context models. |
| “Summaries are memory.” | Summaries are lossy compression artifacts. | Track what was compressed, when, and what facts are no longer directly evidenced. |
| “Tool output is free scratch space.” | Tool results consume tokens and attention. | Filter, slice, cite, or externalize tool results before reinserting them. |
| “The prompt is the system.” | The prompt is one rendered view of a larger state machine. | Build a context assembly pipeline. |
The practical rule is: store broadly, expose narrowly. The application may maintain files, databases, plans, vector indexes, event streams, and audit logs. The model should see the smallest set of high-signal tokens needed for the next decision.
4. The context object: what the agent sees vs. what the application knows
A mature agent system separates application state from model-visible context.
Application state can include the full transcript, durable user preferences, task plans, files, database rows, tool outputs, traces, evaluation scores, and human annotations. Model-visible context is the subset rendered for one generation. Conflating the two creates transcript bloat: every previous message, tool result, failed attempt, and stale assumption remains visible simply because it happened earlier.
OpenAI’s Responses and Conversations APIs illustrate the distinction. The Responses API can chain state through previous_response_id, while the Conversations API provides a durable identifier that stores messages, tool calls, tool outputs, and related items. The state exists outside the model’s immediate context and can be rendered selectively. ([OpenAI Docs: Conversation state][5]) OpenAI Developers
OpenAI’s Agents SDK session memory also treats session history as an application-level mechanism: before a run it retrieves prior items and prepends them, and after a run it stores new items. The documentation warns not to combine multiple state mechanisms casually, which reflects the broader point that memory is an application design choice, not an automatic property of the model. ([OpenAI Agents SDK: Sessions][6]) OpenAI GitHub
Google’s ADK framing is even more explicit: the session is described as definitive state, containing metadata, a state scratchpad, and chronological events rather than merely raw prompt strings. Context processors then select, transform, and inject material into the model context. ([Google ADK: Context-aware multi-agent systems][16]) Google Developers Blog
This separation produces a useful abstraction: the context compiler. It takes application state as input and emits a model-readable context packet.
application state ├─ transcript ├─ durable memory ├─ plans and tasks ├─ files and artifacts ├─ tool results ├─ retrieval indexes ├─ traces and evaluations └─ user/session metadata ↓ context compilermodel-visible context for this turn ├─ stable prefix ├─ current task ├─ selected state ├─ selected tools ├─ retrieved evidence ├─ recent turns ├─ summaries or compaction artifacts └─ output constraints
The context compiler is where most agent reliability work now lives.
5. Static-prefix / dynamic-suffix architecture
The most important low-level pattern in current vendor guidance is the Static Prefix / Dynamic Suffix split.
A static prefix contains stable, reusable material: root instructions, product contract, safety or policy constraints, durable style guidance, stable tool definitions, output schemas, and long-lived reference material. A dynamic suffix contains per-turn material: the current user request, selected conversation history, retrieved snippets, fresh tool results, temporary plans, and task-specific constraints.
OpenAI’s GPT-5.5 prompting guide states this pattern directly: optimize caching by putting stable content first and dynamic user-specific content last; start with the smallest prompt that preserves the product contract; keep tool-specific guidance in tool descriptions; and preserve completed actions, assumptions, IDs, tool outcomes, blockers, and the next goal during compaction. ([OpenAI Guide: GPT-5.5 prompting][9]) OpenAI Developers
Anthropic’s prompt-caching documentation recommends placing static content such as tool definitions, system instructions, context, and examples at the beginning of the prompt, because cache prefixes are computed over ordered prompt prefixes and changes before a breakpoint can invalidate reuse. ([Anthropic Docs: Prompt caching][3]) Claude Platform
Google’s context caching documentation gives the same operational advice: put large common content at the beginning of the prompt to maximize cache hits, and understand that cached content acts as a prefix within the model request. ([Google Gemini: Context caching][14]) Google AI for Developers
A simplified architecture looks like this:
STABLE PREFIX root/system instructions product contract global refusal/safety rules stable output format stable tool namespaces or tool summaries cached reference docs few-shot examples, if still worth their token costDYNAMIC SUFFIX current user turn current task decomposition selected recent turns selected durable memory selected retrieval pack selected tool results current constraints and deadlines compaction or summary state
The static prefix is not merely an optimization for cost. It stabilizes model behavior by keeping invariant constraints in a consistent location. The dynamic suffix is not merely appended chat history. It is the per-turn evidence and state needed for the next action.
6. Prompt caching as a context architecture constraint
Prompt caching turns context layout into an economic and latency concern.
Anthropic’s prompt caching lets clients resume from specific prompt prefixes, reducing processing time and cost when repeated prefix content is reused. Its documentation describes explicit cache breakpoints, automatic caching in multi-turn conversations, and a cache-prefix order that includes tools, system messages, and user messages. ([Anthropic Docs: Prompt caching][3]) Claude Platform
OpenAI’s prompt-caching documentation describes cached tokens for prompts above minimum lengths and notes that messages, images, tool use, and tools can be part of the cached prompt. The GPT-5.5 guide also instructs developers to place stable content at the beginning and dynamic content at the end, then track cached tokens in usage metrics. ([OpenAI Docs: Prompt caching][11]; [OpenAI Guide: GPT-5.5 prompting][9]) OpenAI Developers
Google supports implicit and explicit context caching. Its Gemini API documentation describes implicit caching for repeated prefix content and explicit caching for substantial initial context reused across shorter requests, with cached content serving as a prefix and token accounting exposed through usage metadata. ([Google Gemini: Context caching][14]) Google AI for Developers
The design implication is straightforward: context should be ordered by expected stability.
| Context component | Prefix or suffix? | Rationale |
|---|---|---|
| Root product contract | Static prefix | Stable, high authority, cacheable. |
| Tool namespace summaries | Static prefix, if stable | Needed for routing; can be cached. |
| Full tool schemas | Mixed | Stable schemas can be cached; large catalogs should be dynamically loaded. |
| Few-shot examples | Static prefix only if still useful | Expensive; cacheable; should be evaluated. |
| User’s current request | Dynamic suffix | Changes every turn. |
| Retrieved passages | Dynamic suffix | Task-specific and volatile. |
| Recent turns | Dynamic suffix | Useful for local coherence, but grows quickly. |
| Compaction state | Dynamic suffix or boundary artifact | Generated from prior context; should be auditable or at least tracked. |
| Large files/logs | Usually external | Retrieve slices rather than load wholesale. |
A poor cache layout forces every turn to reprocess mostly identical instructions. A poor context layout also makes it harder to reason about what the model actually saw.
7. Tool schemas as context consumers
Tools are not just capabilities. They are also tokens.
Every function name, description, parameter schema, enum, and usage instruction competes with other context. Anthropic’s tool-use documentation says input pricing includes the total input sent to the model, including the tools parameter, and identifies tool parameter names, descriptions, schemas, tool_use blocks, tool_result blocks, and tool-related system prompts as token contributors. ([Anthropic Docs: Tool use pricing and token accounting][4]) Claude API Docs
OpenAI’s tool documentation similarly defines function tools through JSON Schema, including names, descriptions, parameters, and strict schemas. Tool outputs are sent back to the model as additional context. ([OpenAI Docs: Function calling and tools][12]) OpenAI Developers
For small applications, including every tool up front may be acceptable. For large agent systems, it becomes a routing and budget problem. Anthropic’s advanced tool-use guidance anticipates agents operating across hundreds or thousands of tools and warns that tool definitions and results can consume very large amounts of context. It recommends patterns such as tool search, programmatic tool calling, and selective examples to manage tool bloat. ([Anthropic Docs: Advanced tool use][17]) Anthropic
OpenAI’s tool-search guidance makes the same move: rather than loading all tool definitions up front, the model can see only namespace-level descriptions and dynamically load detailed tool definitions when needed. OpenAI also notes that loaded tools are placed at the end of context to preserve cache behavior. ([OpenAI Docs: Tool search][18]) OpenAI Developers
Tool schemas therefore need the same engineering discipline as prompts:
| Tool-context problem | Symptom | Engineering response |
|---|---|---|
| Overloaded tool catalog | Model calls wrong tool or pays high context cost. | Group tools by namespace; use tool search or dynamic loading. |
| Verbose descriptions | Tool instructions crowd out task evidence. | Move only stable, discriminative routing guidance into descriptions. |
| Ambiguous schemas | Wrong arguments, retries, brittle repairs. | Use strict schemas, enums, validation, and concise examples where justified. |
| Large tool results | Context flooded with raw logs, tables, or documents. | Return IDs, snippets, summaries, or handles; load details on demand. |
| Hidden tool semantics | Model cannot infer side effects. | Put side effects and preconditions in the tool description or wrapper. |
The emerging rule is: tool descriptions are part of the prompt, and tool outputs are part of the memory budget.
8. Retrieval as context curation
Retrieval-Augmented Generation is often described as giving the model access to external knowledge. For agent systems, a better framing is retrieval as context curation. Retrieval decides which evidence becomes visible, in what order, at what granularity, with what metadata, and for which step of the task.
Anthropic’s guidance emphasizes “just-in-time” context: store lightweight identifiers, query targeted information, and avoid loading full data objects unless needed. It recommends progressive disclosure — keeping only necessary information in working memory and using note-taking or external stores for persistence. ([Anthropic Engineering: Effective Context Engineering][1]) Anthropic
Google’s long-context guidance is notable because it does not simply say “use the full million-token window.” It says the primary optimization for long context is context caching, notes that multiple-needle retrieval is harder than single-needle retrieval, and advises developers not to pass unneeded tokens. It also observes that placing the query near the end of a long prompt can improve performance. ([Google Gemini: Long context][13]) Google AI for Developers
Retrieval quality is not only recall. It is also salience. A retrieval system that returns ten vaguely relevant chunks may make the answer worse than a system that returns three sharply relevant passages with metadata and provenance.
A practical retrieval pack should usually include:
| Field | Purpose |
|---|---|
| Source identifier | Enables citation, re-fetching, and audit. |
| Relevance rationale | Explains why this item was selected. |
| Minimal excerpt | Avoids dumping whole documents. |
| Timestamp or version | Prevents stale facts from masquerading as current. |
| Authority level | Distinguishes primary docs, user files, logs, and secondary summaries. |
| Task-specific instruction | Tells the model how to use or ignore the retrieved material. |
Retrieval becomes especially important when paired with externalized state. Instead of replaying all prior work, the system can retrieve the current plan, open issues, relevant files, and recent decisions. Instead of embedding entire documentation sites, it can retrieve targeted sections and ask the model to request more if needed.
The contested point is how much long-context models reduce the need for retrieval. Vendor documentation supports the claim that long context changes the tradeoff, but not the claim that it eliminates curation. Google explicitly says filtering, summarization, and RAG remain valuable techniques even with long context; Anthropic warns about context rot; and OpenAI warns that large windows can still be overwhelmed by uncurated histories and noisy retrieval. ([Google Gemini: Long context][13]; [Anthropic Docs: Context windows][2]; [OpenAI Cookbook: Session memory][7]) Google AI for Developers+2Claude Platform+2
9. Compaction, summarization, and context editing as lossy compression
When context grows beyond the useful or affordable window, systems compress. But compression is not neutral. It chooses what survives.
OpenAI’s session-memory cookbook distinguishes trimming from summarization. Trimming keeps the last N turns and is deterministic with no extra model latency, but can create abrupt amnesia. Summarization preserves longer-range information but introduces loss, bias, latency, cost spikes, compounding errors, and observability challenges. ([OpenAI Cookbook: Session memory][7]) OpenAI Developers
OpenAI’s server-side compaction reduces long-running conversation context while preserving state. Its documentation describes a context_management mechanism with compaction thresholds, an emitted compaction item, and an opaque encrypted artifact that carries state and reasoning. A standalone compact endpoint can also return a compacted window to use as canonical context for the next call. ([OpenAI Docs: Compaction][8]) OpenAI Developers+2OpenAI Developers+2
Anthropic’s context editing is different from summarization. It selectively clears specific content as a conversation grows, such as old tool results or thinking blocks, replacing cleared tool results with placeholders while the client application can still maintain the full unmodified history. Anthropic also exposes token counting to preview the effects of context editing. ([Anthropic Docs: Context editing][19]) Claude Platform+2Claude Platform+2
These techniques have different failure modes:
| Technique | What it does | Strength | Main failure mode |
|---|---|---|---|
| Trimming | Drops older turns, keeps recent turns verbatim. | Simple, deterministic, cheap. | Loses old requirements, decisions, and user preferences abruptly. |
| Summarization | Rewrites older context into a shorter natural-language summary. | Preserves some long-range continuity. | Introduces omissions, distortions, stale assumptions, and summary drift. |
| Server-side compaction | Produces a compacted continuation state, often opaque to the developer. | Enables long runs without manual prompt surgery. | Harder to audit exactly what was preserved or lost. |
| Context editing | Clears selected content classes, such as old tool results. | Removes low-value bulk while preserving structure. | May delete details later needed for debugging or grounding. |
| Retrieval replacement | Stores full material externally and retrieves slices when needed. | Keeps active context small and grounded. | Retrieval misses, ranking errors, or stale indexes. |
| External artifacts | Writes durable state to files, plans, or reports. | Human-auditable and stable. | Requires disciplined update and reloading policy. |
Calling all of these “memory” hides the engineering tradeoff. A summary is not the original. A compaction item is not an audit log. A cleared tool result is not recoverable to the model unless the application retrieves it again. Compression should therefore be designed around explicit invariants: facts that must never be lost, decisions that must remain auditable, IDs that must be preserved exactly, and uncertainty that must not be converted into false certainty.
10. Externalized state artifacts vs. transcript-based memory
The most robust agent systems do not rely on the transcript as the only memory substrate. They externalize state into artifacts: files, plans, tables, todo lists, issue trackers, databases, notebooks, citations, test results, and human-reviewed outputs.
Anthropic’s Claude Code memory system illustrates an artifact-oriented approach. It uses a MEMORY.md index, loads only the first portion at conversation start, and reads topic files on demand through standard file tools. This is a concrete example of persistent memory that is addressable rather than always in-context. ([Anthropic Docs: Claude Code memory][20]) Claude API Docs
OpenAI’s cookbook on memory and compaction makes a related distinction: compaction supports long-running conversations with finite context, memory lets later runs reuse workflow learnings without replaying every turn, and the final artifact remains the human-reviewed source of truth. ([OpenAI Cookbook: Memory and compaction][21]) OpenAI Developers
Transcript-based memory is convenient because it preserves chronology. It is also noisy. The transcript contains false starts, corrected assumptions, redundant tool outputs, abandoned plans, and user phrasing that may no longer reflect the task. External artifacts can be cleaner because they can represent current truth rather than historical chatter.
| Memory substrate | Good for | Bad for |
|---|---|---|
| Raw transcript | Local conversational continuity, audit trail. | Long-term factual state; grows unbounded; includes stale assumptions. |
| Recent-turn window | Immediate coherence. | Old constraints and decisions. |
| Summary memory | Carrying broad continuity across context limits. | Exact facts, citations, edge cases, unresolved uncertainty. |
| Durable user profile | Stable preferences and defaults. | Task-specific facts; high-risk assumptions. |
| Task artifact | Current plan, report, code, table, or decision record. | Implicit conversational nuance unless updated. |
| Database or file store | Exact state, IDs, references, reproducibility. | Requires retrieval and rendering policy. |
| Vector index | Semantic recall over large corpora. | Precision, provenance, and freshness unless carefully managed. |
The strongest pattern is to maintain both:
Transcript for chronology.
Artifacts for current state.
Retrieval for selective re-entry into context.
Compaction or summaries for continuity when exact replay is impossible.
This makes the model less dependent on remembering what happened and more capable of inspecting the state that matters.
11. Context budgeting
Context Budgeting is the practice of allocating the available token budget across competing context sources. It is emerging because every category of context has an opportunity cost.
The vendors do not provide a universal percentage allocation, and any fixed ratio would be misleading. The right budget depends on model, task, latency constraints, tool surface, retrieval quality, output length, safety requirements, and evaluation results. What the documentation does support is the need to account for distinct token consumers: tool schemas and results, system instructions, retrieved content, conversation history, cached prefixes, compaction artifacts, and output budget. ([Anthropic Docs: Tool use pricing and token accounting][4]; [OpenAI Cookbook: Session memory][7]; [Google Gemini: Context caching][14]) Claude API Docs+2OpenAI Developers+2
A useful context budget ledger looks like this:
| Budget line | What it contains | Main risk if overfunded | Main risk if underfunded |
|---|---|---|---|
| Root/system instructions | Product contract, policies, role, invariants. | Verbose rules crowd out evidence. | Model violates core behavior. |
| Tool catalog | Names, descriptions, schemas, examples. | Tool bloat; wrong attention; high cost. | Model cannot discover or call needed tools. |
| Retrieved evidence | Documents, snippets, citations, data rows. | Noise, contradiction, lost salience. | Ungrounded answers or missing facts. |
| Conversation history | Recent turns, user constraints, assistant commitments. | Transcript bloat and stale assumptions. | Loss of conversational continuity. |
| External state summary | Current plan, open tasks, durable memory. | Summary drift; false certainty. | Agent repeats work or loses task state. |
| Tool results | Observations, execution outputs, errors. | Raw logs overwhelm reasoning. | Agent lacks evidence for next step. |
| Examples | Few-shot demonstrations or style exemplars. | Expensive prompt ballast. | Poor format or behavior consistency. |
| Output reserve | Tokens left for answer, reasoning, or tool call. | Excessive generation budget. | Truncated answers or incomplete tool calls. |
Context budgeting should be implemented as accounting, not vibes. Count tokens before and after context editing, track cached tokens, log retrieval-pack size, measure tool-catalog size, and record whether failures correlate with particular context classes. Anthropic exposes token counting after context editing, OpenAI exposes cached-token accounting, and Google exposes cache-related usage metadata. ([Anthropic Docs: Context editing][19]; [OpenAI Docs: Prompt caching][11]; [Google Gemini: Context caching][14]) Claude Platform+2OpenAI Developers+2
A minimal budget policy for an agent loop can be expressed as:
1. Reserve output tokens first.2. Insert stable prefix from cacheable system and tool material.3. Insert the current user request.4. Insert hard state: IDs, deadlines, constraints, active plan.5. Insert retrieved evidence up to a measured cap.6. Insert recent turns until the history budget is exhausted.7. Insert summaries or compaction state only when needed.8. Drop, retrieve, or externalize everything else.
The important move is to treat budget allocation as part of product behavior. A legal-research agent should allocate more budget to cited source text. A coding agent should allocate more budget to active files, failing tests, and tool outputs. A customer-support agent may allocate more budget to policy snippets and user account state. A planning agent may allocate more budget to current goals and durable commitments.
12. Vendor comparison: Anthropic, OpenAI, and Google
The three major vendors converge on the same general architecture but emphasize different primitives.
| Dimension | Anthropic | OpenAI | |
|---|---|---|---|
| Core framing | Explicitly names “context engineering” as curating limited attention and high-signal tokens. | Frames context management through Responses, Agents SDK sessions, prompt caching, compaction, skills, shell, and tool search. | Frames around long context, context caching, Gemini tool use, and ADK context processors. |
| Context window model | Working memory distinct from training data; warns about context rot. | Total input-plus-output window; warns that large windows can be overwhelmed by uncurated history and tool results. | Short-term memory with large context windows; advises filtering, summarization, RAG, and caching. |
| Cache strategy | Prompt caching with explicit and automatic cache behavior; static content should appear early. | Prompt caching for reusable prefixes; GPT-5.5 guide says stable first, dynamic last. | Implicit and explicit context caching; large common content should be placed at the beginning. |
| Tool strategy | Advanced tool use emphasizes tool search, programmatic tool calling, and selective examples. | Tool search and deferred loading reduce up-front schema cost; tool descriptions and JSON schemas define function behavior. | Built-in and custom tools; function calling returns structured calls that the app executes and reinserts. |
| Compression strategy | Context editing clears selected old tool results or thinking blocks while client can retain full history. | Server-side and standalone compaction; trimming and summarization patterns in session memory. | ADK processors provide insertion points for filtering, compaction, caching, and routing; long-context docs discuss summarization and filtering. |
| State strategy | Memory tool and file-based memory patterns; Claude Code loads memory indexes and reads topic files on demand. | Responses state, Conversations API, Agents SDK sessions, memory and compaction cookbooks. | ADK sessions, state scratchpads, events, and context processors. |
| Auditability emphasis | Context editing preserves client-side full history while clearing model-visible content. | Compaction may create opaque encrypted state; artifacts remain the human-reviewed source of truth in cookbook guidance. | Session/event model separates definitive state from rendered prompt. |
([Anthropic Engineering: Effective Context Engineering][1]; [Anthropic Docs: Context editing][19]; [OpenAI Docs: Compaction][8]; [OpenAI Agents SDK: Sessions][6]; [Google Gemini: Long context][13]; [Google ADK: Context-aware multi-agent systems][16]) Google Developers Blog+5Anthropic+5Claude Platform+5
The differences matter in implementation. Anthropic’s context editing is closer to selective deletion of low-value context. OpenAI’s compaction is closer to generating a compact continuation state. Google’s long-context and caching guidance encourages using large windows where appropriate, while still preserving filtering and caching as performance tools. None of these approaches removes the need for a context budget.
13. The context compiler pattern
A context compiler is a deterministic or semi-deterministic layer that assembles the per-turn prompt from durable state.
A typical compiler pipeline:
input: user request session state durable memory available tools retrieval indexes recent transcript task artifacts token and latency budgetpipeline: classify task select tool namespace retrieve evidence select relevant state trim or summarize history choose cacheable prefix assemble context count tokens run model capture tool calls and outputs update artifacts update memory log rendered context and outcome
This pattern is useful because it separates policy questions from runtime mechanics. The team can evaluate each stage independently:
| Compiler stage | Evaluation question |
|---|---|
| Task classification | Did the system identify the right task type and risk level? |
| Tool selection | Were only necessary tools exposed? |
| Retrieval | Did the evidence pack include the facts needed to answer correctly? |
| State selection | Did the model see active constraints and durable commitments? |
| History policy | Did trimming or summarization remove something necessary? |
| Cache layout | Was stable content reused efficiently? |
| Token accounting | Did the context fit with enough output reserve? |
| Post-run update | Were new facts written to the right artifact or memory store? |
The context compiler also creates observability. If an agent fails, one can inspect the rendered context and ask: Did the model fail despite seeing the right information, or did the system fail to show it the right information? That distinction is central. A model error may require prompt or model changes. A context-selection error requires retrieval, budgeting, memory, or state-management changes.
14. Failure modes specific to context engineering
Context engineering creates its own failure modes. They are not identical to classical prompt failures.
14.1 Context poisoning
Bad context can dominate good instructions. A stale summary, incorrect retrieved passage, or misleading tool result can steer the model away from the user’s actual request. This is especially dangerous when summaries convert uncertainty into declarative state.
Mitigation: tag source authority, preserve uncertainty, include timestamps, and make summaries cite their source artifacts where possible.
14.2 Instruction dilution
Long prompts can bury the most important constraints. Even if the model technically receives the instruction, it may not attend to it reliably. Anthropic’s “context rot” warning and OpenAI’s caution about uncurated long histories both support treating excessive context as a reliability risk rather than a harmless surplus. ([Anthropic Docs: Context windows][2]; [OpenAI Cookbook: Session memory][7]) Claude Platform
Mitigation: keep root instructions short, stable, and high-priority; move tool-specific rules into tool descriptions; avoid duplicative policy text.
14.3 Tool-schema crowding
Large tool catalogs can consume enough context that task evidence or conversation state gets squeezed out. Tool search and dynamic loading exist because tool surfaces themselves can become too large. ([Anthropic Docs: Advanced tool use][17]; [OpenAI Docs: Tool search][18]) Anthropic
Mitigation: expose namespaces, not every function; dynamically load details; evaluate whether examples improve tool accuracy enough to justify their tokens.
14.4 Retrieval overstuffing
Retrieval systems often return too much because it feels safer. In practice, extra chunks can introduce contradictions, stale facts, or irrelevant language that changes the model’s answer.
Mitigation: retrieve fewer, better passages; include source metadata and relevance rationales; let the model request more evidence when needed.
14.5 Summary drift
A summary may be locally plausible but globally wrong. Errors compound when future summaries summarize prior summaries rather than original sources.
Mitigation: summarize from source artifacts where possible; preserve exact IDs, quotes, and unresolved questions; periodically regenerate summaries from canonical state.
14.6 Opaque compaction
Server-side compaction can preserve continuity, but opaque artifacts make it harder for developers to inspect exactly what survived. OpenAI documents compaction items that carry state and reasoning while being opaque/encrypted in certain flows. ([OpenAI Docs: Compaction][8]) OpenAI Developers
Mitigation: pair compaction with explicit external artifacts for critical state; log milestones; preserve human-readable plans and decision records.
14.7 Cache invalidation by accidental prefix changes
A small change early in the prompt can reduce cache reuse, raising latency and cost. Anthropic and Google both emphasize prefix stability for caching. ([Anthropic Docs: Prompt caching][3]; [Google Gemini: Context caching][14]) Claude Platform
Mitigation: isolate dynamic content at the suffix; avoid injecting timestamps, user-specific values, or retrieval snippets into the stable prefix.
15. Design rules for agent-powered applications
Rule 1: Treat context as a scarce resource even when the window is large
Large windows change the feasible design space, but they do not eliminate attention, ranking, or cost constraints. Anthropic warns about degradation as context grows; Google says not to pass unneeded tokens; OpenAI warns that uncurated histories and tool results can overwhelm large windows. ([Anthropic Docs: Context windows][2]; [Google Gemini: Long context][13]; [OpenAI Cookbook: Session memory][7]) Claude Platform+2Google AI for Developers+2
Rule 2: Separate durable state from rendered context
The model-visible context should be a view over state, not the state itself. Use transcripts for audit, artifacts for current truth, and retrieval for selective re-entry.
Rule 3: Put stable content first and volatile content last
This supports caching and behavioral stability. Anthropic, OpenAI, and Google all recommend placing stable or common prefix content early for cache reuse. ([Anthropic Docs: Prompt caching][3]; [OpenAI Guide: GPT-5.5 prompting][9]; [Google Gemini: Context caching][14]) Claude Platform+2OpenAI Developers+2
Rule 4: Make tool exposure dynamic
Expose only the tool surface needed for the task. For large systems, use namespaces, tool search, or deferred schema loading. Treat every tool description and schema as part of the token budget.
Rule 5: Prefer handles over bulk
When a tool returns a large object, the agent often needs a handle, summary, or slice rather than the entire object. Anthropic’s just-in-time context guidance and programmatic tool-calling direction both support loading detailed data only when needed. ([Anthropic Engineering: Effective Context Engineering][1]; [Anthropic Docs: Advanced tool use][17]) Anthropic
Rule 6: Compress only with known invariants
Before trimming, summarizing, or compacting, define what must survive: user goals, exact IDs, unresolved uncertainties, tool outcomes, deadlines, assumptions, and next actions. OpenAI’s GPT-5.5 guide explicitly calls out preserving completed actions, assumptions, IDs, tool outcomes, blockers, and the next goal during compaction. ([OpenAI Guide: GPT-5.5 prompting][9]) OpenAI Developers
Rule 7: Keep human-auditable artifacts for high-stakes work
For code, legal research, data analysis, long-form writing, and operations, the final artifact or state file should be the source of truth, not the model’s remembered conversation. OpenAI’s memory-and-compaction cookbook makes this point directly by treating the final artifact as the human-reviewed source of truth. ([OpenAI Cookbook: Memory and compaction][21]) OpenAI Developers
Rule 8: Evaluate context policies, not just model outputs
A failure may come from the model, the prompt, retrieval, tool exposure, memory, compression, or cache layout. Log rendered context, token allocations, retrieved sources, exposed tools, compaction events, and final outcomes.
16. What is still thin or contested
The evidence for context engineering as an engineering discipline is strong in vendor documentation, but several claims remain under-standardized.
First, there is no universal context-budget formula. Vendor guidance supports accounting for token consumers, caching stable prefixes, and trimming or compressing history, but it does not establish general percentages for system instructions, tools, retrieval, and conversation history. Percentages should be treated as local configuration learned from evaluations, not as industry best practice.
Second, long-context performance is task-dependent. Google’s documentation supports long-context use cases, but also notes retrieval/cost tradeoffs and performance differences in complex retrieval settings. Anthropic warns about context rot, and OpenAI warns against uncurated long histories. The contested question is not whether long context is useful — it is — but when raw long context outperforms selective retrieval and when it merely hides curation failures. ([Google Gemini: Long context][13]; [Anthropic Docs: Context windows][2]; [OpenAI Cookbook: Session memory][7]) Google AI for Developers+2Claude Platform+2
Third, compaction quality is hard to audit. Summaries can be read but may be wrong; opaque compaction artifacts may work operationally but provide limited human inspectability. This is why externalized artifacts remain important even when automated compaction is available.
Fourth, vendor primitives are moving quickly. Anthropic’s context editing and memory tooling, OpenAI’s compaction and session memory, and Google’s caching and ADK context-processing patterns are active product areas. Current designs should isolate vendor-specific mechanisms behind an application-level context compiler so that caching, compaction, and state policies can change without rewriting the whole agent.
17. Reference implementation sketch
A production agent can implement context engineering as an explicit runtime module.
class ContextCompiler: def compile(self, request, state, budget): output_budget = budget.reserve_output_tokens() prefix = self.build_static_prefix( product_contract=state.product_contract, safety_rules=state.safety_rules, tool_namespaces=self.select_tool_namespaces(request, state), output_schema=state.output_schema, ) task_state = self.select_task_state( request=request, active_plan=state.active_plan, durable_memory=state.memory, artifacts=state.artifacts, ) evidence = self.retrieve_evidence( request=request, indexes=state.indexes, max_tokens=budget.retrieval_tokens, ) history = self.select_history( transcript=state.transcript, max_tokens=budget.history_tokens, summary=state.summary, ) context = self.assemble( prefix=prefix, current_request=request, task_state=task_state, evidence=evidence, history=history, output_budget=output_budget, ) return self.validate_and_count(context, budget)
The sketch is intentionally simple. The important feature is not the class; it is the separation of responsibilities. Tool selection, retrieval, state selection, history policy, and token accounting are independently testable. That is the operational difference between prompt engineering and context engineering.
18. References
[1] Anthropic Engineering: “Effective context engineering for AI agents.” Official Anthropic engineering article on just-in-time context, progressive disclosure, memory, subagents, and the shift from prompt engineering to context engineering. Anthropic+2Anthropic+2
[2] Anthropic Docs: “Context windows.” Official documentation defining context windows as working memory and warning about context rot. Claude Platform
[3] Anthropic Docs: “Prompt caching.” Official documentation on cache breakpoints, automatic caching, and stable prefix ordering. Claude Platform+2Claude Platform+2
[4] Anthropic Docs: “Tool use pricing and token accounting.” Official documentation describing tool definitions, schemas, tool-use blocks, and tool-result blocks as token contributors. Claude API Docs
[5] OpenAI Docs: “Conversation state.” Official documentation for Responses and Conversations API state management. OpenAI Developers
[6] OpenAI Agents SDK: “Sessions.” Official documentation for built-in session memory across agent runs. OpenAI GitHub
[7] OpenAI Cookbook: “Context engineering — session memory.” Official cookbook discussion of context limits, trimming, summarization, and session memory tradeoffs. OpenAI Developers+2OpenAI Developers+2
[8] OpenAI Docs: “Compaction.” Official documentation for server-side and standalone compaction. OpenAI Developers+2OpenAI Developers+2
[9] OpenAI Guide: “GPT-5.5 prompting.” Official guide discussing stable-first/dynamic-last prompt layout, prompt caching, hosted tools, tool search, and compaction-preservation guidance. OpenAI Developers
[10] OpenAI Blog: “Shell + Skills + Compaction.” Official OpenAI article on long-running agents, skills, shell execution, and compaction. OpenAI Developers
[11] OpenAI Docs: “Prompt caching.” Official documentation on cached tokens and prompt components eligible for caching. OpenAI Developers
[12] OpenAI Docs: “Function calling and tools.” Official documentation on function tools, JSON schemas, descriptions, and tool outputs. OpenAI Developers
[13] Google Gemini Docs: “Long context.” Official Gemini guidance on long context, short-term memory, RAG, filtering, summarization, caching, and query placement. Google AI for Developers+2Google AI for Developers+2
[14] Google Gemini Docs: “Context caching.” Official Gemini documentation on implicit and explicit context caching, prefix reuse, TTLs, and usage metadata. Google AI for Developers
[15] Google Vertex AI Docs: “Context cache overview.” Official Vertex AI documentation on context cache cost/latency reduction and prefix placement. Google Cloud Documentation
[16] Google ADK Blog: “Context-aware multi-agent systems.” Official Google ADK article on sessions, events, state, context processors, filtering, compaction, caching, and routing. Google Developers Blog
[17] Anthropic Docs: “Advanced tool use.” Official guidance on tool search, programmatic tool calling, examples, and context bloat from large tool surfaces. Anthropic
[18] OpenAI Docs: “Tool search.” Official documentation on dynamically loading tool definitions and reducing up-front tool-schema context. OpenAI Developers
[19] Anthropic Docs: “Context editing.” Official documentation on clearing old tool results, thinking blocks, token-count previews, and client-side history preservation. Claude Platform+2Claude Platform+2
[20] Anthropic Docs: “Claude Code memory.” Official documentation on MEMORY.md, on-demand topic files, and memory file loading. Claude API Docs
[21] OpenAI Cookbook: “Memory and compaction.” Official cookbook guidance distinguishing compaction, reusable memory, and final artifacts as human-reviewed sources of truth. OpenAI Developers
Companion entries
Core theory: Context Engineering, Prompt Engineering, Working Memory, Attention Budget, Context Rot, Agent Runtime, Stateful Agents, Long-Context Models
Architecture patterns: Static Prefix / Dynamic Suffix, Context Compiler, Context Budgeting, Just-in-Time Context, Progressive Disclosure, Externalized State, Transcript-Based Memory, Artifact-Centric Agents
Retrieval and memory: Retrieval-Augmented Generation, Retrieval as Context Curation, Vector Indexes, Session Memory, Durable Memory, Memory Summarization, Compaction, Context Editing
Tools and execution: Tool Schemas, Tool Search, Programmatic Tool Calling, Function Calling, Tool Result Management, Hosted Tools, Agent Tool Routing
Evaluation and reliability: Context Observability, Prompt Caching, Context Cache Invalidation, Summary Drift, Context Poisoning, Instruction Dilution, Agent Trace Analysis, Human-Auditable Artifacts