Agent Scaffolding
Agent scaffolding is the software layer that turns a foundation model into a system that can do bounded useful work: it supplies tools, memory, evaluators, routing, guardrails, and an execution loop. In 2025–2026, most practical “agent” capability is best understood as system capability System Capability vs Model Capability rather than raw model capability, but the same period also shows an opposite pressure: as base models improve, surprising amounts of scaffold become deletable, and what remains is increasingly infrastructure and governance rather than ornate cognitive choreography. (Anthropic)
Coverage note: verified through April 5, 2026.
Thesis
The strongest version of the scaffolding thesis is easy to caricature, so it is worth stating precisely. The claim is not that model quality stopped mattering. It is that by 2025–2026, once a team is already using a frontier-capable model, the difference between a demo and a dependable worker often comes from the surrounding software: tool contracts, context selection, state handling, evaluation loops, stopping conditions, permissions, and human handoff. OpenAI’s practical guide defines an agent as model + tools + instructions; Anthropic defines agentic systems as workflows or agents built from an augmented LLM; and the self-improving coding-agent literature is even blunter, describing agents as code that orchestrates LLM calls and provides tools. (OpenAI)
Anthropic’s infrastructure-noise study makes the same point from the evaluation side. In agentic coding benchmarks, the runtime is not a passive container but part of the problem-solving system, and Anthropic reports that infrastructure configuration alone shifted Terminal-Bench scores by 6 percentage points in internal experiments. That does not prove a universal law that scaffold dominates model, but it does show that “agent capability” is routinely measured as a property of whole systems, not naked weights. Evidence outside coding is thinner; most clean quantitative evidence still comes from software-engineering tasks, vendor deployment guidance, and a young self-improvement literature. (Anthropic)
This matters because the word agent was used loosely in 2024 and became more architecturally specific in 2025–2026. Anthropic draws a clean distinction between workflows—LLMs and tools arranged along predefined code paths—and agents—systems in which the model dynamically directs tool use and process flow. OpenAI’s guide lands nearby: agents are systems that independently accomplish tasks by managing workflow execution through a model, tools, and explicit instructions. On both views, agency is not a property of the base model alone. It is a property of the organized interaction between model and environment. (Anthropic)
What counts as scaffold
Agent scaffold Agent Scaffolding names the mutable software layer between a Foundation Model and the world it must act in. It includes the Agent-Computer Interface, tool registry, memory/state store, planner or router, evaluator or test harness, guardrails, permissioning, and the execution loop that keeps the system running until an exit condition is met. Anthropic’s production taxonomy starts from the augmented LLM and then adds workflows like prompt chaining, routing, orchestrator-workers, and evaluator-optimizer; OpenAI condenses the foundation into model, tools, and instructions, then adds orchestration and guardrails around a run loop. (Anthropic)
| Scaffold component | Core job | Failure if missing | Failure if overbuilt |
|---|---|---|---|
| Tool interface / Agent-Computer Interface | Expose actions and observations in model-usable form | The model is trapped in text or misuses the environment | Tool sprawl, context overload, brittle contracts |
| Memory / Agent Memory | Retain relevant facts, progress, and policies | Repetition, forgetting, no resumability | Stale state, hidden coupling, debugging opacity |
| Planner / decomposer Planning in Agents | Turn large tasks into workable subproblems | Flailing, premature action, bad sequencing | Latency, token tax, ceremony without lift |
| Router / handoff Routing and Handoffs | Dispatch cases to specialized prompts, tools, or agents | One-size-fits-none prompts | Misclassification, fragmentation, maintenance burden |
| Evaluator / Evaluation Harness | Verify outputs and decide retries | Silent failure, unjustified confidence | Expensive critique loops, duplicated competence |
| Guardrails / Guardrails | Constrain risk, scope, and side effects | Unsafe or irreversible actions | False positives, blocked workflows |
| Runtime substrate Stateful Agent Runtimes | Manage loops, retries, checkpoints, tracing, sandboxing | Non-reproducible runs, poor observability | Framework lock-in, abstraction tax |
Note: this taxonomy is a synthesis of Anthropic’s augmented-LLM/workflow breakdown, OpenAI’s model-tools-instructions foundation plus data/action/orchestration tool types, and runtime features exposed in LangGraph, CrewAI, Claude Code, and Codex documentation. (Anthropic)
The key point is that scaffolding is not synonymous with “framework.” A shell loop with good tools, repo-local instructions, tests, and a sandbox is already a scaffold. So is a production graph runtime with checkpoints and human-in-the-loop interrupts. The relevant distinction is not whether there is a scaffold, but where the system’s intelligence is externalized.
Why so much capability lived in the scaffold
Tool interfaces
Tool design is not an implementation detail. Anthropic’s 2025 tool-design note describes tools as a contract between deterministic systems and non-deterministic agents, which is exactly right: a bad tool contract is both a usability bug and a capability bottleneck. Its earlier agent guide says that while building a SWE-bench agent, Anthropic spent more time optimizing tools than the overall prompt. The 2024 SWE-agent paper says much the same thing empirically: a custom ACI significantly improved the agent’s ability to create and edit code files, navigate repositories, and run tests, reaching 12.5% pass@1 on SWE-bench at the time. (Anthropic)
Tool count and tool shape matter too. Anthropic’s MCP code-execution note argues that directly loading large tool catalogs into context becomes expensive and slow; in its example, switching to code-mediated, on-demand tool access cut token use from roughly 150,000 to 2,000 and allowed progressive disclosure of tool definitions. OpenAI’s guide makes the design space explicit by distinguishing data, action, and orchestration tools, which is useful because those categories stress the runtime differently: retrieval tools expand context, action tools create side effects, and orchestration tools compound control logic. (Anthropic)
That is why tool interfaces are often the first place agent capability shows up in practice. A base model may know what a CRM is, what a repo is, or what a spreadsheet is. But unless the scaffold maps those abstractions into callable, documented, state-aware operations, the model cannot reliably do the relevant work.
Memory and state
Memory is actually three different problems: working state, durable task state, and policy memory. OpenAI’s Responses API is stateful by default, which means conversation state can persist without client-side reconstruction; LangGraph persists checkpoints so a thread can be restored later; CrewAI Flows add structured state management and persistence; and CrewAI Crews expose short-term, long-term, and entity memory. These are not glamorous features, but without them an agent cannot pause, resume, audit, or hand off work cleanly. (OpenAI Developers)
Minimal coding agents still use memory, just in thinner forms. Claude Code supports project instructions through CLAUDE.md, builds auto memory while it works, and exposes custom commands and hooks; Codex loads AGENTS.md automatically, supports layered global/repo/local instruction files, and uses config as durable behavioral memory. In both cases the system is “minimal” only relative to graph-heavy orchestration. It is not memoryless. (Claude)
This is a useful correction to a common simplification. Retrieval is only one kind of memory. In production agents, much of the real leverage comes from remembered progress and remembered constraints: where the run left off, what has already been tried, what “done” means, which commands are safe, which build/test procedure is canonical, and which files or tools matter. Those are classic software concerns, not pure language-model concerns.
Planners, routers, evaluators
Planning and evaluation are the most visibly “agentic” parts of the scaffold. Anthropic’s production taxonomy names prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer as common patterns; LangGraph teaches almost the same pattern set in its workflows docs. The overlap matters. These are no longer quirky research inventions. They are standard orchestration motifs recognized across vendors and frameworks. (Anthropic)
These layers can move real performance. Anthropic reports that giving Claude testing tools dramatically improved performance on long-running coding tasks, and its March 2026 harness writeup says flatly that harness design is key at the frontier of agentic coding. OpenAI’s guide, meanwhile, emphasizes that every orchestration strategy ultimately comes down to a run loop with exit conditions, whether that loop belongs to one agent or several. In other words: planning and evaluation do not float above the runtime; they are implemented by it. (Anthropic)
But planners, routers, and evaluators are also the most pruneable parts of the scaffold. Anthropic’s 2026 harness note shows exactly why: as Opus 4.6 improved, some decomposition and evaluation stages that had been load-bearing for 4.5 became unnecessary overhead. The same mechanism that once justified more scaffold later justifies deleting it. That tension is the central design fact of the period. (Anthropic)
Guardrails and permissions
Guardrails are not optional polish. OpenAI recommends a layered defense that can combine relevance and safety classifiers, PII filters, moderation, tool safeguards, rules-based protections, and output validation, with human intervention triggered by repeated failures or high-risk actions. Anthropic’s own examples treat guardrail screening as a distinct workload that can benefit from parallelization rather than being naively folded into a single prompt. (OpenAI)
Minimal runtimes still encode this layer heavily. Claude Code’s auto mode runs commands behind a classifier and supports sandboxing and allowlists; Codex ships with OS-level sandboxing, approval policies, workspace-write modes, and network-off defaults unless explicitly enabled. This is scaffolding in the strongest sense: not a reasoning aid, but software governance. It survives even if the base model gets much better. (Claude)
Convergent vendor guidance: simple first, then add pieces
By late 2024 the major vendors were already converging toward architectural minimalism. Anthropic says its most successful customer implementations use simple, composable patterns rather than complex frameworks, recommends finding the simplest solution possible (often not an agent at all), distinguishes workflows from agents, and warns that frameworks can obscure prompts and responses while tempting needless complexity. Its summary rule is plain: add complexity only when it demonstrably improves outcomes, and reduce abstraction layers as you move to production. (Anthropic)
OpenAI’s official guidance lands in almost the same place. An agent is model + tools + instructions; orchestration should usually start as a single-agent loop; a single agent can handle many tasks by incrementally adding tools; and the general recommendation is to maximize one agent’s capabilities before introducing multiple agents. Even OpenAI’s developer track frames the Agents SDK as another layer on top of core APIs whose convenience comes with less fine-grained control. (OpenAI)
This is not an anti-framework consensus. It is an anti-premature-architecture consensus. Both Anthropic and OpenAI are saying that the baseline should look more like an augmented LLM in a loop—with well-chosen tools, explicit tests, and clear constraints—than like a committee of specialized agents connected by elaborate ceremony. That is an inference from the guides rather than a line either vendor states verbatim, but it is the obvious design center of both documents. (Anthropic)
The pruning principle
A useful analytic label for this period is the Pruning Principle: every scaffold layer must periodically re-justify itself against the current base model. If a new model can absorb a decomposition, routing decision, or critique loop without measurable loss, the layer should be removed rather than ritualized.
Anthropic’s March 2026 harness writeup is the clearest primary-source expression of this logic. After Opus 4.6 improved planning, long-horizon coherence, debugging, and long-context retrieval, Anthropic revisited a more complex harness built for 4.5. Sprint decomposition was removed; evaluator use became more selective; and the author explicitly recommends stripping away pieces that are no longer load-bearing when a new model lands. OpenAI’s single-agent-first advice and Anthropic’s “complexity only when it helps” rule amount to the same principle from different angles. (Anthropic)
Codex’s subagent guidance adds an operating-cost model to the same idea: subagents are not spawned automatically, and they consume more tokens than comparable single-agent runs. Complexity is not free even when it works. Mature scaffold design therefore has two motions, not one: add the right layer when the model genuinely needs help, and delete it when the model no longer does. (OpenAI Developers)
The pruning principle is important because teams naturally accumulate scaffold faster than they remove it. A planner is added for weak long-horizon performance, then survives after the model learns to plan better. A critique loop is added for bad first drafts, then survives after self-review improves. A router is added for mixed workloads, then survives after a larger context window and better instructions make the split unnecessary. The result is architectural sediment: layers whose historical reason has vanished but whose maintenance cost remains.
SWE-bench and the case for minimal scaffolds
SWE-bench is the cleanest public case study because it exposes both scaffold-rich and scaffold-thin regimes inside the same evaluation family. SWE-bench Verified is a human-validated subset of 500 instances, and its full leaderboard includes everything from simple LM agent loops to retrieval-heavy and multi-rollout/review systems. That breadth matters: the benchmark is already evaluating system design, not just model weights. (SWE-bench)
But the most revealing part of the site may be the bash-only comparison. For apples-to-apples model comparison, SWE-bench evaluates language models with mini-SWE-agent in a minimal bash environment: no extra tools, no special scaffold structure, just a simple ReAct loop. The site says outright that these results represent state-of-the-art LM performance when given only a bash shell and a problem. (SWE-bench)
That result should not be caricatured. “Minimal” does not mean “no scaffold”: bash itself is an expressive interface, and the harness still provides a loop, history, sandboxability, and exit logic. But it does mean that many layers once treated as essential can disappear. The SWE-bench site highlighted in July 2025 that mini-SWE-agent reached 65% on Verified in about 100 lines of Python; the current repository advertises above 74% on Verified, while emphasizing bash-only action, linear history, subprocess isolation, and simple control flow. The benchmark docs also warn that different mini-SWE-agent releases are not necessarily comparable, so the exact number is less important than the qualitative trend. (SWE-bench)
Set against the 2024 SWE-agent paper, the change is striking. In 2024, a custom ACI was itself the breakthrough, materially improving repo navigation, editing, and test execution. By 2025–2026, mini-SWE-agent’s own authors were explicitly arguing that much of the earlier interface machinery was no longer needed to build a useful agent once models became stronger. The lesson is not that ACI work was misguided. The lesson is that scaffold value is model-relative. (arXiv)
There is also a counter-reading worth taking seriously. One could look at mini-SWE-agent and conclude that the model is the whole story after all. But that reading confuses thin scaffold with no scaffold. The bash interface, message history, subprocess execution model, sandboxability, repo environment, and evaluation harness are all software decisions. The right inference is not that scaffolding disappears; it is that the scaffold can become radically thinner when the base model is good enough. (GitHub)
Benchmark evidence here still needs discipline. Anthropic’s infrastructure-noise study shows that resource configuration alone can move agentic coding scores by more than the margins that separate leaderboard positions, and that tight budgets favor lean strategies while generous budgets reward heavyweight tooling. So “minimal scaffolds win” is not a timeless law. It is a regime-dependent claim about strong models under particular task and resource distributions. (Anthropic)
Framework-heavy versus minimal-scaffold systems
The practical ecosystem split is therefore not between “frameworks” and “no frameworks.” It is between thicker orchestration substrates and thinner runtimes. Even the thin runtimes are still scaffolds; they simply keep more structure in the operating environment and less in a bespoke orchestration layer.
| System | Design center | Built-in scaffold | Hidden cost | Best fit |
|---|---|---|---|---|
| LangGraph | Long-running, stateful workflow/agent runtime | Durable execution, human-in-the-loop control, memory, checkpointing, debugging, deployment | Graph sprawl and engineering overhead if used where a loop would suffice | Production workflows where persistence, auditability, and explicit control matter |
| CrewAI | Batteries-included multi-agent and flow platform | Guardrails, memory, knowledge, observability, structured flows, hierarchical crews, persistence | More ceremony, more manager/process overhead, more architectural surface area | Teams that want explicit multi-agent collaboration and packaged operational features |
| Claude Code | Thin agentic coding environment | Codebase access, command execution, MCP, CLAUDE.md, auto memory, hooks, optional agent teams, permission controls |
Leans heavily on base-model quality and the coding environment; less explicit for general business workflows | Interactive and long-running coding work with human supervision nearby |
| Codex CLI | Thin local coding runtime | Local repo access, planning, edits, commands, AGENTS.md, config layers, MCP, sandbox/approvals, exec, optional subagents |
Narrower domain, token overhead for parallel subagents, less built-in orchestration ceremony | Local repos, CI, scriptable automation, and evaluation-friendly coding loops |
Note: these characterizations follow the docs. LangGraph describes itself as low-level infrastructure for long-running, stateful workflows and agents with durable execution, human-in-the-loop, memory, and debugging; CrewAI markets guardrails, memory, knowledge, and observability “baked in,” with structured flows and hierarchical crews; Claude Code centers an agentic coding loop with MCP, CLAUDE.md, auto memory, hooks, and optional agent teams; Codex CLI centers local repo work, AGENTS.md, MCP, sandbox/approval controls, scripted execution, and explicit subagent activation. (LangChain Docs)
LangGraph and CrewAI are attractive when long-running state, explicit branching, auditability, and multi-actor coordination are part of the product. Claude Code and Codex CLI become attractive when the task environment is already richly expressive—filesystem, shell, tests, git, MCP—and a strong model can do most of the cognitive work without much additional ceremony. The dividing line is not sophistication. It is where you want the complexity to live. (LangChain Docs)
Claude Code and Codex are especially instructive because they show what “minimal scaffold” looks like in mature products. Claude Code explicitly says it follows the Unix philosophy, can plan/write/verify across multiple files, and can still escalate to optional agent teams. Codex similarly runs locally, crafts a plan, supports AGENTS.md and MCP, can be used non-interactively via codex exec, and keeps subagent workflows explicit because they cost more. These are not framework-free systems. They are thin, composable runtimes that assume a powerful base model and a powerful operating environment. (Claude)
Scaffold as mutable substrate
The self-improving-systems literature makes the strongest case that scaffolds are substantive software objects rather than disposable wrappers. A Self-Improving Coding Agent defines agent systems directly as code that orchestrates LLM calls and tools, then shows that such a system can edit itself and improve from 17% to 53% on a random subset of SWE-bench Verified. That only makes sense if the scaffold itself is part of the capability object. (arXiv)
Darwin Gödel Machine generalizes that idea: fixed human-designed architectures cannot continuously improve themselves, so the system searches over modifications to its own code. Its reported gains on SWE-bench—from 20% to 50%—came partly from scaffold changes such as better code-editing tools, long-context management, and peer-review mechanisms. In other words, performance improved not just by giving the same model more chances, but by changing the machinery around the model. (arXiv)
Live-SWE-agent makes the point even more explicitly. It argues that manually designing an optimal software-agent scaffold is extremely difficult because the design space is effectively unbounded, starts from a bash-only minimal agent, and lets the agent synthesize tools during the run. The paper reports 75.4% on SWE-bench Verified and 45.8% on SWE-Bench Pro without test-time scaling. Those are recent research claims rather than settled consensus, but the architectural message is clear: one path beyond manual scaffold engineering is to let the scaffold change itself online. (arXiv)
AgentBreeder extends the same logic into safety. Its premise is that scaffolding models into multi-agent systems often improves performance on complex tasks, but also changes safety behavior; its evolutionary search over scaffolds reports large safety gains in one mode and adversarially weak scaffolds in another. If this line of work continues to hold up, scaffold design will not be mere deployment glue. It will be part of the capability-and-safety object we are actually optimizing. (OpenReview)
This is the deepest reason the topic matters. In the self-improving regime, the interesting architectural unit is no longer “model versus wrapper.” It is the search space over wrappers Mutable Scaffolds, constrained by evaluations, costs, and safety boundaries. That is a different engineering worldview from the one implied by static prompting alone.
Permanent layer or temporary compensation?
The right answer is two-layered. Some scaffolding is permanent because it externalizes concerns no model can own by itself: environment integration, side effects, security boundaries, auditing, permissions, and process state. Other scaffolding is temporary because it compensates for contingent model weaknesses: fragile long-horizon coherence, poor self-evaluation, clumsy tool use, or weak decomposition. Model progress mostly prunes the second category. (Anthropic)
| Likely permanent | Why | Likely to thin or disappear | Why |
|---|---|---|---|
| Tool and data integration | External systems still need explicit interfaces and contracts | Hand-authored task decomposition stacks | Better models internalize more planning and sequencing |
| Durable state, checkpoints, resumability | Operational continuity is a software concern | Per-step evaluator loops on moderate tasks | Stronger models self-review better; critique becomes selective |
| Guardrails, permissions, human handoff | Governance and risk constraints are external to the model | Manager/worker structures for modest workloads | One strong agent often suffices until genuine overload appears |
| Test harnesses and ground-truth validation | External correctness signals remain indispensable | Large static tool catalogs in prompt context | On-demand tool loading and code-mediated access scale better |
Note: this permanent/transient split is a synthesis of vendor docs and the coding-agent literature, especially Anthropic’s harness note, the SWE-agent/mini-SWE-agent contrast, and runtime/security documentation from Claude Code and Codex. (Anthropic)
The best synthesis from current evidence is therefore this: scaffolding is a permanent architectural layer, but not in its current thickness. The durable core is infrastructural—tools, state, evaluation, guardrails, permissions, observability. The most perishable part is hand-authored cognitive choreography. As models improve, agent engineering becomes less about inventing elaborate prompt societies and more about deciding which external capabilities truly need software embodiment. (Anthropic)
Design consequences
For AI engineers, the immediate practical consequence is methodological. Start with a strong single-agent baseline. Invest early in tool contracts, state management, tests, and permissioning. Add routing, evaluators, or multi-agent structures only when they produce measurable lift on the task distribution that actually matters. Then, when a new model arrives, re-audit the harness and prune anything that has turned into ornamental overhead. Anthropic, OpenAI, Claude Code, Codex, and even mini-SWE-agent all converge on this operational attitude despite their very different product surfaces. (OpenAI)
The second consequence is epistemic. If the unit of capability is the scaffolded system, then evaluation must control runtime conditions, document resources, and measure end-to-end behavior rather than isolated completions. Anthropic’s infrastructure-noise work makes that unavoidable. Agent engineering is moving closer to systems engineering, and its standards of evidence have to move with it. (Anthropic)
The philosophical consequence is smaller but important. The interesting question is no longer merely “what does the model know?” It is “what can the organized system stably do?” In 2025–2026, useful agency emerged less from a mysterious new faculty inside the model than from disciplined choices about interfaces, state, evaluation, and control. The surprise is that the better models become, the more of that discipline migrates away from ornate cognition scripts and toward clean runtime design.
Selected primary sources
- Anthropic Engineering — Building effective agents. Primary guidance on augmented LLMs, workflow taxonomy, simple composable patterns, ACI quality, and the warning against unnecessary abstraction. (Anthropic)
- OpenAI — A practical guide to building agents. Primary guidance on model/tools/instructions, single-agent-first architecture, guardrails, and human intervention. (OpenAI)
- Anthropic Engineering — Writing effective tools for agents; Code execution with MCP; Harness design for long-running application development; Quantifying infrastructure noise in agentic coding evals. Best primary-source cluster on tool contracts, context-efficient tool use, harness/model tradeoffs, and the fact that agentic evals are end-to-end system tests. (Anthropic)
- SWE-bench — Verified leaderboard and bash-only setup. Best public evidence that strong coding performance can emerge from very small scaffolds, with important caveats about comparability. (SWE-bench)
- SWE-agent paper; mini-SWE-agent repository. The clearest before/after contrast between interface-heavy early scaffolds and later minimal bash-first scaffolds. (arXiv)
- LangGraph, CrewAI, Claude Code, Codex docs. Useful primary docs for comparing thick orchestration substrates with thin local runtimes. (LangChain Docs)
- Self-improving agent literature — A Self-Improving Coding Agent, Darwin Gödel Machine, Live-SWE-agent, AgentBreeder. Best evidence that scaffold is not just glue but a mutable optimization target. (arXiv)
Companion entries
Core theory: Agent Scaffolding, System Capability vs Model Capability, Agent-Computer Interface, Augmented LLM, Pruning Principle, Mutable Scaffolds
Practice: Tool Design for Agents, Context Engineering, Evaluation Harness, Guardrails, Stateful Agent Runtimes, Single-Agent vs Multi-Agent, Long-Running Agents
Systems and frameworks: LangGraph, CrewAI, Claude Code, Codex CLI, Model Context Protocol
Benchmarks and case studies: SWE-bench, mini-SWE-agent, SWE-agent, Terminal-Bench
Counterarguments and open questions: Self-Improving Agents, Darwin Gödel Machine, Live-SWE-agent, AgentBreeder, The Case Against Agent Frameworks