ReAct Agents
ReAct agents made “agentic AI” concrete by turning language-model inference into a loop: reason about the current state, choose an action, observe the result, and repeat. The core insight of ReAct was not that models should “think” in text, but that reasoning and environment interaction should be coupled tightly enough that each can correct the other. This article covers the original Yao et al. paper, the canonical Thought/Action/Observation algorithm, benchmark results, later variants, product-scale descendants such as OpenAI Operator and Anthropic Computer Use, and the unresolved question of whether explicit reasoning traces remain necessary as reasoning models internalize the loop.
Coverage note: verified through May 11, 2026.
Core claim
ReAct Agents are language-model agents that interleave verbal reasoning with actions against an external environment. In the original formulation, a trajectory is not simply:
Question -> Answer
or even:
Question -> Chain of thought -> Answer
but instead:
QuestionThought: identify what is known, unknown, or nextAction: call a tool / query an environment / take a stepObservation: receive the external resultThought: update the planAction: ...Observation: ...Final answer / terminal action
That loop became foundational because it solved a specific weakness of early Chain-of-Thought Prompting: ungrounded reasoning could produce plausible but false intermediate claims, and those claims could compound. ReAct gave the model a way to interrupt its own speculation with external evidence. At the same time, it solved a weakness of early tool-using or acting-only agents: tool calls without an explicit working memory tended to wander, repeat actions, or fail to decompose long-horizon goals. Yao et al. state the motivation directly: reasoning traces help induce, track, and update action plans, while actions let the model gather external information from sources such as knowledge bases or environments. arXiv
The important architectural move was to treat “thoughts” as part of the action space. In the paper’s formalization, an agent normally selects actions from an environment action set; ReAct augments that action space with language, where a language action is a “thought” that does not change the environment but helps the agent compose useful information over the current context. arXiv This makes ReAct a bridge between Reasoning Traces, Tool Use, and Closed-Loop Agents.
Background: why ReAct appeared when it did
Before ReAct, two lines of work were advancing quickly but mostly separately. The first was explicit reasoning prompting: Chain-of-Thought Prompting showed that large models could improve on multi-step reasoning tasks by emitting intermediate natural-language rationales. The second was language-model action generation: systems such as WebGPT, SayCan, and MRKL Systems showed that language models could select browser actions, robot skills, or external modules. WebGPT fine-tuned GPT-3 to answer long-form questions using a text-based browser, imitation learning, and human feedback; SayCan combined language-model action likelihoods with affordance/value estimates for robot skills; MRKL proposed modular systems combining language models with external knowledge and discrete reasoning modules. arXiv+2say-can.github.io+2
ReAct’s contribution was to make the alternation itself the unit of agency. It did not merely say “LLMs should call tools,” and it did not merely say “LLMs should think step by step.” It said that a language model should be prompted, or later trained, to produce a trajectory where reasoning, acting, and observing form one coherent context window. Yao et al. explicitly contrast ReAct with static chain-of-thought reasoning, which can be “not grounded in the external world,” and with action-generation approaches that do not use language reasoning to maintain high-level plans. arXiv
That distinction matters because an agent’s hardest failures are often state failures, not single-step reasoning failures. In a web task, the agent must remember what it searched, which page it chose, why it rejected alternatives, and whether the current observation satisfies the original instruction. In a household-simulation task, it must know whether it already picked up an object, whether a subgoal is complete, and what to inspect next. ReAct made those latent state updates textually inspectable.
The canonical ReAct loop
The canonical ReAct agent has four conceptual parts:
| Component | Role in the loop | Typical modern representation |
|---|---|---|
| Thought | Model-generated reasoning, plan update, uncertainty statement, or subgoal tracking | Hidden reasoning tokens, visible scratchpad, reasoning summary, or explicit message |
| Action | Tool call, browser click, API call, environment command, database query, or final-answer action | Function call, JSON tool call, GUI command, browser action, shell command |
| Observation | Result returned by tool or environment | Tool result, webpage text, screenshot, compiler output, database result |
| Controller | Harness that parses action, executes tool, appends observation, enforces limits | Agent runtime, graph executor, SDK loop, sandbox, browser controller |
A minimal ReAct Loop looks like this:
initialize context with user task, tool descriptions, and examplesfor step in 1..max_steps: model_output = LLM(context) if model_output contains Final Answer: return final answer parse Thought, Action from model_output if Action is invalid: observation = error message or repair prompt else: observation = execute(Action) append model_output and Observation to contextreturn failure / ask human / fallback policy
The original paper’s benchmark prompts usually used a visible format such as:
Thought: I need to find the entity mentioned in the question.Action: Search[entity]Observation: The entity page says ...Thought: Now I need to compare this with ...Action: Lookup[...]Observation: ...Thought: The evidence supports ...Action: Finish[answer]
For knowledge-intensive tasks, Yao et al. used a deliberately simple Wikipedia API with three actions: search[entity], lookup[string], and finish[answer]. The paper notes that this API was weaker than state-of-the-art retrievers; that weakness was intentional because the goal was to force explicit retrieval through reasoning rather than hide retrieval inside a powerful search component. arXiv
Modern implementations often keep the ReAct structure but remove the literal “Thought:” text. OpenAI’s Agents SDK, for example, describes an agent loop where the runner calls the model, inspects output, executes tool calls, appends results, handles handoffs, and stops when there is a final answer with no more tool work. OpenAI Developers Anthropic’s tool-use documentation similarly describes Claude deciding whether to use a tool, emitting a structured tool_use, the developer executing that tool and returning a tool_result, and the loop repeating until completion. Claude API Docs The syntax changed; the control loop largely did not.
Dense versus sparse reasoning
A subtle but important part of the original ReAct paper is that thoughts are not always emitted at the same density. Yao et al. used dense thought-action-observation steps for knowledge-intensive reasoning tasks such as HotpotQA and FEVER, where each retrieval step may change the answer. For interactive decision-making tasks such as ALFWorld and WebShop, they used sparser thoughts because action horizons can be long and emitting a full reasoning trace before every primitive action can be inefficient. arXiv
This distinction is still underappreciated. ReAct is not equivalent to “think before every tool call.” It is a family of loop designs with different reasoning granularities:
| Reasoning density | Best fit | Failure if overused | Failure if underused |
|---|---|---|---|
| Dense reasoning | Multi-hop QA, fact verification, debugging, policy-heavy actions | Token bloat, verbosity, rationalization | Ungrounded jumps, missed contradictions |
| Sparse reasoning | GUI navigation, shopping tasks, robotics, many-step workflows | State drift between thoughts | Repetitive low-level micromanagement |
| Hidden/internal reasoning | Production agents using reasoning models | Reduced auditability | Less controllable recovery |
| Structured summaries | Compliance logs, trace review, human oversight | Summary can omit decisive internal details | Harder debugging without enough state |
A production ReAct-style agent should not blindly expose or store every intermediate thought. It should decide which reasoning artifacts are useful for state tracking, which are useful for human oversight, and which should remain internal.
Empirical results that made ReAct foundational
ReAct became influential because it was tested across both knowledge tasks and interactive decision-making tasks. The paper’s four headline benchmarks were HotpotQA, FEVER, ALFWorld, and WebShop. That breadth mattered: it showed that the same loop could support retrieval-grounded question answering, fact verification, text-game planning, and web shopping. arXiv
HotpotQA and FEVER
HotpotQA is a multi-hop question-answering benchmark; FEVER is a fact-verification benchmark where claims are labeled as supported, refuted, or not enough information. In the ReAct setup, the model received only the question or claim and had to rely on internal knowledge or the Wikipedia API to retrieve evidence. arXiv
The main PaLM-540B prompting results were:
| Method | HotpotQA EM | FEVER accuracy |
|---|---|---|
| Standard prompting | 28.7 | 57.1 |
| Chain-of-thought | 29.4 | 56.3 |
| CoT self-consistency | 33.4 | 60.4 |
| Act-only | 25.7 | 58.9 |
| ReAct | 27.4 | 60.9 |
| CoT-SC → ReAct | 34.2 | 64.6 |
| ReAct → CoT-SC | 35.1 | 62.0 |
| Supervised SOTA cited by paper | 67.5 | 89.5 |
These numbers should not be misread as “ReAct always beats CoT.” It did not: on HotpotQA, plain ReAct slightly lagged plain CoT, while on FEVER it beat CoT. The more important result was that combinations of CoT self-consistency and ReAct performed best among the prompting methods, suggesting that internal reasoning and external grounding were complementary rather than mutually exclusive. arXiv
The failure analysis is more important than the headline accuracy. In a human-labeled sample of HotpotQA trajectories, ReAct had fewer hallucination-related failures than CoT, but more failures due to reasoning rigidity and search-result problems. The authors report that CoT hallucination made up 56% of CoT failure cases in the sample, while ReAct had a notable 23% “search result error” category where non-informative retrieval derailed the reasoning process. arXiv
That result is the core trade-off of Tool-Grounded Reasoning: external observations reduce hallucination only when the observations are relevant, correctly interpreted, and not adversarial. A bad observation can be worse than no observation because it enters the agent’s state as apparently authoritative evidence.
ALFWorld
ALFWorld is a text-based household environment aligned with ALFRED. Tasks require the agent to complete household goals by navigating rooms and manipulating objects through text actions. ReAct’s sparse thoughts were used to decompose goals, track subgoal completion, select next subgoals, and use commonsense to decide where objects were likely to be found. arXiv
The result was striking. On ALFWorld, the best ReAct trial achieved a 71% average success rate, compared with 45% for the best Act-only trial and 37% for the BUTLER imitation-learning baseline. The paper emphasizes that even the worst ReAct trial beat the best Act-only and BUTLER trials, and that ReAct’s advantage over Act was consistent across controlled trials. arXiv
The ALFWorld result made ReAct feel less like a QA prompting trick and more like a general agentic pattern. In a long-horizon environment, the reasoning trace served as a compact working memory: what is the goal, what has already been done, what object is needed, where might it be, and what should happen next?
WebShop
WebShop is an online-shopping environment with real-world product data and natural-language instructions. The agent must search, choose products, select options, and buy an item satisfying the instruction. In the ReAct paper, WebShop evaluation used average score and success rate over 500 test instructions. arXiv
The reported WebShop results were:
| Method | Score | Success rate |
|---|---|---|
| Act-only | 62.3 | 30.1 |
| ReAct | 66.6 | 40.0 |
| Imitation learning | 59.9 | 29.1 |
| Imitation + RL | 62.4 | 28.7 |
| Human expert | 82.1 | 59.6 |
ReAct improved success rate by about 10 percentage points over the previous best method in that table, but it remained far below human expert performance. The paper attributes part of the gap to humans performing more product exploration and query reformulation than prompt-based methods. arXiv
This is an early appearance of a limitation that persists in web agents: reasoning helps bridge noisy observations and user intent, but it does not automatically solve exploration. The agent still needs policies for search breadth, backtracking, stopping, uncertainty, and product comparison.
Why the Thought/Action/Observation triple worked
The Thought/Action/Observation format worked because it separated three things that are often confounded in ordinary chat completions:
Belief update: What does the agent currently think is true?
Intervention: What external action should it take?
Evidence: What did the world return?
That separation made trajectories inspectable. A human could see whether the model’s problem was bad reasoning, wrong tool choice, missing evidence, tool failure, or bad final synthesis. The ReAct paper explicitly argues that reasoning-action trajectories improve interpretability, trustworthiness, and diagnosability because humans can distinguish internal knowledge from external observations and inspect the basis for actions. arXiv
It also created a natural place to insert Human-in-the-Loop Control. A human can edit a thought, block an action, repair a tool call, or provide a missing observation. In modern systems, the same role is played by approvals, sandboxes, max-turn limits, tool schemas, policy classifiers, and trace viewers.
Influence on modern agent harnesses
Most modern agent harnesses inherit some version of ReAct even when they no longer call it that. The core pattern is: call model, let model request a tool or action, execute externally, return result, repeat until final output or stop condition.
LangChain’s older create_react_agent explicitly creates an agent using ReAct prompting and references the ReAct paper. LangGraph’s prebuilt create_react_agent is described as creating an agent graph that calls tools in a loop until a stopping condition is met. reference.langchain.com LangChain’s newer agent documentation generalizes this into graph-based runtimes with model nodes, tool nodes, middleware, and state. docs.langchain.com
LlamaIndex’s ReActAgent documentation presents a simple ReAct agent over calculator tools and emphasizes step-by-step reasoning over tool use; its agent API reference also notes that an AgentWorkflow uses a FunctionAgent when the model supports function calling and otherwise uses a ReActAgent. Developer Documentation Microsoft AutoGen’s documentation likewise lists ReAct alongside Reflection/Self-Critique as a supported prompting and reasoning strategy. microsoft.github.io
OpenAI’s Agents SDK gives a more contemporary version: agents are applications that plan, call tools, collaborate across specialists, and maintain enough state for multi-step work. Its runner loop calls the current agent’s model, inspects output, executes tool calls, handles handoffs, and returns only when the model produces a final answer with no more tool work. OpenAI Developers This is not the original textual ReAct prompt, but it is recognizably the same control skeleton.
Limitations surfaced by the field
ReAct’s strengths are also sources of failure. Once a model can call tools in a loop, the system inherits all the problems of planning under uncertainty, long-horizon search, partial observability, and side-effectful action.
Action explosion and token cost
A naive ReAct agent can turn a simple task into many calls. Each call appends more context, which increases cost and can dilute the relevant state. ReWOO’s authors criticize the standard interleaved reasoning-observation paradigm for “huge computation complexity” caused by redundant prompts and repeated execution, and propose decoupling reasoning from observations partly to reduce token consumption. OpenReview
Modern reasoning models add a second cost channel: hidden reasoning tokens. OpenAI’s reasoning documentation states that reasoning models use internal reasoning tokens before producing a response; those tokens are not visible through the API, but still occupy context and are billed as output tokens. OpenAI Developers The economic version of the ReAct problem is therefore: every extra step can cost prompt tokens, tool latency, hidden reasoning tokens, and sometimes money or side effects in the external system.
Cycles and repetitive behavior
ReAct can get stuck. The original paper reports a frequent ReAct-specific error pattern where the model repetitively generated prior thoughts and actions and failed to jump out of the loop. arXiv This is why modern harnesses almost always include max-turn limits, loop detectors, tool-call budgets, and escalation paths. OpenAI’s Agents SDK explicitly raises a MaxTurnsExceeded exception when a run exceeds its configured turn limit. OpenAI GitHub
Cycles are not merely annoying. In side-effectful environments, they can create repeated purchases, repeated emails, duplicated database writes, or excessive API usage unless actions are idempotent or guarded.
Reasoning drift
Visible reasoning can drift away from the task. The model may form an early hypothesis, then interpret later observations to support it. ReAct does not guarantee faithful reasoning; it only gives the model more opportunities to update. The ReAct paper itself notes that the structural constraint of interleaving reasoning, actions, and observations improved groundedness but reduced flexibility, contributing to higher reasoning-error rates than CoT in part of the HotpotQA analysis. arXiv
This is one reason production systems increasingly distinguish between internal reasoning, user-visible summaries, and durable execution state. The durable state should be facts, commitments, tool results, approvals, and constraints—not arbitrary self-talk.
Observation handling failures
Observation quality is decisive. Yao et al. found that non-informative search results accounted for 23% of ReAct error cases in their HotpotQA sample, and that such results derailed the model and made recovery difficult. arXiv Anthropic’s “think” tool guidance is essentially an operational response to the same issue: it recommends a dedicated thinking step when Claude needs to analyze tool outputs carefully in long chains of tool calls, policy-heavy environments, or sequential decisions where mistakes are costly. Anthropic
Observation handling is harder in GUI agents than in text APIs. OpenAI’s Operator system card reports that CUA was hindered by visual input and cursor output modalities, including OCR mistakes when copying complex strings and errors when editing code visually. OpenAI Anthropic’s computer-use research post similarly says Claude’s computer use remained slow and often error-prone, with actions such as dragging and zooming still challenging. Anthropic
Tool-description and routing failures
Tool descriptions are part of the agent’s world model. Anthropic’s engineering write-up on its multi-agent research system states that bad tool descriptions can send agents down the wrong path, so each tool needs a distinct purpose and clear description. Anthropic This is a ReAct limitation at the harness level: the model’s “Action” can only be as good as its understanding of available actions.
Prompt injection and unsafe side effects
ReAct agents read external observations and then act on them. That creates a prompt-injection channel: adversarial webpage text, document text, or UI content can be smuggled into the agent’s observation stream. OpenAI’s Operator system card identifies prompt injection as a risk where malicious instructions in third-party websites can mislead the model away from the user’s intended task, and describes mitigations such as refusals, confirmations, and monitoring. OpenAI Anthropic’s computer-use safety post likewise highlights prompt injection as a concern because Claude can interpret screenshots from internet-connected computers. Anthropic
The ReAct loop is therefore not just an algorithmic loop; it is a security boundary.
Modern variants
ReWOO: plan first, observe later
ReWOO stands for Reasoning WithOut Observation. It attacks a specific inefficiency in ReAct: if the model reasons, calls one tool, waits, reasons again with the entire growing prompt, calls another tool, and repeats, it may spend many tokens reprocessing old context. ReWOO decouples reasoning from observations: the model first produces a plan with tool calls represented as variables, workers execute those calls, and a solver integrates the results. OpenReview
The ReWOO paper reports 5× token efficiency and a 4% accuracy improvement on HotpotQA, along with robustness under tool-failure scenarios. It also explores offloading reasoning from a 175B GPT-3.5 model into a 7B LLaMA model through instruction fine-tuning. OpenReview
ReWOO’s deeper lesson is that ReAct is not always the optimal execution plan. If the next actions are foreseeable without intermediate observations, batching or preplanning can dominate stepwise interaction. If later observations are likely to change the plan, ReAct’s adaptivity is more valuable.
Reflexion: learning from verbal feedback
Reflexion adds memory and trial-level self-improvement to a ReAct-style agent. Instead of updating model weights through reinforcement learning, a Reflexion agent converts task feedback into natural-language reflections and stores those reflections in episodic memory for later attempts. arXiv
Its architecture separates Actor, Evaluator, and Self-Reflection components. The Actor generates actions, the Evaluator scores the trajectory, and the Self-Reflection model produces verbal reinforcement cues. Reflexion’s authors report improvements across ALFWorld, HotpotQA, and HumanEval, including a 91% pass@1 result on HumanEval and absolute improvements of 22% on ALFWorld and 20% on HotpotQA over their baselines. arXiv
Reflexion changes the ReAct time scale. ReAct is intra-episode adaptation: observe and update within one task attempt. Reflexion is inter-episode adaptation: fail, summarize why, and condition the next attempt on that memory.
LATS: search over action trajectories
Language Agent Tree Search generalizes ReAct from a single greedy trajectory to a search process. LATS combines language-model reasoning, acting, planning, value estimates, self-reflection, and environment feedback using a Monte Carlo Tree Search-inspired framework. OpenReview
The key limitation it addresses is that a single ReAct rollout can commit too early. If the first search query, click, or code-edit direction is wrong, the agent may never explore the better branch. LATS samples and evaluates alternative trajectories, using the model not just as an actor but also as a value function and optimizer. The OpenReview abstract reports experiments across programming, HotpotQA, and WebShop, with 94.4% on HumanEval using GPT-4 and an average score of 75.9 on WebShop. OpenReview
LATS shows that ReAct is a local policy; tree search wraps that policy in deliberation. The price is compute: multiple rollouts multiply model calls, tool calls, and evaluation complexity.
Toolformer and trained tool use
Toolformer is not a ReAct variant in the narrow prompt-format sense, but it is part of the same lineage. It trains models to decide which APIs to call, when to call them, what arguments to pass, and how to incorporate results into future token prediction using a self-supervised procedure with only a handful of demonstrations per API. arXiv
The Toolformer line points toward a future where the ReAct loop is not handwritten in prompts but internalized through training. The control question remains: even if the model learns when to call tools, the system still needs an execution boundary for actions, observations, state, and safety.
OpenAI Operator and Anthropic Computer Use as ReAct-style systems at scale
OpenAI Operator and Anthropic Computer Use are not merely “tool-calling demos.” They are ReAct-style agents scaled from text APIs to graphical user interfaces.
OpenAI introduced Operator in January 2025 as a research preview of an agent that could use its own browser by looking at webpages and interacting through typing, clicking, and scrolling. The page later noted that Operator was integrated into ChatGPT as ChatGPT agent in July 2025. OpenAI Operator is powered by OpenAI’s Computer-Using Agent, which combines GPT-4o vision capabilities with advanced reasoning through reinforcement learning and is trained to interact with GUIs through buttons, menus, and text fields. OpenAI
OpenAI reported that CUA achieved 38.1% on OSWorld, 58.1% on WebArena, and 87.0% on WebVoyager, using a universal screen/mouse/keyboard interface. OpenAI Structurally, this is ReAct over pixels: observe screenshot, reason about task state, choose GUI action, observe new screenshot, continue. The “Thought” may be hidden or summarized, but the loop remains.
Anthropic introduced computer use for Claude 3.5 Sonnet in public beta in October 2024, saying developers could direct Claude to use computers by looking at a screen, moving a cursor, clicking buttons, and typing text. Anthropic reported 14.9% on OSWorld in the screenshot-only category and 22.0% when afforded more steps. Anthropic Its API documentation describes the loop explicitly: provide Claude with the computer-use tool, Claude emits a tool-use request, the developer executes it in a container or VM and returns a tool result, and Claude continues calling tools until the task is complete. Claude API Docs
The relationship to ReAct is therefore structural rather than textual. Original ReAct used visible Thought, symbolic Action, and textual Observation. Operator and Computer Use use internal reasoning or thinking blocks, GUI actions, and screenshots/tool results. The abstraction is the same:
state estimate -> action -> environment feedback -> updated state estimate
Explicit reasoning versus internalized reasoning
The open question is whether ReAct’s explicit reasoning-action interleave remains useful as reasoning models internalize the loop.
Current reasoning models already blur the boundary. OpenAI’s reasoning-model documentation says models such as GPT-5.5 use internal reasoning tokens to plan, use tools, inspect alternatives, recover from ambiguity, and solve multi-step tasks. It also states that reasoning models support interleaved thinking: they can think between tool calls. OpenAI Developers Anthropic’s Claude documentation similarly describes interleaved thinking for Claude 4 models, allowing Claude to reason about tool results before deciding what to do next and chain multiple tool calls with reasoning steps in between. platform.claude.com
This weakens the case for visible Thought: text as a universal best practice. If a model can internally reason between tool calls, forcing it to emit a verbose natural-language scratchpad may waste tokens, expose misleading rationales, or create security and privacy issues. OpenAI’s docs state that raw reasoning tokens are not exposed through the API, though summaries can be requested; Anthropic’s docs distinguish thinking blocks, summaries, preservation, and redacted thinking. OpenAI Developers
But it does not weaken the case for the ReAct loop as a control structure. Tool calls still require an execution boundary. Observations still need to be returned to the model. Multi-step work still needs stopping criteria, state, memory, approvals, and failure recovery. The visible “Thought” may become optional; the interleaving of reasoning, acting, and observing remains central.
A reasonable current position is:
| Claim | Status |
|---|---|
| Visible “Thought:” traces are always necessary | False |
| Tool-call loops are still necessary for grounded agents | True |
| Hidden reasoning can replace all external state tracking | Not supported |
| Explicit summaries can help debugging and oversight | True, but summaries are not guaranteed faithful |
| The best granularity of reasoning-action interleave is task-dependent | Strongly supported by practice |
| Whether trained reasoning models eliminate hand-authored ReAct prompts | Open |
Evidence is thin on the strongest version of the question because frontier product agents combine many ingredients: model scale, RL, hidden reasoning, tool-use training, UI controllers, memory, browser sandboxes, safety classifiers, and product-specific heuristics. Public benchmark comparisons rarely isolate “explicit ReAct scratchpad” versus “internal reasoning with the same tools and controller.” The field has evidence that the loop matters; it has weaker evidence that visible natural-language thoughts remain the best representation of the loop.
Design guidance for production ReAct-style agents
A mature Agent Harness should treat ReAct as a control architecture, not a prompt template. The following principles generalize from the paper and later systems:
| Design issue | Recommendation |
|---|---|
| Tool granularity | Prefer actions that are meaningful but reversible; avoid huge tools that hide too much and tiny tools that cause action explosion |
| Observation format | Return concise, structured, task-relevant observations; do not dump irrelevant pages or logs |
| Reasoning visibility | Use hidden reasoning for model cognition, structured summaries for audit, and durable state for facts |
| Step limits | Enforce max turns, max tool calls, max cost, and domain-specific stop conditions |
| Recovery | Give the model explicit repair paths for invalid actions, empty results, and failed tool calls |
| Human approval | Require confirmation for irreversible, financial, privacy-sensitive, or external side-effect actions |
| Security | Treat observations as untrusted input; isolate browser/OS environments; block prompt-injection channels where possible |
| Memory | Store outcomes, evidence, constraints, and reflections separately from raw transcript text |
| Evaluation | Evaluate full trajectories, not just final answers; inspect failure modes by reasoning error, tool error, observation error, and stopping error |
The most common production mistake is to preserve the surface form of ReAct while ignoring the controller. A prompt that says “Thought, Action, Observation” is not an agent. An agent is the whole loop: model, parser, tools, environment, state, permissions, budgets, memory, trace, and stop policy.
Selected primary sources
| Source | Why it matters |
|---|---|
| Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models” | Introduced the reasoning-action-observation paradigm and evaluated it on HotpotQA, FEVER, ALFWorld, and WebShop. arXiv |
| Google Research ReAct blog | Public summary of the ReAct results, including ALFWorld and WebShop improvements. research.google |
| Xu et al., “ReWOO” | Proposed decoupling reasoning from observations to reduce redundant prompting and tool-call cost. OpenReview |
| Shinn et al., “Reflexion” | Added verbal reinforcement, episodic memory, and self-reflection to language agents. arXiv |
| Zhou et al., “Language Agent Tree Search” | Generalized ReAct-like acting into search over trajectories with environment feedback. OpenReview |
| OpenAI Computer-Using Agent / Operator docs | Shows ReAct-style observe-act loops scaled to browser and GUI use. OpenAI |
| Anthropic Computer Use and extended-thinking docs | Shows tool-use agent loops, computer interaction, and interleaved thinking between tool calls. Claude API Docs |
Companion entries
Core theory: ReAct Agents, Chain-of-Thought Prompting, Tool-Grounded Reasoning, Closed-Loop Agents, Partially Observable Environments, Reasoning Traces, Agent State, Agentic Control Loops
Algorithms and variants: Thought Action Observation, ReWOO, Reflexion, Language Agent Tree Search, Tree Search for Language Agents, Self-Reflection in LLM Agents, Plan-and-Execute Agents, Toolformer
Benchmarks: HotpotQA, FEVER, ALFWorld, WebShop, WebArena, WebVoyager, OSWorld, HumanEval, TAU-Bench
Implementation practice: Agent Harness, Tool Calling, Function Calling, Observation Design, Tool Schema Design, Agent Sandboxing, Human-in-the-Loop Approval, Agent Trace Evaluation, Max-Turn Limits, Prompt Injection in Agents
Product systems: OpenAI Operator, OpenAI Computer-Using Agent, OpenAI Agents SDK, Anthropic Computer Use, Claude Extended Thinking, LangGraph, LangChain Agents, LlamaIndex ReActAgent, Microsoft AutoGen
Limitations and counterarguments: Action Explosion, Reasoning Drift, Agent Loops, Observation Handling Failures, Prompt Injection, Hidden Chain of Thought, Reasoning Summaries, Internalized Tool Use, Agent Reliability