Language Agent Tree Search

Language Agent Tree Search (LATS) is an inference-time framework that treats an LLM agent’s partial trajectories as search states and applies Monte Carlo Tree Search over possible next thoughts, actions, observations, and answers. Its central claim is that an LLM can serve simultaneously as policy, value estimator, and reflection module, while external environments provide grounded feedback that pure language-only deliberation often lacks.

Coverage note: verified through May 11, 2026.

1. Core idea

Language Agent Tree Search was introduced by Zhou et al. as “Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models”, published at ICML 2024 in PMLR. The paper proposes a search procedure for LLM Agents in which the model samples candidate actions, evaluates partial trajectories, receives environment feedback, and reflects on failed attempts, all inside a Monte Carlo tree search loop. The final PMLR abstract reports headline results of 92.7% pass@1 on HumanEval with GPT-4 and 75.9 average score on WebShop with GPT-3.5, with HotpotQA results showing especially large gains when reasoning and acting are combined. Proceedings of Machine Learning Research

The important shift is from search over answers to search over agent trajectories. In a normal single-pass agent, the LLM emits a sequence such as thought → action → observation → thought → action → final answer. In LATS, each partial trajectory becomes a node in a tree; the algorithm can branch, revisit, score, abandon, or refine trajectories before committing. That makes LATS part of a broader Agent Trajectory Search family: inference-time systems that spend extra compute to explore multiple possible courses of action rather than trusting the first decoded path.

A compact way to state the contribution is:

LATS wraps an LLM agent in MCTS, using the LLM as policy, value function, and self-reflection generator, while using real environment feedback as part of the search state.

This matters because many agent failures are not merely “bad final answers.” They are bad intermediate choices: searching the wrong query, selecting the wrong product, writing code before understanding the test, or prematurely finishing a reasoning chain. LATS attacks that failure mode by making intermediate choices searchable.

2. Lineage: CoT, ReAct, Reflexion, ToT, RAP

LATS is best understood as a synthesis of several earlier lines: Chain of Thought, ReAct Agents, Reflexion, Tree of Thoughts, and MCTS-style planning methods such as RAP. Zhou et al. explicitly position LATS against methods that either reason in a single trajectory, act without deep lookahead, or search using an internal world model rather than real environment feedback. arXiv

Method	Search object	Feedback source	Evaluator	Reflection / memory	Main limitation relative to LATS
Chain of Thought	One reasoning trace	Usually none	Implicit next-token likelihood	No	No branching or grounded feedback
ReAct	One thought/action trajectory	External observations	Mostly implicit	No	Can recover from observations, but does not systematically search alternatives
Reflexion	Repeated full attempts	Environment / task feedback	LLM-generated reflection	Yes	Reflection across trials, not tree search over partial trajectories
Tree of Thoughts	Intermediate “thought” states	Usually internal task feedback or LM scoring	LLM value/vote	Limited	Searches reasoning states, but not full agent-environment trajectories
RAP	Reasoning/planning states	LLM-simulated world model	MCTS reward/value	Limited	Relies heavily on model-internal simulation of the world
LATS	Agent trajectories: thoughts, actions, observations	Real environment feedback plus LM evaluation	LM value + self-consistency + terminal reward	Yes	Higher inference cost; depends on resettable/searchable environments

Tree of Thoughts is the most direct conceptual predecessor. ToT generalizes chain-of-thought prompting by letting an LLM generate intermediate “thoughts,” evaluate them, and use search procedures such as breadth-first or depth-first search to explore alternatives. Its original paper reports a dramatic improvement on the Game of 24 benchmark, where GPT-4 chain-of-thought solved 4% of tasks while ToT solved 74%, illustrating the value of structured search when local reasoning errors are common. OpenReview

ReAct supplies the acting substrate. ReAct interleaves reasoning traces with task-specific actions, letting the model query external tools or environments and then incorporate observations. This reduces some hallucination and error-propagation problems because the model does not need to invent all intermediate facts internally. arXiv

Reflexion supplies the self-improvement substrate. Reflexion-style agents use verbal feedback from prior attempts as memory for later attempts, without changing model weights; the method is especially relevant because LATS also stores natural-language reflections after failed terminal trajectories. arXiv

RAP supplies the closest MCTS analogy before LATS. In RAP, the LLM acts both as agent and world model, using Monte Carlo tree search to reason about possible future states. LATS preserves the MCTS spirit but shifts the environment component outward: rather than asking the LLM to simulate the world, it queries the real task environment when possible. arXiv

3. What LATS searches

A LATS node represents a partial agent trajectory. In Zhou et al.’s formulation, each node’s state contains the original input, the accumulated action sequence, and the accumulated observation sequence. Actions may be natural-language thoughts, tool calls, web-search operations, code solutions, product-selection steps, or terminal finish actions, depending on the task. arXiv

This makes LATS different from a generic “generate N answers and pick one” method. The tree is not merely over final answers. It is over prefixes of behavior:

root: task input  node A: thought_1, action_1, observation_1    node A1: thought_2, action_2, observation_2    node A2: alternate thought_2, alternate action_2, observation_2'  node B: alternate thought_1, alternate action_1, observation_1'    ...

The agent can therefore discover that a bad early search query, bad tool choice, or bad decomposition is leading nowhere before the final answer is generated. This is the key benefit of tree search over trajectories: it can allocate more compute to promising partial behaviors and stop investing in unpromising ones.

The paper’s implementation uses the LLM in three roles:

LLM role	What it does in LATS	Analogy
Policy module	Samples candidate next actions from a state	Policy network in search-guided RL
Value module	Scores partial trajectories, often with self-consistency-style support	Value function / heuristic evaluator
Reflection module	Produces verbal lessons from failed terminal trajectories	Episodic memory / verbal reinforcement

The environment supplies observations and, when available, terminal rewards. For example, HumanEval can provide compiler or unit-test feedback; WebShop can provide page observations and purchase-score outcomes; HotpotQA can use search observations and answer correctness feedback in the experimental setup. arXiv+2arXiv+2

4. Algorithmic structure

Zhou et al. describe LATS as an MCTS process with selection, expansion, evaluation, simulation, backpropagation, and reflection. The paper emphasizes that the language model is not merely a generator: it also evaluates states and generates reflections, while the environment returns external feedback after actions. arXiv

4.1 Selection

During selection, LATS descends from the root through already-expanded nodes using an upper-confidence-style criterion. The usual MCTS intuition applies: prefer nodes with high estimated value, but also give underexplored nodes a bonus so the tree does not collapse too early onto a locally plausible trajectory.

A generic UCT-style score has the form:

score(child) = value(child) + exploration_weight × sqrt(log visits(parent) / visits(child))

The exact implementation details vary by task and hyperparameter setting, but the role is stable: selection chooses which partial trajectory deserves more simulation budget.

4.2 Expansion

Expansion samples candidate next actions from the LLM policy. In a web or QA task, those actions may be search, lookup, or finish commands; in programming, they may be candidate programs or edits; in a shopping task, they may be search queries, product-page clicks, or attribute choices. LATS then executes these actions in the environment when appropriate and attaches the resulting observations to new child nodes. arXiv

This is where LATS departs from purely language-internal deliberation. The search tree is not built only out of guessed thoughts; it is grounded, at least partially, in actual tool or environment responses.

4.3 Evaluation

Evaluation estimates the value of a newly expanded state. LATS uses an LLM-powered state evaluator and self-consistency-style evidence rather than relying only on terminal reward. This is necessary because many useful partial states are non-terminal: a promising search result, a partially correct proof, or a code solution that has not yet been fully tested needs a heuristic score. arXiv

This is also one of the central weak points of the method. LLM self-evaluation is not automatically reliable. Work on self-correction has found that LLMs often struggle to correct their own reasoning without external feedback and can even degrade initially correct answers. LATS partly mitigates this by incorporating environment observations and terminal rewards, but the learned or prompted evaluator remains a bottleneck. OpenReview

4.4 Simulation

Simulation continues a trajectory until a terminal condition or depth limit. In classic MCTS, simulation can mean rollout to the end of a game. In LATS, it may mean continuing an agent trajectory until a final answer, a code submission, a purchase decision, or a maximum step count. arXiv

The “simulation” term can be slightly misleading in LATS because the environment need not be simulated by the model. In tasks like WebShop or HotpotQA, the agent can actually interact with the environment or API. That is one reason LATS is closer to tool-grounded planning than to purely imagined rollouts.

4.5 Backpropagation

After evaluation or terminal reward, LATS backpropagates value estimates up the tree. Parent nodes update their visit counts and value statistics based on child outcomes, allowing selection to become increasingly informed over iterations. This is the MCTS mechanism that turns scattered trials into a progressively biased search. arXiv

4.6 Reflection

Reflection is LATS’s most distinct addition relative to ordinary MCTS. When a trajectory fails, the LLM generates a verbal reflection on what went wrong and how future trajectories should differ. This reflection is stored in memory and injected into later prompts for policy generation and value estimation. arXiv

Reflection is not a learned weight update. It is prompt-level memory. That makes it cheap to implement, but also brittle: the quality of the reflection depends on whether the model can accurately diagnose failure from the available evidence. For code tasks with unit tests, this can be useful. For ambiguous open-world tasks without reliable feedback, reflection can become rationalization.

5. Minimal pseudocode

A simplified LATS loop looks like this:

initialize tree with root stateinitialize reflection memory M = []for iteration in search_budget:    node = select(root)                         # UCT-style descent    children = expand(node, policy_llm, M)      # sample next actions    for child in children:        observation = environment.step(child.action)        child.state = append(node.state, child.action, observation)        value = evaluate(child.state, value_llm, M)        if terminal(child.state):            reward = environment.reward(child.state)            value = combine(value, reward)            if failure(child.state):                reflection = reflect(child.state, reflection_llm)                M.append(reflection)        backpropagate(child, value)return best terminal trajectory or best-valued frontier trajectory

The actual paper includes more detailed task-specific implementations and an appendix algorithm showing memory, expansion, evaluation, reflection, selection, and backpropagation loops. The appendix also highlights practical limitations: increased cost, tradeoffs around the expansion count, and the assumption that the environment can be reverted or reset during search. arXiv

6. Empirical results

The LATS paper evaluates on HotpotQA, HumanEval, and WebShop. These are deliberately different: multi-hop question answering, code generation, and interactive web shopping. That diversity is important because LATS is not just a reasoning prompt; it is an agent-search wrapper. arXiv+2arXiv+2

6.1 HotpotQA

HotpotQA is a multi-hop QA benchmark where the agent needs to gather and combine evidence. In the LATS setup, the agent can use actions such as search, lookup, and finish, and the experiments use a 100-question subset with environment feedback. arXiv

Method family	Method	HotpotQA result reported in LATS
Reasoning-only	Base	0.32
Reasoning-only	CoT	0.34
Reasoning-only	CoT-SC	0.38
Reasoning-search	ToT	0.55
Reasoning-search	RAP	0.60
Reasoning-search	LATS(CoT)	0.62
Acting	ReAct	0.32
Acting	ReAct best-of	0.38
Acting + reflection	Reflexion	0.51
Acting-search	ToT(ReAct)	0.39
Acting-search	RAP(ReAct)	0.54
Acting-search	LATS(ReAct)	0.63
Combined	LATS(CoT + ReAct)	0.71

The strongest reported HotpotQA configuration is LATS(CoT + ReAct) at 0.71, suggesting that LATS benefits from combining internal reasoning with external action and observation. The paper also reports that naïvely adapting ToT to ReAct can underperform, which supports the claim that simply adding a tree around an agent is not enough; the search policy, value estimate, and feedback design matter. arXiv

6.2 HumanEval

HumanEval tests code generation from programming prompts. LATS uses compiler and unit-test feedback as observations, making it a favorable setting for search because the feedback is relatively crisp. The paper reports that, with GPT-3.5, LATS reaches 83.8%, while with GPT-4 it reaches 92.7% pass@1 in the final table. arXiv

Model / method	HumanEval result reported in LATS
GPT-3.5 CoT	46.9
GPT-3.5 ReAct	56.9
GPT-3.5 Reflexion	68.1
GPT-3.5 ToT	54.4
GPT-3.5 RAP	63.1
GPT-3.5 LATS	83.8
GPT-4 base	80.1
GPT-4 Reflexion	91.0
GPT-4 LATS	92.7

HumanEval is the cleanest demonstration of LATS’s value because code has executable feedback. A failing candidate can reveal syntax errors, failed edge cases, or missing conditions. That makes reflection less speculative than in open-ended reasoning tasks: the model can reflect over concrete test failures rather than merely second-guessing itself.

6.3 WebShop

WebShop is an interactive shopping environment with natural-language instructions, product pages, and a score based on how well the selected item satisfies the instruction. The benchmark contains over a million products and thousands of instructions; Zhou et al. evaluate LATS on 50 instructions and report both average score and success rate. arXiv

Method	WebShop average score	WebShop success rate
ReAct	53.8	28
ReAct best-of	59.1	32
Reflexion	64.2	35
LATS	75.9	38
Imitation learning baseline	59.9	29.1
Imitation + reinforcement learning baseline	62.4	28.7
Fine-tuning baseline	67.5	45
Human expert	82.1	59.6

The WebShop result is notable because LATS improves average score substantially over ReAct and Reflexion, but its success rate remains below the fine-tuning baseline and far below expert humans. That split is informative: LATS may improve partial product fit without always crossing the benchmark’s binary success threshold. The evidence supports LATS as a useful inference-time planner, not as a complete substitute for task-specific training or strong environment models. arXiv

7. Ablations: what actually helps?

The LATS ablation studies suggest that both the LM heuristic and reflection matter, and that naïve search is not enough. On HotpotQA, Zhou et al. compare LATS to variants without the LM heuristic, with DFS, and without reflection. LATS outperforms these ablations, while the no-reflection version drops from 0.63 to 0.58 in the reported ReAct-style setting. arXiv

Variant	HotpotQA result reported
ToT(ReAct)	0.39
RAP(ReAct)	0.54
LATS without LM heuristic	0.37
LATS with DFS	0.42
LATS without reflection	0.58
LATS	0.63

The same section reports that LATS can use fewer average nodes than ToT and RAP at comparable trajectory budgets while achieving higher accuracy, although this should not be misread as “cheap.” It is cheaper than some competing tree-search variants in that experiment, but still much more expensive than a single ReAct trajectory. arXiv

8. Relationship to Tree of Thoughts

Relative to Tree of Thoughts, LATS adds four major ingredients.

First, LATS searches agent trajectories, not only abstract reasoning thoughts. A ToT node is usually an intermediate reasoning state. A LATS node can include actions and observations, so the search state can be grounded in tool results or environment responses. OpenReview

Second, LATS uses external feedback when available. ToT often depends on LLM-generated value estimates or task-specific validators. LATS can incorporate real observations, tests, answer correctness, or environment rewards. That makes it more suitable for interactive tasks but also more operationally complex.

Third, LATS adds reflection memory. ToT can backtrack or choose among candidate thoughts, but LATS explicitly asks the LLM to summarize failures and condition future search on those reflections. This connects it to Reflexion and Self-Refine-style methods, where verbal feedback becomes an input to later attempts. arXiv

Fourth, LATS adopts MCTS-style value backpropagation rather than only BFS/DFS-style search. The official ToT implementation exposes generation, evaluation, and selection hyperparameters such as n_generate_sample, n_evaluate_sample, and n_select_sample; LATS has analogous knobs but embeds them in a visit-counted tree search loop. GitHub

A concise comparison:

Dimension	Tree of Thoughts	LATS
Unit of search	Thoughts / intermediate reasoning states	Full or partial agent trajectories
Environment interaction	Usually absent or task-local	Central when available
Search method	BFS/DFS-style tree search, voting/value prompts	MCTS with selection, expansion, evaluation, backpropagation
Failure handling	Backtracking / re-selection	Backtracking plus natural-language reflection memory
Best fit	Puzzle-like reasoning with decomposable states	Interactive reasoning, tool use, code, web tasks
Core risk	LM value estimates can be wrong	Same, plus higher environment and model-call cost

The most honest framing is that LATS is not merely “ToT but bigger.” It is ToT’s search-over-intermediate-states idea transplanted into an agent setting, with ReAct-style observations, Reflexion-style memory, and MCTS-style allocation of search budget.

9. Relationship to AlphaGo and classical MCTS

The AlphaGo analogy is useful but easy to overstate. AlphaGo combined deep neural networks with search: one network proposed moves, another predicted the winner, and reinforcement learning improved play through self-play. DeepMind’s public description frames AlphaGo as combining deep neural networks with advanced search algorithms, with a policy network for move selection and a value network for winner prediction. Google DeepMind

LATS borrows the shape of that system, not its training regime. In AlphaGo, the policy and value functions are learned for a fixed game with precise legal moves and terminal outcomes. In LATS, the policy and value functions are prompted LLM calls operating over language, tools, and task-specific environments. The LLM is frozen at inference time in the base LATS setup; the improvement comes from search and prompting, not from gradient updates. Zhou et al. explicitly describe LATS as using MCTS with LM-powered value functions and self-reflections without additional training. Proceedings of Machine Learning Research

Dimension	AlphaGo-style MCTS	LATS
Domain	Go board game	Language-agent tasks
State	Board position	Prompt + action history + observation history
Action	Legal move	Thought, tool call, search query, code candidate, finish action
Policy	Trained neural policy	Prompted LLM generation
Value	Trained neural value network	Prompted LLM evaluator plus self-consistency and terminal reward
Environment	Exact game rules	External tools, APIs, benchmarks, code tests, web-like environments
Rollback	Exact and cheap	Assumed possible; can be expensive or impossible in real environments
Learning	Reinforcement learning / self-play	Inference-time search; optional reflection memory
Main failure mode	Misvalued game states, search horizon limits	Bad self-evaluation, hallucinated reflection, tool cost, irreversible actions

The analogy becomes strongest when the task has: legal actions, reliable observations, resettable states, and terminal rewards. It becomes weakest in open-ended real-world agent settings where actions are irreversible, observations are incomplete, and values are subjective.

10. Cost geometry: search depth × node count × model cost

LATS buys quality with inference-time compute. The paper is explicit that LATS has higher cost than simpler baselines and that expansion count creates a tradeoff between performance and compute. It also notes a practical assumption: the environment must be reversible or resettable so that the agent can explore alternative branches. arXiv

A useful approximate cost model is:

total_cost≈ Σ over LLM calls (input_tokens × input_price + output_tokens × output_price)  + environment_step_costs  + latency_parallelization_penalties

The number of LLM calls is roughly controlled by:

iterations × expansion_width × (policy_calls + value_calls + reflection_calls)

The token size of each call is controlled by:

base_task_prompt + trajectory_context + reflection_memory + tool_observation_text

Thus, even when the tree does not expand exponentially, the context length per node tends to grow with depth. A deep trajectory is not just more nodes; each deeper node carries more history. For language agents, this means cost has two interacting axes: more branches and longer prompts.

Cost knob	Effect on quality	Effect on cost	Failure mode
Search iterations	More chances to find good trajectory	More policy/evaluation calls	Diminishing returns
Expansion width	More alternatives per state	Branching explosion	Many low-quality branches
Depth limit	Can solve longer-horizon tasks	Longer contexts, more environment steps	Search gets myopic or too expensive
Evaluation samples	Better value estimate if evaluator is noisy	More value calls	Self-evaluation may still be wrong
Reflection memory size	Can avoid repeated mistakes	Larger prompts	Irrelevant or misleading reflections
Model choice	Stronger policy/value/reflection	Higher per-token cost	Expensive overkill on easy tasks
Environment feedback	Grounds search	Tool latency / API cost	Irreversible or noisy feedback

The LATS paper reports that its sample complexity is comparable to ToT and RAP in the relevant analysis and that it can use fewer nodes upon success in some experiments. But that does not make LATS cheap in absolute terms. The right comparison is not “LATS versus one answer”; it is “LATS versus other ways of spending the same inference budget,” including self-consistency, best-of-N sampling, verifier-guided reranking, and stronger base models. arXiv

This is why Budget-Aware Evaluation matters. Recent work on budget-aware LLM reasoning argues that methods are often compared unfairly unless token budget, sample count, and inference cost are normalized; it finds that chain-of-thought self-consistency can remain a very strong baseline when budgets are matched. ACL Anthology

11. When LATS is likely to help

LATS is most plausible when four conditions hold.

First, feedback is reliable. Code tests, compiler errors, structured environment rewards, and exact answer checks are useful. Vague self-critique is less useful.

Second, the environment is resettable. Tree search assumes that the agent can try branch A, return to the parent, then try branch B. This is natural in a benchmark simulator or local code runner. It is unsafe in environments involving irreversible purchases, messages, financial trades, account changes, or destructive commands.

Third, intermediate mistakes are common and recoverable. LATS is overkill if one strong model call already solves the task. It is valuable when early choices strongly affect final success and when alternative choices can be explored cheaply.

Fourth, the value signal is good enough to guide search. A bad value function makes MCTS confidently allocate compute to bad branches. This is the central technical constraint in LATS-style systems.

The most defensible current use cases are code repair, tool-using QA, web-navigation simulations, mathematical reasoning with verifiers, and training-data generation for later distilled models. The weakest use cases are subjective open-ended writing, irreversible real-world operation, and tasks where the evaluator is no better than the generator.

12. Recent extensions and adjacent work

12.1 Tree search for web agents

Recent work on tree search for language model agents applies inference-time search to more realistic web-agent benchmarks. Koh et al. propose a best-first tree search method for web agents and report large relative improvements on VisualWebArena and WebArena when using GPT-4o, while also noting limitations around speed, destructive actions, and domain-specific value functions. arXiv

This reinforces the LATS thesis that agent trajectories can benefit from search. It also sharpens the engineering critique: real web tasks make environment reset, latency, and action safety much harder than benchmark simulators.

12.2 Reflective MCTS and exploratory learning

Reflective-MCTS extends the LATS-like idea by adding contrastive reflection and multi-agent debate for vision-language model agents. Its associated “Exploratory Learning” stage attempts to transfer search-time experience into the model, reducing inference compute while preserving a large fraction of search performance. arXiv

This is an important direction because it suggests a lifecycle: use expensive search to generate better trajectories, train or adapt a model on those trajectories, then rely less on search at deployment. In other words, LATS-style search may be as valuable as a data-generation mechanism as it is as an online inference algorithm.

12.3 rStar and rStar-Math

The rStar line applies MCTS-style reasoning to smaller language models. The original rStar work uses self-play mutual reasoning with MCTS and a second model as discriminator, reporting large improvements on math reasoning benchmarks for models such as LLaMA2-7B, Mistral-7B, and LLaMA3-8B. ICLR Proceedings

Microsoft’s rStar-Math extends this idea with code-augmented chain-of-thought synthesis, MCTS-guided process reward modeling, and iterative self-evolution. The official project page reports large improvements on MATH and AIME-style evaluations for small models, framing MCTS as part of a training-and-search pipeline rather than only an inference wrapper. Microsoft

The connection to LATS is architectural: search explores multiple reasoning/action paths; a value or reward model selects among them; successful traces can become training data. The difference is that rStar-style systems are more explicitly about bootstrapping reasoning capability in smaller models, while LATS is framed around agent planning with external feedback.

12.4 Step-level Q-value models

Another extension uses MCTS to collect trajectories annotated with step-level Q-values, then trains a model to evaluate or choose reasoning steps. This moves the expensive search signal into a learned process-value model, allowing inference-time selection among candidate steps without running a full tree search every time. ojs.aaai.org

This is a natural response to LATS’s cost geometry. If the search tree is expensive but informative, distill the tree’s preferences into a cheaper evaluator.

12.5 Graphs of Thoughts and non-tree structures

Graph of Thoughts generalizes the search structure beyond trees. Instead of only branching and selecting among thought paths, it models LLM reasoning units as arbitrary graphs, allowing aggregation, refinement, and feedback loops among thoughts. The original GoT paper reports improvements over ToT on selected tasks such as sorting, including quality and cost gains in those settings. arXiv

This matters because LATS’s tree is not necessarily the final form of agent search. Some tasks require merging partial evidence from multiple branches rather than choosing one branch. A graph, blackboard, or belief-state architecture may be more appropriate when multiple partial trajectories each discover useful facts.

13. Criticisms and negative evidence

The strongest criticism of LATS-style search is not that search is useless. It is that search is only as good as its discriminator. A 2024 ACL paper, “When is Tree Search Useful for LLM Planning?”, argues that advanced planning methods require very high discriminator accuracy and that current LLM-based discriminators often cannot balance accuracy and efficiency; it reports that tree search can be 10–20× slower with negligible gains in some settings. ACL Anthology

Budget-aware evaluation makes a related point: many impressive reasoning-search results weaken when compared against simpler methods under equal compute. Best-of-N sampling, self-consistency, and verifier reranking can be brutally competitive baselines, especially in domains where final answers are automatically checkable. ACL Anthology

The “Large Language Monkeys” result pushes this further. It shows that repeated sampling can scale strongly across some code and reasoning tasks, especially when a verifier exists; for SWE-bench Lite, the paper reports improvement from 15.9% with one sample to 56% with 250 samples. But it also finds that in domains without good verifiers, majority voting and reward-model selection can plateau. OpenReview

These critiques do not refute LATS. They narrow its claim. LATS is not “tree search always beats sampling.” It is “structured search over trajectories can be worth its cost when intermediate decisions matter, feedback is meaningful, and the evaluator is strong enough to allocate budget better than blind sampling.”

14. The open question: does external search still matter as reasoning models internalize search?

The most important open question for LATS is whether explicit external search remains valuable as base models internalize more search-like behavior through Reasoning Tokens and reinforcement learning.

OpenAI’s o1 documentation describes models trained with reinforcement learning to reason using chain-of-thought, refine strategies, recognize mistakes, and try different approaches before answering. OpenAI’s API documentation also distinguishes hidden reasoning tokens from visible output tokens: reasoning models may spend tokens internally to consider multiple approaches, and those tokens occupy context and budget even though they are not shown to the user. OpenAI+2OpenAI Developers+2

This creates a direct tension:

Approach	Where search happens	What is visible	What is controllable	Main advantage	Main limitation
LATS-style external search	Outside the model, in an explicit tree	Tree states, actions, observations, scores	Branching, depth, evaluator, reflection, tool use	Auditable and environment-grounded	Expensive and operationally complex
Reasoning-token internal search	Inside the model’s hidden deliberation	Usually only final answer / summary	Reasoning effort and prompt constraints	Simple API surface; strong base reasoning	Less transparent; harder to inject external branch feedback

For pure reasoning tasks, stronger reasoning models may reduce LATS’s marginal value. If a model can already internally explore alternatives, detect contradictions, and revise plans, an external MCTS wrapper may duplicate work while adding latency and cost.

For agent tasks, the answer is less clear. Internal reasoning tokens do not automatically solve environment branching. A model can think about possible web actions, but it cannot observe the result of two mutually exclusive tool calls unless the system actually executes them. LATS remains conceptually valuable when the search tree contains external observations, not just internal thoughts.

The current evidence is thin. The original LATS experiments mostly use GPT-3.5/GPT-4-era non-reasoning or early reasoning-adjacent models, while newer reasoning-token systems change the baseline. There is not yet a definitive, budget-matched body of evidence showing when LATS-style external MCTS beats strong modern reasoning models plus simpler sampling, especially across realistic interactive environments. The right empirical question is no longer “does search help an ordinary LLM?” but “does explicit external trajectory search outperform the model’s internal test-time reasoning under the same total dollar, latency, and safety budget?”

15. Engineering interpretation

LATS should be treated as a design pattern, not a universal agent architecture.

Use it when:

Condition	Why it matters
The task has reliable intermediate or terminal feedback	Search needs a signal better than model self-confidence
The environment can be reset	MCTS assumes alternative branches can be explored safely
Early actions strongly affect final success	Trajectory search can correct bad initial branches
Latency is acceptable	Tree search adds many model and tool calls
The domain supports verifiers or tests	Value estimates become less speculative
Failed attempts teach useful lessons	Reflection memory can improve later branches

Avoid it when:

Condition	Why it is dangerous or wasteful
Actions are irreversible	Search branches may cause real-world side effects
Feedback is subjective or delayed	The value function becomes guesswork
A single strong model call is already reliable	Search adds cost without benefit
The evaluator is weaker than the generator	MCTS amplifies bad preferences
Tool calls are expensive or rate-limited	Branching multiplies operational cost
The task rewards creativity rather than correctness	Tree search may over-optimize shallow criteria

The practical future of LATS-like systems is likely hybrid. Explicit search may be used during development, evaluation, difficult cases, or data generation; distilled process-value models and reasoning-token models may handle routine cases; external tree search may activate only when uncertainty, risk, or verifier feedback justifies the compute. That is a more realistic endpoint than always-on MCTS for every agent action.

16. Selected primary references

Zhou et al., “Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models.” The primary LATS paper, published at ICML 2024 in PMLR, introducing MCTS over language-agent trajectories with LM policy, value, and reflection modules. Proceedings of Machine Learning Research

Yao et al., “Tree of Thoughts.” The key predecessor for deliberate search over intermediate reasoning states, using LLM generation and evaluation to guide search. OpenReview

Yao et al., “ReAct.” The key predecessor for interleaving reasoning traces with external actions and observations. arXiv

Shinn et al., “Reflexion.” The key predecessor for verbal reflection over failed attempts and reuse of that reflection as memory. arXiv

Hao et al., “Reasoning with Language Model is Planning with World Model.” The RAP line, using LLMs with MCTS-style planning and model-based reasoning. arXiv

DeepMind AlphaGo technical lineage. The canonical neural-policy/value-plus-search analogy, useful for understanding what LATS borrows and what it does not. Google DeepMind

Koh et al., “Tree Search for Language Model Agents.” A more recent web-agent search paper showing that inference-time tree search can improve realistic web tasks while exposing latency and action-safety limits. arXiv

Chen et al., “When is Tree Search Useful for LLM Planning?” A direct criticism of LLM tree search, emphasizing discriminator quality and compute-normalized evaluation. ACL Anthology

Snell et al., “Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters.” / related budget-aware literature. The broader context for evaluating LATS as one member of the inference-time compute family rather than as an isolated algorithm. ACL Anthology