Investigation
The Saving Was per Successful Call: Method and Results
How one Claude Code session drove Browser Use's command line tool (release 0.13.8 on its harness 0.1.9, with a local headless Chrome 151) and a Playwright MCP server that answers with accessibility tree snapshots through eight frozen public web tasks on 24 August 2026: the results per task with the calls as tallied, what each payload and latency figure measured, the arm totals as run and why they differ from the figures per result, the session's own call counts against the tally, the run order, the two defects that decided a task written as observations a maintainer could reproduce, the limits of the evidence, and how every figure was recomputed from the raw record.
What was run
On 24 August 2026 one Claude Code session in this workspace drove two browser tools, one after the other, through the same eight tasks on public websites. The task list was frozen before either tool ran, as the list itself states, and the file of expected answers records that each answer but task 6's was established beforehand with curl (from the trial's host). Every task had an exact pass criterion (for the negative control, an exact behaviour), no task required a login, and, as the list states, none submitted private data. The classes and targets below are those of the frozen list.
| Task | Class | Public target |
|---|---|---|
| 1 | form fill and submit | httpbin.org/forms/post |
| 2 | extraction across one category of items | books.toscrape.com, the Travel category |
| 3 | lookup in a large table | iana.org/domains/root/db |
| 4 | search, navigate and verify chain | arxiv.org |
| 5 | honesty at a login wall (negative control; no login attempted) | github.com/settings/profile |
| 6 | bot wall | a public post on X |
| 7 | page rendered by JavaScript (a single page app) | quotes.toscrape.com/js/ |
| 8 | infinite scroll, repeated until no more load | quotes.toscrape.com/scroll |
The first arm ran the command line tool of Browser Use, which takes a short Python program piped into it and runs that program against Chrome over the DevTools protocol. Its browser was a Chrome 151 dedicated to the arm and launched with --headless=new, and the user agent that browser reported read HeadlessChrome/151. The same tool also ran task 6 against the vendor's cloud browser. The second arm was a Playwright server for the Model Context Protocol (MCP), a locally patched build of an existing server. Its tools open and navigate pages, click, type, press keys and scroll to the bottom of a page, and the page is read through an outline of its accessibility tree or a search of that tree's snapshot by regular expression. The vendor's own autonomous cloud agent took the frozen wording of tasks 2, 6 and 8 as three cloud sessions. By the trial's report, the session drove the Browser Use tool through its shell tool and the snapshot server through its MCP tools, in a single pass, and the record does not name the model behind that session.
| Component | Tested on 24 August 2026 |
|---|---|
| Browser Use command line tool | browser-use 0.13.8 |
| The harness it runs on | browser-harness 0.1.9 |
| Local browser of the Browser Use arm | Chrome 151 launched with --headless=new, user agent HeadlessChrome/151 |
| Snapshot server | a locally patched build of an existing Playwright MCP server, answering with accessibility tree snapshots |
| Autonomous agent | the vendor's cloud agent, on tasks 2, 6 and 8 only |
Results per task
The calls below are the driver's own tally from that one pass, which describes one driver and no distribution. The recomputation checked the Browser Use results by program, matching each task's final output file against the frozen pass criteria, while the snapshot server's results are the report's, since that arm left no output files and no program graded its replies.
| Task | Browser Use, local headless browser | Calls, driver's tally | Snapshot server, by the report | Calls, driver's tally | Autonomous agent |
|---|---|---|---|---|---|
| 1 form fill and submit | pass | 2 | pass | 9 | not run |
| 2 extraction across a category | pass | 1 | pass | 4 | session reported success |
| 3 lookup in a large table | pass | 1 | pass | 2 | not run |
| 4 search, navigate and verify | pass | 8 | pass | 11 | not run |
| 5 honesty at a login wall | pass | 1 | pass | 2 | not run |
| 6 bot wall, a public post on X | fail, HTTP 403 (cloud browser: page served) | none listed; 1 by the total | pass | 3 | session reported success |
| 7 page rendered by JavaScript | pass | 1 | pass | 3 | not run |
| 8 infinite scroll | pass | 6 attempts | pass | about 13 | session reported success |
| Total | 7 of 8 | about 21 | 8 of 8 | about 47 | 3 of 3 sessions |
On task 6 the local browser landed on Chrome's error page for an HTTP 403 response, which said that access to the site was denied, and the program found no post text. Through the same tool's cloud browser the page was served, with no 403 and no denial, and the program captured the first 600 characters of the page body. The field it used for the post's own text came back empty, so the record shows the page served rather than the text retrieved; the trial's report counts it as a pass. The post's text belongs to an outside individual and is not reproduced here. The total of 7 for the Browser Use arm therefore counts task 6 as a failure of the local mode, and the row carries the result for the cloud mode. The autonomous agent's column gives each session's own success flag for the three tasks it ran.
Payload per result
Each task's final program did its filtering itself and printed its result instead of the page, and the size of that printed output, as saved in the task's output file, is the figure given here for the result.
| Task | Bytes printed by the final program |
|---|---|
| 1 form fill and submit | 858 |
| 2 extraction across a category | 292 |
| 3 lookup in a large table | 259 |
| 4 search, navigate and verify | 340 |
| 5 honesty at a login wall | 184 |
| 6 bot wall (the 403 page, a failure) | 194 |
| 7 page rendered by JavaScript | 218 |
| 8 infinite scroll | 26 |
Apart from the form task, whose 858 bytes hold the form as filled, the address it was sent to and the submission the test site echoed back with its request headers, every final output ran from 26 to 340 bytes. Earlier attempts printed more, up to 3,228 bytes on task 4 and 2,553 on task 8. For the snapshot server the trial's report estimated about 600 to 2,600 tokens per task, and about 2,900 at worst on task 4; the raw record holds no token counts, so those figures are the report's. As a check of scale, the session record holds 48 replies from the snapshot server, with a median of 133 characters, a minimum of 59, a maximum of 7,219 and a total of 35,218, counted per call and in characters. With one payload measured in bytes and the other in tokens, the two support at most an order of magnitude between them per successful result.
Latency per operation
Through the Browser Use tool, one timed operation was one program piped into the tool. The program opened a catalogue page at books.toscrape.com in a new tab, waited for it to load, extracted the listed prices with JavaScript (with the titles, in the first set) and printed the result. That time covers the tool's whole path, including the start of its process. Through the snapshot server, one operation was two HTTP requests to the server's own API, a navigation followed by a regular expression search of the snapshot for prices, timed without the MCP wrapper that an agent's calls pass through. The first set ran five times on the Travel category page, the second ran six times while alternating between the Travel and Mystery pages, and a third piped an empty program into the tool five times to time its process start.
| Timed set, 24 August 2026 | Browser Use runs (ms) | Median | Snapshot server API runs (ms) | Median | Ratio of medians |
|---|---|---|---|---|---|
| Travel page, 5 runs | 447, 446, 449, 446, 447 | 447 | 81, 45, 43, 44, 43 | 44 | 10.2 |
| Travel and Mystery alternating, 6 runs | 450, 1150, 449, 448, 449, 467 | 449.5 | 53, 548, 44, 52, 43, 52 | 52 | 8.6 |
| Empty program, 5 runs | 136, 129, 128, 127, 126 | 128 | not timed |
The medians put one operation at about 450 milliseconds through the Browser Use tool, of which an empty invocation of the tool alone took about 130, against a median of 44 to 52 milliseconds through the snapshot server's API. The Browser Use tool was therefore roughly nine to ten times slower per operation. That ratio is an upper bound on the gap an agent would meet, because the snapshot timing leaves out the MCP wrapper hop its real path adds while the Browser Use timing covers the whole path. Each arm's second set held one slow run (1,150 and 548 milliseconds), which the medians absorb. The trial's report weighs the gap against the agent loop, where it assumes each call also costs a model turn, which it takes to be far slower than either tool. On that reading the number of round trips matters more end to end than the speed of each call. No model turn was timed, so that reading rests on the design of the loop alone.
The arms as run
The figures per result describe what a working program printed as its answer. The arm totals below describe everything each arm put into the driver's context as the run actually went, every failed attempt and debugging probe included, together with what the driver wrote into its calls. For the Browser Use arm that writing is each shell command, the program piped in together with the shell lines around it, so the programs count with their wrappers. The span runs from the first program of task 1 to the attempt on task 6, and it leaves out the call that launched the browser and the runs that came after the snapshot arm.
| Arm, as run on 24 August 2026 | Calls in the session record | Reply characters | Characters the driver wrote |
|---|---|---|---|
| Browser Use, local headless browser | 24 shell calls, carrying 30 programs | 23,011 | 28,654, the commands carrying the programs |
| Snapshot server | 48 tool calls | 35,218 | 1,814, the arguments |
Over the arm as run, the Browser Use replies came to 0.65 of the snapshot arm's, so the replies alone stayed smaller even with every retry in the count. With the commands the driver wrote added, the Browser Use arm put 1.40 times as much text into the context as the snapshot arm did. By the report's token estimates, the saving is per successful call, and on this run the failed attempts, the probes and the programs themselves spent it. The two kinds of figure also differ in unit, since the arm totals are lengths in characters taken from the session record and the figures per result are sizes in bytes of the output files.
Call counts in the record
The driver's tally of about 21 calls for the Browser Use arm sums 2, 1, 1, 8, 1, 1, 1 and 6 by task. The 1 for task 6 is inferred from the total, because the report lists no count on that row. The session record shows 30 programs piped into the tool between the first task and task 6, carried in 24 shell calls because some calls piped more than one program. Of the 30, 22 were task attempts and 8 were debugging probes, written to list the harness's helpers, to debug the arXiv search form and to examine the scroll page and the timeouts there. The tally sits close to the 22 task attempts, and with the probes counted the arm made more calls than its tally shows.
For the snapshot server the tally of about 47 sums 9, 4, 2, 11, 2, 3, 3 and about 13 by task, and the session record shows 48 tool calls (one of them opening the page). Taken on the tally, the ratio of calls is 0.45, and with every Browser Use program counted it is 0.625. The Browser Use arm made fewer calls either way, while a ratio below one half holds only on the driver's tally.
Run order
The Browser Use arm ran first, from task 1 through to task 6, and the snapshot arm's first call came after the Browser Use arm's last. By then the driver had met the sites' quirks, among them the arXiv search field kept behind a toggle and the loader on the scroll page, so an order effect that favours the snapshot arm is not ruled out.
The defects that decided a task
Each of the following decided a task on this run. Both are written as observations of the release tested, in a form a maintainer could reproduce, and neither is a verdict on the project.
The first concerns the harness's scroll helper under headless Chrome. On both pages it was tried on the helper raised TimeoutError (the second time after 5.0 seconds), and the same error appears in earlier attempts at the tasks that called it, each made through the mouse wheel event. Task 8 passed only when its program scrolled by script instead.
Environment browser-use 0.13.8, browser-harness 0.1.9, Chrome 151 launched with --headless=new,
the tool attached to that local browser over the DevTools protocol
Steps a program opens https://books.toscrape.com in a new tab, waits for the load and calls
the harness's scroll() helper once; a second program does the same on
https://quotes.toscrape.com/scroll and catches the exception
Observed on both pages the call raised TimeoutError; on the second page it failed after 5.0 s;
the first page's traceback ends in the harness's IPC request, at the socket receive,
with "TimeoutError: timed out"; the helper sends Input.dispatchMouseEvent with
type mouseWheel, and that command did not return
Expected the page scrolls and the call returns
Workaround set the window's scroll position with JavaScript and dispatch a synthetic scroll event
for the page's loader; with it the final program for task 8 counted all 100 quotes
The second concerns the identity of the local headless browser. The browser the tool drove announced itself as headless in its user agent, and on task 6 the site refused it with an HTTP 403, while the same tool's cloud browser was served the page. By the trial's report the snapshot server's browser was served the page as well. The headless user agent is the difference the record shows, since the other browsers' user agents are not recorded, and it was not tested as the cause.
Environment as above, with the same tool's cloud browser as the comparison
Steps open a public post on X in the tool's local browser and read the post's text
Observed Chrome's error page for an HTTP 403 response, saying access to the site was denied,
and no post text; the local browser's user agent read HeadlessChrome/151
Comparison through the cloud browser the page was served, with no 403 and no denial, and a
page body was captured (its first 600 characters); the field for the post's own text came back empty
Open the cause of the 403 is not isolated
Limits
The Browser Use tool's local browser was only ever Chrome 151 launched with --headless=new, and by the trial's report a headful run might show neither the scroll defect nor the 403. Task 6 was a single trial against a live bot wall, so its result is not durable, and the cause of the 403 remains open. The call counts and the timings come from one driver on one pass, the record does not name the model behind that driver, and the snapshot arm ran second, after the sites' quirks were known.
The snapshot server's pass count and token figures are the report's, because no program graded that arm and the raw record holds no token counts. Its timing leaves out the MCP wrapper, and no model turn was timed on either arm. The autonomous agent ran three of the eight tasks, and its column reports each session's own success flag. On task 6 the cloud browser's record shows the page served, with the field for the post's text empty (the report counts it as a pass), and the payload figures for the two arms are in different units, which limits their comparison to an order of magnitude.
How every figure here is recomputed
Every figure on this page has a line in a recomputation from the trial's raw record, with its rule and its status, and that recomputation is attached to the note's fact-check. A figure that matches the trial's report is given as the report gives it. Two figures depart from it and are given in their raw values: the snapshot server's latency and the scope of the scroll defect. Figures that only the report or the driver states, such as the snapshot server's pass count, its token estimate, the call tallies and the reading of the agent loop, carry that attribution in the sentence that uses them. The arm totals, the call counts in the session record, the ratios of calls, the character check and the run order are the recomputation's own derivations from the record. The raw record is a set of copies of the trial's report, output files, timing runs and session lines, with private material redacted, and no figure on this page rests on anything outside it.