Browser Use (open-source library)

Summary

Browser Use is an open-source Python library (MIT licence) that lets a language model of the user's choice control a web browser to carry out tasks. Results depend heavily on the model used. In independent evaluations (Online-Mind2Web, COLM 2025; Princeton's Holistic Agent Leaderboard; CAP, COLM 2026; a 2026 preprint), success rates ranged from about 2–5% full success on hard cross-site tasks to 18–45% on live-website benchmarks, well below the vendor's own figures. Independent security studies (VPI-Bench, ICLR 2026; five preprints) found it often followed instructions hidden in web pages, could be misdirected when a page changed between check and action, and often revealed private facts through its actions, and a domain-restriction bypass (CVE-2025-47241) was fixed in version 0.1.45. The vendor's Cloud service is not covered. Overall score 51. Informational only; not a guarantee, endorsement, or certification.

Overall score: 51 · Evidence grade: A · Independent sources: 11 performance, 3 other

Evidence grade describes the amount and type of evidence, not the agent's quality.

Factor breakdown

Overall = 0.30×40 + 0.20×50 + 0.25×30 + 0.25×85 = 50.75, shown as 51. See the methodology.

Scope

This entry covers the open-source browser-use Python library (MIT licence) run with a language model the user chooses. Results depend heavily on that model, so independent results are shown as ranges across the models tested. Browser Use Cloud and the vendor’s own models are not covered or scored; the only figures for them are the vendor’s own. The library changed substantially during 2025 (current release 0.13.11), and most studies do not state the version tested. Tests of 2025 versions are counted because the project is continuous.

Independent results by model

Security and disputes

No lawsuit or regulatory action was found. One security advisory (GHSA-x39x-9qw5-ghrf / CVE-2025-47241) concerned a bypass of the library’s allowed_domains restriction; it was fixed in version 0.1.45. Severity ratings differ between sources: the CVE record lists CVSS 3.1 score 4.0 (Medium), GitHub’s reviewed advisory lists it as critical, and the researchers who reported it rated it 9.3. The same researchers demonstrated credential exfiltration through prompt injection (no CVE assigned). Community pull requests propose further prompt-injection mitigations; their status was not verified. Whether the earlier security findings still apply to current versions has not been independently tested.

Uncertainties

• Results depend heavily on the language model the user connects; scores describe the library across tested models, not any single setup.
• Most sources do not state the library version; the library changed substantially in 2025 (for example the 0.5 release in July 2025). The current version is 0.13.11.
• CAP (late 2025), Skim (2026), BreakingWeb (2026) and AgentTell (2026, version 0.13.1) tested recent versions.
• Four of the eleven sources are security tests of earlier versions.
• Excluded: vendor figures (89.1% WebVoyager, 97% Cloud, BU Bench) and Cybernaut (Amazon preprint), whose 82.2% Browser-Use figure appears to restate the vendor result.
• The Cloud service, which uses the vendor's own models and infrastructure, is not covered and has no independent tests.
• Atomicity for Agents and Online-Mind2Web are both from Ohio State University; there is no author overlap.
• Privacy Practices of Browser Agents: three of the four authors work at Brave Software, which makes a web browser; Brave's own assistant was not evaluated.
• BreakingWeb: five of the eleven authors are at Amazon, which offers a competing browser agent (Nova Act) that was not evaluated.

Sources

Independent performance sources (counted)

• Online-Mind2Web, "An Illusion of Progress?" (COLM 2025; OSU): Browser Use with GPT-4o 30.0% human-evaluated success on 300 live tasks (26% without search, 31% with search); says earlier WebVoyager figures were inflated by automatic evaluation — independent performance, accessed 2026-10-17
• Holistic Agent Leaderboard (Princeton; preprint): Browser-Use scaffold with 12 models, Online-Mind2Web 18.3–40.0% and AssistantBench subset 2.0–38.8%; costs per run are reported — independent performance, accessed 2026-10-17
• HAL Online-Mind2Web leaderboard page (same study as above; counted once) — independent performance, accessed 2026-10-17
• CAP (COLM 2026): Browser-Use with GPT-5 / Claude 4.5 Sonnet / DeepSeek-V4-Flash scored 15.0/21.0/25.0% partial completion and 2.0/5.0/2.9% full success on 198 cross-site tasks (human 35.0/10.0%); run Nov–Dec 2025. Competitor disclosure: one author interned at FellouAI, whose system was also evaluated in CAP — independent performance, accessed 2026-10-17
• Skim (preprint, 2026): BrowserUse with GPT-4o 45.0% on a 300+ task WebVoyager/WebShop subset; authors propose an acceleration method (competing-method disclosure) — independent performance, accessed 2026-10-17
• VPI-Bench (ICLR 2026; NUS): visual prompt injection; Browser-Use attack success 73.7–100% on Amazon, 75.4–98.3% on Booking, 16.7–76.7% on Messenger, 30.0–50.0% on Email, across six models — independent performance, accessed 2026-10-17
• Patlan et al. (Princeton; preprint): plan/context injection; Browser-use up to 63% attack success with context-chained injections, over 80% for privacy prompt injection; GPT-4o — independent performance, accessed 2026-10-17
• ARIMLABS "The Hidden Dangers of Browsing AI Agents" (preprint; security audit of v0.1.44): domain-restriction bypass and credential exfiltration via prompt injection (proof of concept) — independent performance, accessed 2026-10-17
• Atomicity for Agents (Ohio State University; preprint, Feb 2026): tested 10 open-source browser- and computer-use agents on 14 cases where a web page changes between the agent's check and its action (9 synthesized pages, 5 real sites). For Browser Use, the vulnerability was triggered in 9 of 14 cases (all data-update and expiring-state cases) and not in the 5 layout-change cases. Security test, not task completion — independent performance, accessed 2026-10-17
• AgentTell (BRAC University, Qatar Computing Research Institute; preprint, Sep 2026): 100 tasks, 9,760 sessions on Browser Use 0.13.1 with six backbones. Agents carrying a private fact revealed it through their actions in 61.1% of sessions despite an instruction not to, and in 34.5% of leaking sessions told the user it had not been disclosed. Privacy test, not task completion — independent performance, accessed 2026-10-17
• Privacy Practices of Browser Agents (UC San Diego, Brave Software; preprint, Dec 2025): eight browser agents, 15 privacy measurements. Browser Use 0.7.7 showed no warnings for phishing sites or revoked certificates, did not partition cross-site tracking state beyond Chromium defaults, accepted cookie banners and notification requests, and disabled password-storage encryption on Linux (since removed). Privacy test, not task completion — independent performance, accessed 2026-10-17
• BreakingWeb (Duke University, Amazon; preprint, Sep 2026): 519 tasks on seven simulated sites with controlled page problems (delays, misleading confirmations, silent failures). Six models on Browser Use lost 17.9–27.6 percentage points of pass rate, and the interventions overturned 39.6–68.0% of tasks each agent solved cleanly — independent performance, accessed 2026-10-17

Other sources (not counted)

• GitHub-reviewed advisory (critical): allowed_domains bypass, versions before 0.1.45 — independent other, accessed 2026-10-17
• CVE-2025-47241 (CVSS 3.1: 4.0 Medium), fixed in 0.1.45 — independent other, accessed 2026-10-17
• Community pull request on indirect prompt injection through prompt delimiter tags (open issue class; status not verified) — independent other, accessed 2026-10-17
• Vendor claim: 89.1% WebVoyager (modified harness, manual corrections); not used as independent evidence — vendor or maintainer, accessed 2026-10-17
• Vendor claim: 97% Online-Mind2Web for Browser Use Cloud with its own judge; Cloud not covered — vendor or maintainer, accessed 2026-10-17

Excluded sources

• Vendor figures (89.1% WebVoyager, 97% Cloud on Online-Mind2Web, BU Bench): vendor-run, not independent.
• Cybernaut (preprint): its 82.2% Browser-Use figure appears to restate the vendor result; authors build a competing framework.
• Digital Applied blog: commentary without original testing.

Decision records: 0016; 0055 (TOCTOU study); 0059 (remaining candidates).