mini-SWE-agent (open-source)

Citation check incomplete — recheck later.

Summary

mini-SWE-agent is a minimal open-source coding agent from the team behind SWE-bench and SWE-agent: about 100 lines of code that let a language model chosen by the user work in a repository through bash commands only. Sixteen independent studies that ran it were found (nine peer-reviewed), most from 2026, and most compared it with other harnesses using the same models. With strong recent models it often scored close to heavier harnesses (within five points of Claude Code on SWE-bench Verified with mini-SWE-agent 2.4.6; 75.5% against 69.8-73.6% for vendor harnesses on SWE-Serve), but several studies found it lower with weaker or differently trained models and on some task types (backend development in ABC-Bench, test generation in SWE-Mutation, small open models in SWEnergy). Security studies found weaknesses in 4.3-6.0% of its test-passing patches, an attack success rate between those of SWE-agent and OpenHands, and runs it reported as successful despite destructive patches. One medium-severity CVE affects its optional Bubblewrap sandbox, with no fix found as of 20 October 2026. Informational only; not a guarantee, endorsement, or certification.

Overall score: 61 · Evidence grade: A · Independent sources: 16 performance (9 peer-reviewed), 10 other

Evidence grade describes the amount and type of evidence, not the agent's quality.

Grade note: Grade A: 16 counted sources (9 peer-reviewed), and recent tests state the version tested: What Does a Harness Buy? (preprint, mini-swe-agent 2.4.6, the latest release), SWE-Serve (v2.4.3), DeepSWE (May 2026 commit) and DCAS (ASE 2026, v2.2.8). Results reflect the model and harness together (see Decision 0085, D3). (see methodology)

Factor breakdown

0.30 × 55 + 0.20 × 55 + 0.25 × 50 + 0.25 × 85 = 61.25, rounded to 61 (rounding rule).

Scope

The open-source mini-SWE-agent (about 100 lines of agent code; the model acts only through bash commands), run with a language model chosen by the user, from v1.0.0 (22 July 2025) through v2.4.6 (23 July 2026) and the main branch (last code change 3 September 2026). v1.x and v2.x are both covered; v2.0.0 (11 February 2026) changed the interface. Studies that run models inside unmodified mini-SWE-agent count as tests of mini-SWE-agent; modified copies, forks and agents built on it are listed as other. SWE-agent, the team's larger agent, is a separate entry, and no study is counted for the same runs in both entries. Results depend heavily on the language model and task.

How sources were counted

The mini-SWE-agent repository, documentation and the SWE-bench leaderboards are the developers' own material and are not counted. Studies by independent authors that run models inside unmodified mini-SWE-agent count, with competing or add-on methods disclosed; most of them compare it with other harnesses using the same models. Modified copies, forks, agents built on it, and re-analyses of runs submitted by others are listed as other. Studies with mini-SWE-agent developers among the authors are excluded. SWE-agent, the team's larger agent, has its own entry, and no runs are counted in both.

Security and disputes

CVE-2026-108592 (published 10 October 2026; CVSS 4.0 score 6.0, Medium, per VulnCheck; NVD status Deferred) describes the optional Bubblewrap sandbox in versions 1.10.0 through 2.4.6 passing host environment variables, including API keys, to sandboxed commands, so a prompt injection in task content could read and send them out. No fix was found in a release or on the main branch as of 20 October 2026. In March 2026 the project excluded two litellm versions compromised in a supply-chain attack within hours of the attack. The default local environment runs commands directly on the user's machine. No legal or regulatory action was found.

Uncertainties

• Results reflect the model and the harness together; most counted studies run several harnesses with the same models, and two use mini-SWE-agent only as a fixed harness to compare models.
• Many more papers run mini-SWE-agent than are counted here; the counted set is a representative sample of independent studies found, not an exhaustive list.
• The SWE-bench and bash-only leaderboards, ProgramBench and CodeClash involve mini-SWE-agent developers and are not counted (0031).
• v2.0.0 (February 2026) changed the interface; most counted studies do not state which version they ran, and older results may describe v1.
• When "Correct" Is Not Safe and A Jagged Frontier share two authors.
• ABC-Bench figures are read from a chart.
• CVE-2026-108592 was published on 10 October 2026; NVD has not analysed it, and no fix was found as of 20 October 2026.
• Citation check incomplete — recheck later (OpenAlex full text used: 458 works matched and 419 downloaded; Semantic Scholar intermittently rate-limited and the arXiv API timed out).

Sources

Independent performance sources (counted)

• When "Correct" Is Not Safe (Carnegie Mellon, Michigan, Peking, Colorado State, Google; ACL 2026): on SWE-bench patches from mini-SWE-agent that passed all tests, 4.3% (Claude Sonnet 4) to 6.0% (Qwen3-Coder) contained security weaknesses without any attack; under the authors' injected-issue attack, average attack success was 10.4% for mini-SWE-agent against 14.7% for SWE-agent and 9.7% for OpenHands with the same four models, up to 67.6% for one weakness type (CWE-538) with Claude Sonnet 4. Disclosure: one author is at Google, which makes competing coding agents; two authors also wrote A Jagged Frontier (counted) — accessed 2026-10-20
• EET (Nanyang Technological University, King's College London, UCL, UIUC, Tsinghua; Findings of ACL 2026): mini-SWE-agent resolved 55.6% of SWE-bench Verified with GPT-5-mini ($16.07 total) and 66.4% with DeepSeek-V3.2 ($52.20); the authors' early-termination add-on cut its cost by about 19% with resolution unchanged (+1.0 and +0.6 points), a smaller saving than for Agentless or Trae Agent. Authors propose an add-on method (disclosed) — accessed 2026-10-20
• DCAS (Huawei Canada, Queen's University; ASE 2026): with mini-swe-agent v2.2.8 on SWE-bench Verified, base Qwen3-Coder-30B-A3B resolved 35.4%, and the authors' plan-trained variant 42.4%; open models fine-tuned on OpenHands trajectories dropped sharply when moved to mini-swe-agent (Nebius-SWE-Rebench-30B 49.7% under OpenHands, 20.4% under mini-swe-agent). Authors propose a training method (disclosed). Disclosure: Huawei makes competing coding tools — accessed 2026-10-20
• SWEnergy (IIIT Hyderabad; AGENT '26 workshop at ICSE, ACM proceedings): four frameworks with two small open models (Gemma-3 4B, Qwen-3 1.7B) on SWE-bench Verified Mini, 150 runs each; mini-SWE-agent resolved none of 50 tasks with either model (only AutoCodeRover resolved any, 4%). Its low energy use reflected early failures when prompts exceeded the model's context and runs it reported as successful despite destructive or invalid patches (one replaced a 2,000-line file with a single invalid line) — accessed 2026-10-20
• Coordinating Agents on a Shared Git Workspace (Istanbul Technical University; ESEM 2026, LIPIcs): mini-swe-agent as the single-agent baseline on 266 SWE-Bench Pro instances resolved 138.3 (MiniMax) and 152.0 (DeepSeek-V4-Pro) on average; the authors' two-agent system built on mini-swe-agent added about 5 points. Authors propose a method built on mini-SWE-agent (disclosed) — accessed 2026-10-20
• ABC-Bench (Fudan University and others; Findings of ACL 2026): backend-development tasks run with DeepSeek-V3.2 and GPT-5 under three frameworks; OpenHands reached about 50% with both models while mini-SWE-agent was markedly lower, below 20% with GPT-5. Results read from a figure — accessed 2026-10-20
• SWE-Mutation (USTC, Beihang, Peking and others; Findings of ACL 2026): test-suite repair and generation in mini-SWE-agent and Claude Code; Claude Code was more effective, and the authors trace part of the gap to mini-SWE-agent's bash-only editing (sed edits causing indentation and syntax errors). Claude Sonnet 4.5 in mini-SWE-agent: 42.60% valid-repair rate on repair, 29.80% on generation — accessed 2026-10-20
• CodeScout (University of Maryland, Amazon Web Services and others; Findings of ACL 2026): on SWE-bench Verified with default settings, mini-SWE-agent resolved 51.8% (GPT-5-mini), 40.2% (Qwen3-Coder) and 11.0% (DeepSeek R1); the authors' problem-statement augmentation raised these to 53.4%, 45.4% and 13.2%. SWE-agent and OpenHands were run in the same way. Authors propose an add-on method (disclosed). Disclosure: Amazon makes competing coding agents — accessed 2026-10-20
• What Does a Harness Buy? Tokens, Mostly (Shandong University; preprint October 2026): mini-SWE-agent 2.4.6, Claude Code and OpenCode with pinned versions on SWE-bench Verified; on 447 tasks with two models, Claude Code and mini-SWE-agent were equivalent within five points; on 45 hard tasks and five models, swapping harness flipped as many tasks (13%) as rerunning the same one. Cost per task differed by up to 3x, driven by the fixed preamble sent each step (829 tokens for mini-SWE-agent against 16,581 for Claude Code). A public PREreview exists; not peer-reviewed — accessed 2026-10-20
• SWE-Serve (UC Berkeley; preprint September 2026): 53 inference-serving engineering tasks, mini-SWE-agent v2.4.3 as the common harness for 11 models; in a harness check with three runs each, mini-SWE-agent reached 75.5% mean pass@1 with both GPT-5.6 Sol and Claude Opus 5, against 73.6% for the same GPT model in Codex and 69.8% for the same Claude model in Claude Code — accessed 2026-10-20
• Beyond the Model: Demystifying Harness Effects (Nanjing University of Science and Technology, Nanjing University, TU Munich; preprint September 2026): mini-SWE-agent and OpenCode with ten Qwen and DeepSeek models on 300 SWE-bench Pro tasks, ProgramBench and GitTaskBench; OpenCode's lead over mini-SWE-agent shrank from about 6 to 2.67 points as Qwen models got stronger and was 0.6 points with DeepSeek-V4-Pro (34.3 against 33.7). The authors' modular harness built on mini-SWE-agent added 6.21-7.37 points. Their SWE-bench Verified comparison table reuses leaderboard figures and is not relied on here — accessed 2026-10-20
• A Jagged Frontier (Carnegie Mellon, UIUC, Colorado State, Microsoft; preprint August 2026): robustness to meaning-preserving code rewrites with four models; averaged over models, mini-SWE-agent lost 1.34 points on SWE-bench Verified and 1.88 on SWE-bench Pro, against 1.88 and 3.65 for OpenCode; unperturbed it resolved 89.05% (SWE-bench Verified) and 76.35% (SWE-bench Pro), against 81.20% and 79.33%. Disclosure: one author is at Microsoft, which makes competing coding agents; two authors also wrote When "Correct" Is Not Safe (counted) — accessed 2026-10-20
• Hearsay (Case Western Reserve, Nanyang Technological University, National University of Singapore; preprint September 2026): five harnesses ran fourteen tasks (28 runs each) and examiners judged whether each harness's own run record could be checked; 23 of 28 mini-SWE-agent runs passed the entry test, and for the primary examiner 75.3% of the record material it cited was corroborated (aider 40.6%, SWE-agent 45.0%, OpenCode 77.0%). None of the sixteen frameworks reviewed writes a fully checkable record — accessed 2026-10-20
• DeepSWE (Datacurve; preprint July 2026): 113 original long-horizon tasks, every model run in mini-swe-agent at a pinned commit (May 2026 runs); this main table compares models. In a small check (10 SWE-Bench Pro tasks, two runs) mini-swe-agent's pass rate equalled or exceeded each model's own product harness (50% vs 40% Claude Opus 4.7, 40% vs 40% GPT-5.5, 40% vs 20% Gemini 3.1 Pro); the authors state these differences are within noise. Disclosure: Datacurve sells training data; the mini-SWE-agent README promotes this result — accessed 2026-10-20
• LiveCANNBench (Beihang, UESTC and others; Findings of ACL 2026): Ascend CANN coding benchmark; mini-SWE-agent with default settings and full terminal access used for the backward task to compare several models. Fixed-scaffold model comparison (counted under the 0037/0038 reading; context only under D3b) — accessed 2026-10-20
• SWE-CC (Singapore Management University, SUTD; preprint October 2026): repository contribution-policy compliance on 12 projects with four models in mini-SWE-agent and OpenHands (runs August 2026); average compliance 55.7% for mini-SWE-agent against 58.2% for OpenHands, resolve rate 81.7% against 83.9%; supplying the policies raised mini-SWE-agent's compliance to 64.4%. Without that, it fetched and retrieved policy text less often than OpenHands with three of four models — accessed 2026-10-20

Other sources (not counted)

• Frozen Judges, Moving Agents (preprint September 2026): re-scores the graded patches of 35 mini-SWE-agent submissions on the SWE-bench Verified leaderboard to study language-model judges. Re-analysis of submitted outputs (0076), not a new run — accessed 2026-10-20
• Same Signal, Different Semantics (preprint May 2026): behavioural analysis of 64,380 public SWE-bench runs from 43 frameworks, including 33 models on mini-swe-agent. Re-analysis of submitted runs (0076) — accessed 2026-10-20
• Trajectory-Level Security Debt (University of Luxembourg; preprint September 2026): scanned final code from ProgramBench runs made with mini-SWE-agent; those runs were published by the ProgramBench team, which includes mini-SWE-agent developers. Re-analysis (0076). Its separate SWE-agent part is counted for SWE-agent — accessed 2026-10-20
• Empowering Autonomous Debugging Agents with Efficient Dynamic Analysis (SUSTech, Ant Group; FSE 2026, PACMSE): mini-SWE-agent baseline figures (52.8% Claude 3.7 Sonnet, 21.6% GPT-4o) are taken from the SWE-bench leaderboard; the authors' own runs use a modified mini-SWE-agent (+10.6% and +18.5% resolved tasks). Modified version (0042) and reused leaderboard figures — accessed 2026-10-20
• Live-SWE-agent (UIUC; preprint November 2025): a self-extending agent built on mini-SWE-agent; its mini-SWE-agent baseline figures (59.8-74.2% on SWE-bench Verified) are reused from published results, not rerun. Derived agent (0042) — accessed 2026-10-20
• GA4GC (SSBSE 2025, LNCS): multi-objective configuration tuning of a mini-SWE-agent fork with Gemini 2.5 Pro on 12 SWE-Perf tasks. Fork, small sample (0042) — accessed 2026-10-20
• Same Model, Different Harness (preprint August 2026): uses mini-SWE-agent v2.2.8 only as a calibration reference (87.6% task-level agreement with the author's own harness on 500 SWE-bench Verified tasks, Cohen's kappa 0.66); mini-SWE-agent's own results are not the subject — accessed 2026-10-20
• CVE-2026-108592 (VulnCheck, published 10 October 2026; CVSS 4.0 6.0 Medium; NVD status Deferred): the optional Bubblewrap sandbox in mini-swe-agent 1.10.0 through 2.4.6 does not clear the host environment, so a prompt injection in task content can make the agent read and send out API keys. No fix found in a release or on the main branch as of 20 October 2026 — accessed 2026-10-20
• GitHub issue #941 (open, 18 August 2026): a user running at scale reports intermittent crashes traced to the unpinned litellm dependency and asks for a lock file. Single user report — accessed 2026-10-20
• GitHub issue #990 (open, 11 October 2026): parallel SWE-bench runs with the Bubblewrap environment can share one writable workspace, so instances can overwrite each other; a fix is proposed in pull request #993 (open). Single user report — accessed 2026-10-20

Not counted

• GitHub repository and README (MIT licence; created 28 June 2025; releases v1.0.0 22 July 2025 to v2.4.6 23 July 2026; v2.0.0 on 11 February 2026 changed the interface): developer claims such as '>74% on SWE-bench Verified' and adoption lists. Not counted — accessed 2026-10-20
• Pull request #794 (merged 24 March 2026, about two hours after it was opened): excludes litellm 1.82.7 and 1.82.8, versions compromised in a same-day supply-chain attack that stole environment secrets. Not counted — accessed 2026-10-20
• CodeClash (Princeton, Stanford; preprint November 2025): runs models in mini-SWE-agent, but Kilian Lieret, Carlos E. Jimenez and Ofir Press, mini-SWE-agent developers, are co-authors. Excluded under 0031 (shared authors) — accessed 2026-10-20
• ProgramBench (Stanford, Princeton; preprint May 2026): runs models in mini-SWE-agent; John Yang, Kilian Lieret and Ofir Press are co-authors. Excluded under 0031 — accessed 2026-10-20
• Results for a language model alone, and studies that use mini-SWE-agent only to generate training data (out of scope).

Decision record: 0085.