Aider (open-source)

Citation check incomplete — recheck later.

Summary

Aider is an open-source command-line tool that edits code in a local git repository using a language model chosen by the user. Eight independent studies that ran it were found (five peer-reviewed). Results depend heavily on the model and task. In most studies that compared agents, Aider completed fewer tasks than OpenHands or SWE-agent with the same model, for example 1.85-16.67% task pass rate on GitTaskBench and 11.2-20.0% of vulnerabilities patched on SEC-bench, although it had lower cost per task. Results on secure-coding benchmarks were mixed. Aider with GPT-5 had the best correct-and-secure score (35.8%) of three agents on MT-Sec and was close to the leaders on SecRepoBench (50.6% secure-pass@1), but had the lowest average correct-and-secure rate (6.7%) on SecureVibeBench. Two CVEs have been published (assigned by VulDB and VulnCheck); the related GitHub issues were open with no fix identified as of 17 October 2026. The latest release is v0.86.2 (February 2026). Informational only; not a guarantee, endorsement, or certification.

Overall score: 52 · Evidence grade: A · Independent sources: 8 performance (6 peer-reviewed), 4 other

Evidence grade describes the amount and type of evidence, not the agent's quality.

Grade note: Grade A: 8 counted sources (6 peer-reviewed), and Sifting the Noise tested v0.86.1 within the last 12 months, which meets the A rule. The evidence-quality score (80) is below the suggested A range (85-100) because the A rests on that single recent versioned test, whose date is inferred from its access dates (December 2025 to January 2026). (see methodology)

Factor breakdown

0.30 × 40 + 0.20 × 45 + 0.25 × 45 + 0.25 × 80 = 52.25, rounded to 52 (rounding rule).

Scope

The open-source Aider command-line tool (aider-chat), run with a language model chosen by the user, from v0.82.0 (April 2025) through v0.86.2 (12 February 2026) and the main branch (last commit 22 May 2026). The Aider leaderboards (Polyglot, code editing, refactoring) are run by the maintainer and are not counted. Results depend heavily on the language model and task, so they are reported as ranges by model.

Version and status note

Aider's latest release is v0.86.2 (12 February 2026); the main branch was last updated on 22 May 2026.

How sources were counted

The Aider maintainer runs the Aider leaderboards, so those results are listed but not counted. Studies with Aider developers as co-authors would be excluded; none were found. Aggregator sites that republish leaderboard figures are not counted.

Disclosures for counted sources

No counted study has Aider developers as authors. One SecRepoBench author is affiliated with Google DeepMind. MT-Sec and SecRepoBench share authors, as do SecureVibeBench and Sifting the Noise.

Security

Aider runs commands and edits files in the user's repository. CVE-2026-85674: a repository's configuration file can set test or lint commands that run without confirmation, affecting versions through v0.86.2 (CVSS 3.1 7.8, VulnCheck; GitHub issue #5254 open). CVE-2026-10175: prompt injection in Architect mode can lead to injected code being committed (CVSS 3.1 6.3, VulDB; NVD status: Not Scheduled; VulDB states the maintainer did not respond; GitHub issue #5058 open). No fix was identified for either as of 17 October 2026. No legal or regulatory action was found.

Uncertainties

• Citation check incomplete — recheck later. Unscreened candidates include AAAI 2026 "Interactive Evaluation of LLMs for Multi-Requirement SE Tasks", two ESEM 2026 papers, a TOSEM paper on process-centric analysis of agentic systems, and a CSIT 2026 benchmarking paper.
• Grade A rests on one recent versioned test (Sifting the Noise, ISSTA 2026). The other versioned tests date from 2025, more than 12 months ago.
• Grade A depends on the inferred test date of Sifting the Noise; if it predates 17 October 2025, the grade would be B.
• CVE-2026-10175 names version 0.86.3, which does not appear on PyPI (latest 0.86.2); it may refer to the main branch.
• MT-Sec's EMNLP 2026 venue comes from an author homepage and was not confirmed in the ACL Anthology.
• Results depend heavily on the model, and many results reflect older models.

Sources

Independent performance sources (counted)

• SEC-bench (University of Illinois Urbana-Champaign, Purdue; NeurIPS 2025): 80 real vulnerabilities; Aider patched 20.0% (Claude 3.7 Sonnet), 11.2% (GPT-4o), 17.5% (o3-mini); proof-of-concept 1.2% (Claude 3.7); lowest of three agents; patches mostly under 10 lines — accessed 2026-10-17
• GitTaskBench (AAAI 2026): Aider execution completion 5.56-20.37%, task pass rate 1.85-16.67% across four models (best DeepSeek V3), lowest of three frameworks; lowest cost (under $0.003 per task with DeepSeek V3) — accessed 2026-10-17
• MT-Sec (University of Maryland; EMNLP 2026 per author homepage): multi-turn secure coding; Aider + GPT-5 (thinking) highest correct-and-secure score (35.8%) of three agents (Aider, OpenHands, Codex); C&S fell by 27.3% in one multi-turn setting. Disclosure: authors overlap with SecRepoBench — accessed 2026-10-17
• SecRepoBench (University of Maryland; LLM4Code workshop at ICSE 2026): repository-level secure code completion in C/C++; Aider secure-pass@1 39.0-50.6% (best GPT-5), pass@1 56.6-70.1%; OpenHands up to 53.5%; 21% of Aider + Claude Sonnet 4.5 code did not compile. Disclosure: one author affiliated with Google DeepMind (competing Gemini models) — accessed 2026-10-17
• OmniCode (ACL Findings 2026): with Gemini-2.5-Flash, Aider vs SWE-agent on bug fixing, test generation, code review and style fixing; Python bug fixing 32.4% vs 38.1%, C++ 1.8% vs 8.0%, Java 19.3% vs 14.7%; style fixing 48.6% vs 57.0% (Python) — accessed 2026-10-17
• SecureVibeBench (Singapore Management University et al.; preprint, v5 June 2026): average correct-and-secure rate Aider 6.7%, SWE-agent 14.3%, OpenHands 13.9% across five models; Aider + Claude Sonnet 4.5 high cost relative to results. Disclosure: authors overlap with Sifting the Noise — accessed 2026-10-17
• Sifting the Noise (Monash University; ISSTA 2026, Proc. ACM Softw. Eng. 3, ISSTA009): agents triaging static-analysis alerts; Aider false-positive rate 14.3% (OpenHands 14.9%, SWE-agent 14.1% with GPT-5); Aider more stable across policy types — accessed 2026-10-17
• Multi-Harness RL for Coding Agents (NYU et al.; preprint Oct 2026): six Qwen3-8B-based models on SWE-bench Verified under four harnesses; mean 2.14% solved under Aider (lowest; OpenHands 9.27%). Small research models, not typical use — accessed 2026-10-17

Other sources (not counted)

• CVE-2026-85674: repository config (.aider.conf.yml) can set test-cmd or lint-cmd that run shell commands without confirmation; through v0.86.2; CVSS 3.1 7.8 / 4.0 8.5 (VulnCheck); GitHub issue #5254 open — accessed 2026-10-17
• CVE-2026-10175: Architect mode code injection, reported in 0.86.3 (VulDB); CVSS 3.1 6.3 / 4.0 2.1; NVD Not Scheduled; VulDB states the maintainer did not respond; GitHub issue #5058 open — accessed 2026-10-17
• Complex Agents, Shallow Tests (preprint Oct 2026): analyses Aider v0.86.2's own test suite; about the project's testing, not agent performance — accessed 2026-10-17
• VibeJam: user-study platform that uses Aider by default; pilot only, no Aider performance results — accessed 2026-10-17

Not counted

• Aider leaderboards (Polyglot, code editing, refactoring): run by the Aider maintainer; listed, not counted (0036 cautions 1 and 3) — accessed 2026-10-17
• Aider's own Polyglot and SWE-bench results, and aggregator sites that republish them.

Decision records: 0041; 0051 (grade-A basis check).