AutoKaggle (open-source)
- Category
- Data science agent
- Type
- Open-source multi-agent framework (Apache-2.0) for Kaggle-style data-science competitions from the Multimodal Art Projection (M-A-P) research community; GitHub multimodal-art-projection/AutoKaggle
- Source
- Project repository
- Last reviewed
- 2026-10-19 (what this means)
Summary
AutoKaggle is an open-source research framework in which several AI agents work through a fixed sequence of data-science steps for tabular Kaggle competitions. Two independent studies that ran it were found, below the three needed for a score. In one, AutoKaggle produced valid outputs in 53-58% of trials across nine datasets, behind two other systems. In the other, its accuracy was close to other baselines. Its developers' own results are not counted. The code has not been updated since December 2024. Informational only; not a guarantee, endorsement, or certification.
Status: Insufficient evidence — no overall score is given (fewer than three independent performance sources). Independent performance sources: 2 (both peer-reviewed).
What the evidence shows
- Completion: VDSAgents (Stat, 2026): valid-output rate 0.577 (DeepSeek-V3) and 0.534 (GPT-4o) across nine datasets, below VDSAgents (0.894, 0.950) and Data Interpreter (0.676, 0.672). SPIO (ACL 2026): accuracy close to other baselines (for example Titanic 0.7781 with GPT-4o). Developer results (valid submission rate 0.85, comprehensive score 0.82 on 8 competitions) are not counted. A re-implementation of the AutoKaggle workflow inside another harness (DS-Lighting, preprint, not counted) averaged 0.78-0.81 on DABench and was the slowest of the agents timed (22.6 minutes per task).
- Disputes: No CVE, legal or regulatory action found. Not maintained since December 2024.
- Adherence to limits: No independent study of AutoKaggle's adherence to limits was found.
- Evidence: Two counted independent sources, both peer-reviewed and both proposing competing methods; below the three-source minimum. Not scored.
Scope
The open-source AutoKaggle framework as released in October-December 2024 (no tagged releases; last commit 4 December 2024). AIDE, a separate project from different developers, has its own entry.
Uncertainties
- Semantic Scholar citation check (19 Oct 2026, Decision 0077) covered 86 citing papers; Tree-Notebook (Findings ACL 2026) and DATAEXPLAINER (SBBD 2026) describe AutoKaggle only. Not read in full: PACE (KDD 2026), AutoDS-Tools (AAMAS 2026), AutoMind (IEEE Access), PiML, CodeViz, AutoCluster, a Nature paper and a bioRxiv benchmark.
- Most papers that cite AutoKaggle only describe it; few run it.
Sources
Independent performance sources (counted)
- VDSAgents (Stat, Wiley, 2026; arXiv 2510.24339): AutoKaggle valid-output rate 0.577 (DeepSeek-V3) and 0.534 (GPT-4o) across nine datasets, below VDSAgents and Data Interpreter. Disclosure: proposes a competing method — accessed 2026-10-17
- SPIO (ACL 2026): AutoKaggle baseline accuracy close to other baselines on tabular tasks (Titanic 0.7781 with GPT-4o), below SPIO. Disclosure: proposes a competing method — accessed 2026-10-17
Other sources (not counted)
- DS-Lighting (preprint, Aug 2026; HKUST-Guangzhou): re-implements the AutoKaggle workflow inside its own harness; DABench overall average 0.78 and 0.81 with two models; 123.1K tokens per task (24 times ReAct) and 22.6 minutes per task, the slowest of the agents timed. Re-implementation, not the released code, so not counted (Decision 0042) — other (re-implementation), accessed 2026-10-19
- LightAutoDS-Tab (Sber AI Lab / ITMO; AAMAS 2026 demo): includes AutoKaggle figures copied from an earlier paper; not a new test, so not counted — other (copied figures), accessed 2026-10-19