Agent Reputation & TrustClearinghouse

Data Interpreter (MetaGPT, open-source)

Category
Data science agent
Type
Open-source agent in the MetaGPT framework (MIT licence), FoundationAgents (formerly DeepWisdom); PyPI metagpt
Source
Project repository
Last reviewed
2026-10-17 (what this means)

Summary

Data Interpreter is an open-source data-science agent in the MetaGPT framework. It plans analysis and machine-learning tasks as a graph and writes and runs Python code step by step. Nine independent studies that ran it were found (two peer-reviewed). Its results depend heavily on the model and task. On InfiAgent-DABench, independent re-runs scored 54-80%, against the 94.93% the developers reported; two papers say they could not reproduce that figure. Results ranged from mid-pack on Kaggle-style tabular tasks to 3.3% on an urban data benchmark. In a security study it had the lowest attack success rate of four open-source data agents, but the authors attribute this to limited functionality rather than stronger defences. Seven CVEs have been published for MetaGPT, including two that affect Data Interpreter directly. The project has had no release since March 2025 and no commits since January 2026. Informational only; not a guarantee, endorsement, or certification.

Overall score: 49 · Evidence grade: B · Independent sources: 9 performance (2 peer-reviewed), 6 other

Evidence grade describes the amount and type of evidence, not the agent's quality. Grade B rather than A: 9 counted sources, but fewer than three are peer-reviewed (see methodology).

Factor breakdown

Factor breakdown
FactorWeightScoreNote
Completion rate30%45InfiAgent-DABench (independent re-runs): 75.78% GPT-4o, 67.70% GPT-4o-mini (DatawiseAgent); 54.09-67.70% pass@1 across four open models (DataMind); 0.716-0.788 with GPT-4o / DeepSeek-V3.1 / o4-mini (EvoDS), below a plain ReAct agent in each case. Developers report 94.93%. KramaBench: 0.256-0.382 (Fail Loudly), 30.34% with Gemini 2.5 Pro (Blackboard). Kaggle/OpenML tabular tasks: competitive, close to AIDE (SPIO). Execution stability 0.672-0.676 (VDSAgents). UrbanDS-Bench 3.3% overall.
Disputes20%35Seven CVEs for MetaGPT. Data Interpreter components: CVE-2026-4516 (write_analysis_code.py, up to 0.8.1) and CVE-2026-19058 (data_interpreter.py, up to 0.8.2). Framework-wide: CVE-2026-0760 and CVE-2026-0761 (ZDI 0-day advisories after no vendor response), CVE-2024-23750, CVE-2026-19059 and CVE-2026-19060. VulDB records that the vendor did not respond for CVE-2026-4516. No GitHub security advisories published. Two independent papers report being unable to reproduce the headline DABench result. No legal or regulatory action found.
Adherence to limits25%45Data Agents Under Attack (NTU, June 2026; DeepSeek-V3.2): mean attack success about 12.9% across 14 techniques (0-56%), the lowest of four open-source data agents. The authors attribute this to limited functionality (frequent failure to connect to the database), not stronger defence. When attacks did connect, Data Interpreter had the largest share of the most severe outcome level (BR4, 9.3%). By design it executes model-written code; no built-in sandbox is documented.
Evidence quality25%70Nine independent sources, two peer-reviewed (EMNLP 2025, ICLR 2026), several from 2026. Fewer than three peer-reviewed, so grade B rather than A (0025 clarification). Most sources are competing-agent papers (disclosed).

0.30 × 45 + 0.20 × 35 + 0.25 × 45 + 0.25 × 70 = 49.25, rounded to 49 (rounding rule).

Scope

The open-source Data Interpreter role in MetaGPT (metagpt/roles/di), run with a language model chosen by the user; MetaGPT releases through 0.8.2 (9 March 2025). MetaGPT's other roles and the commercial MGX / MetaGPT X service are not covered. Results depend heavily on the language model and task, so results are reported as ranges by model.

Version and status note

Data Interpreter is treated as one continuous component from its 2024 release through MetaGPT 0.8.2. The project appears dormant: the last PyPI release was 0.8.2 (9 March 2025) and the last commit to the repository was on 21 January 2026. The repository is not archived.

Disclosures for counted sources

Eight of the nine counted studies propose a competing data-science agent and compare it against Data Interpreter. The ninth, Data Agents Under Attack, is a security study.

Disputes: reproduction gap

The developers report 94.93% on InfiAgent-DABench with GPT-4o. DatawiseAgent (EMNLP 2025) and DataMind (ICLR 2026) report that they could not reproduce this; their re-runs scored 54.09–75.78% depending on the model. This is the papers’ finding; the difference may reflect evaluation settings.

Security

Data Interpreter runs Python code written by the model. Published CVEs (scores as assigned by the named scorer; status as listed by NVD on 17 October 2026):

Data Interpreter components

MetaGPT framework

The project has not published any GitHub security advisories. In the Data Agents Under Attack study, Data Interpreter had the lowest attack success rate of four open-source data agents (about 12.9% on average). The authors attribute this to limited functionality (it often failed to connect to the database) rather than stronger defences. When attacks did connect, it had the largest share of the most severe outcome level (9.3%).

Uncertainties

Sources

Independent performance sources (counted)

Other sources (not counted)

Not counted

Decision record: 0029.