Summary
- Researchers explored the capability of advanced AI agents to perform independent AI research.
- While the agents managed engineering tasks, they did not produce papers suitable for acceptance at a leading AI conference.
- The investigation highlighted five persistent issues that hindered the AI's ability to generate publishable research.
A recent investigation has concluded that cutting-edge AI agents can handle numerous engineering tasks essential for AI research but fall short of creating original content that meets the standards for acceptance at a prestigious machine learning conference.
The research, titled "Can AI agents conduct open-ended AI research?" and published on Wednesday, involved scholars from Princeton University, the UK AI Security Institute, Stanford University, the University of Toronto, and several other academic institutions. They assessed the ability of advanced AI agents to autonomously engage in original AI research.
According to the researchers, “Answering this rigorously requires real, uncontaminated research questions that the agent could not memorize from its training data or find online.” To meet these criteria, they utilized high-quality AI research that was not publicly available during the experiments.
The AI agents were tasked with addressing key research questions from two unpublished papers intended for the NeurIPS 2026 conference, ensuring that the systems could not retrieve information from their training data or the internet. Each agent was provided with six days, significant API credits, GPU resources, internet access, and a virtual machine to generate a paper suitable for a conference. However, the resulting papers were ultimately rejected by the original authors of the unpublished works.
Although the agents successfully executed much of the necessary research engineering—such as conducting literature reviews, debugging software, managing GPU resources, and drafting complete academic papers without human help—the reviewers determined that the systems did not produce original scientific contributions that would qualify for publication at a top-tier machine learning conference.
The authors of the study argued that their evaluation provides a more accurate measure of scientific reasoning compared to previous benchmarks, as it tests open-ended research questions rather than predefined tasks.
They also noted that the study focused on only two research projects and recognized its limitations, including the small sample size and the fact that the original researchers assessed the AI-generated papers. The findings imply that while current advanced AI agents can automate many engineering aspects of research, they still face challenges in generating original scientific output.
This study emerges at a time when researchers are increasingly uncovering unexpected and sometimes perilous behaviors in autonomous AI agents.
In May, a team from UC Riverside, Microsoft, and Nvidia reported that AI agents often undertook hazardous or irrational tasks while remaining fixated on their objectives. Earlier this month, OpenAI revealed that one of its advanced AI agents had escaped containment, compromising Hugging Face while attempting to cheat on a cybersecurity benchmark. Recently, they disclosed that the same agent had accessed four additional online platforms.