In March 2026, the journal Nature published a paper about a system called The AI Scientist [1]. The team came from Sakana AI, the University of Oxford, the University of British Columbia and the Vector Institute. Their system comes up with research ideas, writes code and runs experiments. It also studies the data, writes the whole paper and then reviews its own work.
The headline result was striking. One of its papers scored high enough with human reviewers to be accepted at a workshop. That workshop was held alongside a major machine learning conference.
To be clear, Nature did not publish a paper written by AI. It published a paper by people about an AI system. The AI’s own paper went to the workshop, where it scored high enough to be accepted. Then the team withdrew it, as they had planned from the start.
In my view, this is a real first for AI. However, it was one paper at one workshop, judged by a few reviewers in a single round of review. Being accepted there is not the same as passing review at a major journal.
What did the AI do, and what did people do?
The system works on machine learning research, because those experiments run entirely on a computer [1]. For this test, the team gave it the broad theme of one workshop. The workshop was part of the International Conference on Learning Representations (ICLR) 2025. It was called I Can’t Believe It’s Not Better (ICBINB), and it is about where deep learning ideas fall short. Deep learning is a kind of AI that learns patterns from large amounts of data.
From there, the authors say, the whole process ran without any human changes. That covered everything from the ideas and code to the finished paper.
Diagram by Better Science, based on Lu et al., Nature (2026), doi:10.1038/s41586-026-10265-5.
People were still involved, though. The authors write that “we manually filtered the most promising outputs at each stage”. They chose three papers using three tests. Did the idea fit the workshop? Did the code run and do what the paper said? Was the formatting right?
In Sakana AI’s March 2025 announcement, the team put it plainly. They gave the system a broad topic and “picked 3 AI-generated papers to submit.”
The team is open about this filtering. The authors argue that without it, the same papers would still exist. There would just be more weak ones beside them, and the cost would be higher. That is a fair point. Still, a person who picks the best few papers out of many is doing part of a scientist’s job. Scientists judge their own work before they submit it.
Imagine a student who has an AI tool write twenty essays overnight. She reads through them and hands in the best three. If one earns a good grade, the tool did the writing. But her choice of which three to hand in did part of the work too.
The Nature paper reports all of this openly. The risk is that a one-line headline leaves all of this out.
What did passing actually mean?
Three AI papers went into a pool of 43 submissions [1]. Reviewers were told that some papers were written by AI, but not which ones. One paper reported a negative result about a way to help neural networks work well on data they have not seen before (Sakana AI). It received scores of 6, 7 and 6, an average of 6.33. The other two did not score high enough.
That paper never received a final decision. The team had decided in advance to withdraw every AI paper after review, whatever the outcome. They did not want to set a precedent before researchers agree on rules for disclosing and judging such work. The organizers said it would very likely have been accepted.
The announcement adds one more detail. Because of the planned withdrawal, the organizers skipped the meta-review. That is the final step, where someone weighs all the reviews together. So in theory, the paper could still have been rejected.
Where the paper was reviewed also matters. The workshop accepted about 70% of papers, or about 7 in 10. The main ICLR 2025 conference accepted 32%, about 1 in 3. The authors are honest about this. Their system, they write, “cannot yet meet the standards of top-tier publications nor even do so consistently for workshops”. Their own researchers reviewed all three papers. They concluded that none met the bar for the main conference.
What can a workshop review catch?
This is the part I find most interesting. The developers list the system’s common failures themselves [1]. These include overly simple ideas and code that carries out the main idea wrongly. They also include weak methods and made-up details, such as wrong citations.
In the announcement, they give an example. The system credited a well-known type of neural network to the wrong authors. Most of these are exactly the problems a reviewer can easily miss. That is especially true for someone who reads a paper once, without the code.
I do not see that as a scandal. Workshops are meant to be a lighter venue. They are a place to share early, rough and even negative results. With seven in ten papers accepted, reviewers mostly ask whether a paper is worth discussing. They are not checking whether every claim in it is correct.
The Sakana team also points out another gap. At top conferences and journals, reviewers and authors go back and forth, and that improves papers. That exchange is not part of the workshop process.
Mostly, the experiment tested whether a machine-made paper can pass as solid research to three reviewers in one reading. That is a real ability. But it only tests what a paper looks like on the surface. That means how the question is presented, the writing, and charts and experiments that look sensible.
In my view, peer review is rarely set up to check whether the code does what the text claims. Papers by human authors can have the same hidden problems and still get through.
Why is it still a milestone?
I still call it a milestone because, starting from a broad topic, a machine produced a complete study. In the team’s words, it scored “higher than many other accepted human-written papers at the workshop,” according to Sakana AI.
The authors also report steady progress [1]. Their own automated reviewer judged the papers better as the underlying language models improved. If that trend continues, the AI may soon reach the level needed for main conferences too.
The authors name the downside themselves. Systems like this could overload review systems and add noise to the scientific record. To me, this is the most important point. This system got one paper in three past a workshop that accepted 70% of papers. It can also be run again and again. That could put real pressure on reviewers who are already overworked.
I build and test deep-learning models myself. I know that a single table of results can hide many choices that a reviewer rarely sees.
The AI Scientist showed that a machine can write a paper that clears a light, single-round review. It did not show that the paper was right. It also reminds us that peer review was never designed to prove that a paper is right.
I would like to see a few changes. Venues should ask for code with every paper, human or AI, and some reviewers should actually run it. Clear rules on disclosing AI-written work should be settled before the next experiment. The authors say they withdrew their paper because researchers have not yet agreed on such rules.
And when you see a headline saying an AI paper passed peer review, check three things. Which venue was it? Who chose what got submitted? Was it actually published? In this case, it was a workshop, humans chose the papers, and the paper was not published.
References
[1] C. Lu, C. Lu, R. T. Lange, Y. Yamada, S. Hu, J. Foerster, et al., “Towards end-to-end automation of AI research,” Nature, vol. 651, no. 8107, pp. 914-919, Mar. 2026, doi: 10.1038/s41586-026-10265-5.
Related Articles
