In September 2024, three Stanford researchers, Chenglei Si, Diyi Yang and Tatsunori Hashimoto, posted a study with a bold question in its title: can large language models (LLMs), the technology behind chatbots, generate novel research ideas [1]? They recruited over 100 researchers in natural language processing (NLP), the field that builds language software, to write ideas and to blind review ideas written by both people and an AI system [1]. The paper was later published at ICLR 2025, a major machine learning conference [1].
The headline result was that the AI’s ideas were judged more novel than the human experts’ ideas, while being judged slightly weaker on feasibility [1].
That is a careful and useful study. It is also easy to overread. None of these ideas were carried out as full research projects in this study. The study measured how experts rated written proposals, not whether any of them led to a discovery. My view is simple: an AI that writes novel ideas has done the easy part. The hard part of science is deciding which idea deserves time and money, and then turning it into evidence.
How the study worked
The idea writers were published NLP researchers (the preprint set a minimum of one paper at a major AI venue), and each was paid $300 to write one idea within 10 days [1]. On the other side was an AI agent built on Claude 3.5 Sonnet, which searched the literature, generated 4,000 seed ideas per topic, and ranked them [1].
To keep the comparison fair, both sides wrote about the same seven topics, such as bias, coding, safety and factuality, all within prompting research, which means finding better ways to instruct a language model [1]. Every proposal followed the same template, and an LLM rewrote all of them into the same style so reviewers could not tell who wrote what by tone alone [1].
Reviewers then scored each idea from 1 to 10 on novelty, excitement, feasibility, expected effectiveness and an overall score, with a written reason for each [1]. There was also a third group of ideas: AI ideas that the first author picked by hand instead of trusting the AI’s own ranking [1].
What the reviewers found
On novelty, the AI ideas averaged 5.64 and the hand-picked AI ideas 5.81, against 4.84 for the human ideas [1]. That gap held up across three different ways of running the statistics [1].
Feasibility went the other way, a little. Human ideas averaged 6.61 and AI ideas 6.34, a difference too small to count as real in this study [1]. In their written comments, reviewers flagged some AI ideas as too demanding, such as one that called for fine-tuning a model with 176 billion parameters, which one reviewer said would need a lot of specialized computer chips [1].
One more detail stood out to me. In the September 2024 preprint, the authors found that reviewers’ overall scores tracked novelty and excitement closely, but had almost no correlation with feasibility [1]. In other words, the reviewers mostly rewarded ideas that sounded fresh and exciting. An idea’s feasibility score told you almost nothing about its overall score.
Imagine a bakery contest judged only on the written recipe. The judges love a cardamom and miso tart because nobody has tried it before. No one bakes it, so no one learns that the crust collapses in the oven.
The ratings in this study are the recipe stage. That is worth knowing, and it is still a long way from the tasting.
What the authors themselves warn about
The authors are open about the limits, and I think they are the most interesting part of the paper.
First, the human ideas may not have been the experts’ best. Of the 49 writers, 37 came up with their idea on the spot, and on average they placed it around the top 43% of their own past ideas [1]. The authors call these “likely the median-level ideas” of the experts [1].
Second, the AI ran out of variety. As it kept generating, it mostly repeated itself: of 4,000 seed ideas per topic, only 200 were unique [1]. The authors also warn that wide use of LLMs could lead to idea homogenization, where generated ideas reflect only a narrow set of perspectives [1].
Third, the AI could not judge ideas well. When the authors tested AI systems as reviewers, the best one agreed with human judgments 53.3% of the time on a task where chance is 50% [1]. The human reviewers were not much better. Their agreement with each other was 56.1%, lower than for full conference papers, which the authors put down to judging ideas “without seeing the actual experiment results” [1].
Finally, the scope was narrow: prompting research in NLP [1]. The authors write that conclusions in other fields could be different [1].
Novel is not the same as worth testing
Put those caveats together and a pattern appears. Generating ideas was cheap for the AI. Judging them was hard for everyone, machine and human alike. That is the part of the work I care most about.
The authors say it plainly: “Ideas that sound novel and exciting might not necessarily turn into successful projects” [1]. They call their study “a preliminary evaluation of AI-generated ideas” and planned a next phase in which researchers would carry out some of the ideas as full projects [1].
They also tried letting an AI agent run the experiments itself. The code ran, but the results could mislead, because the agent “often skips or modifies steps” in the methods [1]. That is the gap between an idea and evidence in one sentence.
Other fields show how wide that gap can be. In 2015, a large team repeated 100 published psychology studies. Ninety-seven percent of the originals had statistically significant results, but only 36% of the repeats did [2]. Those were not untested ideas. They were published findings, and many still did not hold up. A study using betting markets on 44 of those replications went further. It estimated that the hypotheses being tested in psychology typically had a low chance of being true before testing, with a median of 9% [3]. That figure is an estimate built from traders’ bets, so I would not treat it as exact. The direction is what matters: in that field, most hypotheses that reached testing were unlikely to be true from the start.
What I would do with an AI idea
As an AI researcher who builds large language model tools that extract structured information from medical records, I think a fluent output should be treated as a draft to be checked, not an answer.
An AI that can write a novel research idea has solved the cheap part of science. The expensive part is choosing which idea to test and then testing it honestly, and that is still our job.
So if you use AI to brainstorm, use it for what it seems good at: a wide first pass. Then ask the questions a written pitch cannot answer: whether you can actually run it with the time and tools you have, what result would prove it wrong, and what it would cost to find out. A novel idea that nobody can test is a nice sentence. An ordinary idea tested well can change what we know.
References
[1] C. Si, D. Yang, and T. Hashimoto, “Can LLMs generate novel research ideas? A large-scale human study with 100+ NLP researchers,” arXiv:2409.04109, Sep. 2024; published in Proc. Int. Conf. Learn. Represent. (ICLR), 2025, doi: 10.48550/arXiv.2409.04109.
[2] Open Science Collaboration, “Estimating the reproducibility of psychological science,” Science, vol. 349, no. 6251, Art. no. aac4716, Aug. 2015, doi: 10.1126/science.aac4716.
[3] A. Dreber, T. Pfeiffer, J. Almenberg, S. Isaksson, B. Wilson, Y. Chen, et al., “Using prediction markets to estimate the reproducibility of scientific research,” Proc. Natl. Acad. Sci. USA, vol. 112, no. 50, pp. 15343-15347, Dec. 2015, doi: 10.1073/pnas.1516179112.
Related Articles
