In October 2022, Emma Henderson and Christopher Chambers published a short guide in PLOS Computational Biology called “Ten simple rules for writing a Registered Report” [1]. It reads like a practical checklist, and it is one. But the format it describes is a quiet challenge to how most science gets published.

Here is the idea. In a Registered Report, you send a journal your question and your plan before you collect any data. Reviewers judge the plan. If it passes, the journal commits to publishing the final paper whatever the results turn out to be, as long as you follow the approved plan [1]. At the time the guide was published, the format was offered in more than 300 journals [1].

My view is simple: for many studies that set out to test a clear idea, this is a better deal than the usual system, for researchers and for the rest of us. It is not the right format for everything, and I will get to that.

How a Registered Report works

The process has two stages. At Stage 1, you write the introduction and methods: what you want to know, why it matters, how you will collect the data, and how you will analyze it. Editors and reviewers “evaluate the empirical validity of your research question(s)” and the soundness of the design [1]. This review “occurs when it matters most,” before the main study is run, at a time when you can still fix the plan [1].

If the plan passes, you get what is called in-principle acceptance. In the guide’s words, it “commits the journal or platform to publishing the final research regardless of the outcome” [1]. The decision to publish is “taken pre-study and results-blind” [1].

Then you run the study and write up the results. At Stage 2, reviewers check the finished paper against the plan they approved [1].

The promise has conditions

It is easy to misread in-principle acceptance as a free pass. It is not. The promise covers the outcome, whether your hypothesis is supported or not. It does not cover a study that was done differently from the plan.

The guide is clear on this. If something forces a change to your methods after Stage 1, you have to report it as a deviation, and for a substantial change you should seek the editor’s approval straight away [1]. Failing to do so “could result in the Stage 2 manuscript being rejected (when reviewers are focused on assessing adherence to the Stage 1 plans)” [1]. And in rare, severe cases, when the outcome-neutral checks and all other critical quality checks fail, a paper can be rejected at Stage 2 [1].

I like this balance. The journal takes the risk of a boring or disappointing result. The researcher takes the responsibility of doing what they said they would do.

Why the usual deal is lopsided

In the standard system, reviewers see the results first. A clean, positive finding is easier to publish than a null one, and everyone knows it. That pressure shapes which studies get written up and how analyses get chosen.

A 2021 study by Anne Scheel, Mitchell Schijen and Daniël Lakens put a number on the gap in psychology. They compared 71 published Registered Reports with a random sample of 152 standard papers that tested a hypothesis [2]. In the standard papers, 96% reported support for the first hypothesis. In Registered Reports, only 44% did [2].

Nearly all predictions coming true should make anyone suspicious. The authors suggest that a plausible factor is less publication bias and/or fewer inflated false positives in the Registered Report literature [2]. Their conclusion is striking: “an unbiased literature would look very different from the existing body of published research” [2].

There is also some evidence on quality. In a study by Courtney Soderberg and colleagues, 353 researchers peer reviewed papers from 29 published Registered Reports in psychology and neuroscience and 57 comparison papers [3]. Registered Reports scored numerically higher on all 19 criteria, with sizeable improvements in the rigor of methods and analysis and in overall paper quality, and no clear difference in novelty or creativity [3].

This way of thinking feels familiar from my own work. I build deep-learning models for breast MRI, including the training and testing pipelines, and in my view deciding how a model will be tested before you see how it performs is the same basic idea as a Registered Report.

The fair case against

The evidence is not airtight. Scheel and colleagues say plainly that their study “was not an experiment” and “precludes strong causal inferences” [2]. Registered Reports may also be used for particularly risky hypotheses, a possible confound that could lower the positive rate by itself [2]. The Soderberg comparison included only 29 Registered Reports, and its authors raise the worry that the format could have “unintended costs such as reducing novelty” [3].

The format also does not fit every kind of research. Most published Registered Reports report confirmatory, experimental work [1]. Exploratory research, where you do not yet know what question to ask, is a poor match for a locked plan. The format does allow exploratory analyses at Stage 2, as long as they are clearly labeled as exploratory [1]. That is useful, but it is a side door, not the main entrance.

Time is a real cost too. The methods section is often longer than in a traditional paper [1], and the guide warns authors to factor ethics approval into their timescale if it comes after in-principle acceptance [1]. In a fast-moving field, waiting for Stage 1 review before starting may feel like losing ground to groups who just start.

I accept all of this. My point is narrower: when the main goal is to test a clear question, the benefits usually outweigh the costs.

What I would do differently

If you are planning a study that tests a specific prediction, check whether your target journal offers Registered Reports, and take the idea seriously. The up-front work is real, but it is work you should do anyway. A plan that survives review before data exist is a better plan.

For readers, one small habit helps. When a study says it was a Registered Report, it means the question and methods were judged before anyone knew the answer. That does not make the result true, but it does tell you the result was not chosen because it looked good. That is a rare and valuable thing to know about a paper.

References

[1] E. L. Henderson and C. D. Chambers, “Ten simple rules for writing a Registered Report,” PLOS Computational Biology, vol. 18, no. 10, Art. no. e1010571, Oct. 2022, doi: 10.1371/journal.pcbi.1010571.

[2] A. M. Scheel, M. R. M. J. Schijen, and D. Lakens, “An excess of positive results: Comparing the standard psychology literature with Registered Reports,” Advances in Methods and Practices in Psychological Science, vol. 4, no. 2, Apr. 2021, doi: 10.1177/25152459211007467.

[3] C. K. Soderberg, T. M. Errington, S. R. Schiavone, J. Bottesini, F. Singleton Thorn, S. Vazire, et al., “Initial evidence of research quality of registered reports compared with the standard publishing model,” Nature Human Behaviour, vol. 5, no. 8, pp. 990-997, Aug. 2021, doi: 10.1038/s41562-021-01142-4.