In December 2023, Nature Communications posted a call for papers for a collection called Clinical trials 2024, with submissions open until the last day of 2024. The journal asked for reports of interventional trials, from phase I to phase IV, and it spelled out that this included trials “yielding negative results for which primary end-points were not met.”

That is an unusual thing for a journal to put in writing. Many researchers learn early that a failed experiment is hard to publish. I think the call deserves attention for that reason alone. A good negative result can save other people years of work, a lot of money, and in medicine, the time and goodwill of patients. But I also think we should be honest about which negative results earn that praise. A careful, well powered negative result is a finding. A small study that simply could not tell is something else.

What the journal actually asked for

Alongside the invitation to send negative trials, the collection page says the journal is particularly interested in papers first reporting the primary and secondary outcomes of a trial, and would also consider long-term follow-up of trials already published.

It also makes clear that nothing about the bar was lowered. According to the collection page, “All submissions will be subject to the same peer review process and editorial processes as regular Nature Communications articles.” It adds, fairly, that the journal “cannot guarantee that any individual paper will be included in the desired collection.”

I read this as a signal, not a solution. One journal saying it welcomes trials that missed their endpoint does not undo decades of habit, and a single call for papers tells us nothing about how many such trials were actually submitted or published. Still, researchers decide what to write up partly based on what they think editors want, and here a journal said it in writing.

Where the null results go

The evidence that negative results get lost is not new. In a 2012 paper with the blunt title Negative results are disappearing from most disciplines and countries, Daniele Fanelli looked at over 4,600 papers published across all disciplines between 1990 and 2007 [1]. He counted how often papers that said they had tested a hypothesis reported support for it, and found that this share grew by over 22% over that period [1].

Fanelli was careful about what that trend could mean. He wrote that the patterns support the hypotheses “that research is becoming less pioneering and/or that the objectivity with which results are produced and published is decreasing” [1]. If the second explanation is right, the literature looks more certain than the research behind it. He also warned that “a system that disfavours negative results” might “discourage high-risk projects” [1].

A 2014 study in Science gets closer to where the losses happen. Annie Franco, Neil Malhotra and Gabor Simonovits tracked 221 social science experiments run through a program called Time-sharing Experiments in the Social Sciences, where they could see what was published and what was not [2]. Every study had passed peer review before it was run, so they all cleared “a substantial quality threshold” [2]. Even so, strong results were 40 percentage points more likely to be published than null results, and 60 percentage points more likely to be written up at all [2].

Their summary of where it goes wrong is one short sentence: “Authors do not write up and submit null findings” [2]. I find that more uncomfortable than a story about unfair journals, because it puts part of the problem on people like me.

In medicine, the cost is people

In most fields, a buried null result wastes time and money. In clinical research, it also wastes something harder to replace: the participation of patients who agreed to take on risk so that others could learn from it.

A 2013 analysis in the BMJ looked at 585 large trials, each with at least 500 participants, registered on ClinicalTrials.gov and completed before 2009 [3]. Of those, 171, or 29%, remained unpublished, and those unpublished trials had an estimated 299,763 participants [3]. For 78% of the unpublished trials, no results were posted on the registry either [3]. The authors concluded that many participants “were exposed to the risks of trial participation without the societal benefits that accompany the dissemination of trial results” [3].

Those figures do not tell us how many of the missing trials were negative. But when a treatment fails in a trial and nobody hears about it, the next team may test the same idea, recruit new patients, and learn the same lesson again.

A published negative trial would have let the second team spend its money and its patients’ goodwill on a better question.

Not every negative result is informative

Here is where I want to be careful, because publishing negative results can turn into a slogan. A negative result is only useful if the study was able to find a positive one.

An informative negative result usually comes from a study that was planned properly: a clear main question set out in advance, enough participants to detect an effect of a size that would matter, and methods good enough that the answer can be trusted. If a trial like that finds no benefit, other researchers learn something real.

An underpowered study is different. If a trial is too small, it can miss a real effect simply by chance. Its finding of no difference really means the study could not tell. Publishing it is still better than hiding it, since its data can be pooled with others later. But it should be described honestly as inconclusive, not sold as proof that something does not work.

The Nature Communications call kept the same peer review as for any other paper in the journal, and it named primary endpoints specifically: the main question a trial is built to answer. That is the kind of framing that separates a real negative finding from a study that simply ran out of steam.

Making a negative result count

My PhD dissertation was on Protein L functionalized microparticles for molecular imaging of early colorectal lesions. Experimental science in general is a reminder that many ideas do not pan out, and that knowing why something failed can be as useful as the thing that worked.

If you run experiments or trials, plan them so that a negative answer would mean something, and then write it up when it comes. If you review or edit, judge a negative paper on its design, not its outcome. And if you read science news, be a little suspicious of a field where everything seems to work. Somewhere in a file drawer, someone may already have the answer to the question you are about to ask.

References

[1] D. Fanelli, “Negative results are disappearing from most disciplines and countries,” Scientometrics, vol. 90, no. 3, pp. 891-904, Mar. 2012, doi: 10.1007/s11192-011-0494-7.

[2] A. Franco, N. Malhotra, and G. Simonovits, “Publication bias in the social sciences: Unlocking the file drawer,” Science, vol. 345, no. 6203, pp. 1502-1505, Sep. 2014, doi: 10.1126/science.1255484.

[3] C. W. Jones, L. Handler, K. E. Crowell, L. G. Keil, M. A. Weaver, and T. F. Platts-Mills, “Non-publication of large randomized clinical trials: cross sectional analysis,” BMJ, vol. 347, Art. no. f6104, Oct. 2013, doi: 10.1136/bmj.f6104.