Homeopathy is used around the world, and horoscopes are easy to find in newspapers and phone apps. Homeopathy says that remedies diluted again and again can treat illness. Astrology says the positions of the planets at your birth shape your personality.

Both claims can be tested, and both have been. In a careful test, astrologers did no better than random guessing, and a large review found no reliable evidence that homeopathy works for any health condition. Even so, many people still use both. Claims like these are called pseudoscience, because they look scientific but do not pass scientific tests.

What makes a claim scientific?

A scientific claim makes a prediction that could fail. The philosopher Karl Popper made this idea famous. According to the Stanford Encyclopedia of Philosophy, he argued that a scientific statement must make predictions that real observations could prove wrong [1]. A claim that fits every possible outcome cannot be checked.

Being testable is only the start. The test also has to be fair. In medicine, that means comparing people who get a treatment with similar people who get a placebo, a dummy treatment with no active ingredient. In the best trials, people are put into each group at random. Neither the patients nor the researchers know who got what until the end. This is called a double-blind trial.

Then other teams repeat the work, because one study can be a fluke. Our post on the scientific method shows how that checking works in real cases.

Last, scientists accept the result. When a test goes against an idea, they change the idea or drop it. The encyclopedia describes how the philosopher Thomas Kuhn compared astronomy with astrology. When an astronomer’s prediction failed, it became a problem to solve. In astrology, as the encyclopedia quotes Kuhn, “particular failures did not give rise to research puzzles.” In other words, astrologers did not treat a failed prediction as a problem to study.

What happened when astrology was tested?

In 1985, Shawn Carlson, of the physics department at the University of California, Berkeley, published a test of astrology in Nature [2]. He wanted a test that both scientists and astrologers would accept as fair. A respected astrology group, the National Council for Geocosmic Research, named astrologers to advise on the study. These advisers reviewed the design and suggested changes. In the end, 28 astrologers took part in the test.

Volunteers gave their exact time and place of birth. They also took a standard personality test that produces a profile of traits such as sociability and self-control.

Each astrologer got a few volunteers’ birth charts, usually four. Each chart came with three personality profiles. One profile belonged to that volunteer. The other two were picked at random from other volunteers. The astrologer’s job was to pick the right one.

A random guess would be right 1 time in 3. The advising astrologers predicted they would do much better, getting at least half right. They approved the design and made that prediction in advance. Volunteers were known only by code numbers, so neither the astrologers nor the researcher knew who was who.

The astrologers picked the correct profile 40 times out of 116 tries. That is about 34 out of 100, about the same as random guessing. Carlson wrote that “astrology failed to perform at a level better than chance.” He concluded that the experiment “clearly refutes the astrological hypothesis.”

The study had limits. Volunteers were recruited in the San Francisco Bay Area, and about 70% were college students. Another part of the study asked volunteers to pick their own astrology reading out of three. They could not do it better than chance. However, they also could not pick their own personality profile. For that reason, Carlson said this part of the study could not count against astrology. This was one test of one kind of astrology, done in one way.

What did a large review find about homeopathy?

Australia’s National Health and Medical Research Council (NHMRC) describes homeopathy as resting on two ideas. The first is that a substance that causes symptoms in a healthy person can, in very small doses, treat those symptoms in a sick person. The second is that a highly diluted preparation keeps a “memory” of the original substance.

In 2015, the NHMRC published a review of the evidence [3]. It found 57 systematic reviews, which are studies that gather and judge all the research on a question. Together they covered 176 individual studies. The council only counted studies that compared people given homeopathy with a similar group that was not. Homeopathy groups and members of the public also sent in more studies on two occasions.

Some studies favored homeopathy. However, the council found that they were too small, poorly designed, or both. For some conditions, the evidence showed homeopathy was no better than a placebo. These included asthma, headache and migraine, and colds. The council concluded that “there are no health conditions for which there is reliable evidence that homeopathy is effective.”

Its public statement added a safety warning [4]. People who choose homeopathy “may put their health at risk” if they reject or delay treatments that work. The council advised talking with a registered health practitioner first and keeping up any prescribed treatment. If you use homeopathy, tell your doctor.

The review had limits too. It relied on how earlier reviews described each study, and some studies were probably never published. The council judged that this was unlikely to change the result, because the studies it found were mostly small and of poor quality.

Why don’t personal stories settle the question?

Many people say homeopathy helped them. The NHMRC explained why such reports cannot show that a treatment works. First, people can feel better simply because they believe a treatment works. This is the placebo effect. Second, health workers “cannot always tell” whether a change in someone’s health came from the treatment or from something else.

This is why researchers use fair comparisons. A placebo group shows how much people improve without the real treatment.

What are the warning signs of pseudoscience?

Philosophers have written many lists of warning signs. The Stanford Encyclopedia of Philosophy gives one list that includes these:

• Belief in authority. Some person is said to have a special ability to decide what is true, and others are expected to accept it.
• Results no one can repeat. Other people cannot get the same outcome.
• Handpicked examples. The examples shown are not typical.
• Unwillingness to test. A claim that could be tested never is.
• Ignoring bad news. Evidence against the claim is left out.
• Built-in excuses. The test is arranged so the claim can only pass, never fail.

Tradition is another warning sign. The NHMRC noted that some traditional practices, “despite their longstanding history of use,” may not have been shown to work. Many years of use is not the same as a fair test.

The claim that experts are hiding the truth is another warning sign. The encyclopedia notes that conspiracy theories often rely on circular reasoning. In these theories, evidence against the conspiracy is treated as evidence for it. Once a seller says doctors are hiding a cure, every doctor who disagrees becomes more “proof.”

Sales pitches also call for extra care. The NHMRC warned that many health products “are promoted as beneficial to health when there is little or no evidence.” In my view, selling something is not a problem by itself. The problem starts when a sales pitch is used instead of published, fair tests.

Is the line between science and pseudoscience always clear?

No. Philosophers call the question of where to draw this line the demarcation problem. The encyclopedia entry notes “much more agreement on particular cases of demarcation than on the general criteria.” Most experts agree that astrology and homeopathy are pseudosciences. They disagree on the exact rule that makes them so.

Popper’s own rule runs into this problem. Astrology can be tested, and it has been, by Carlson among others. So being testable is not enough on its own to make a claim scientific. The encyclopedia notes that some defenders of Popper read him as asking for two more things. People must actually run the tests, and they must accept the results.

The encyclopedia also points out that science changes over time, and that established science is not free of mistakes. Some ideas that went against expert opinion later held up when the evidence came in. In 1982, stress and lifestyle were thought to be the main causes of stomach ulcers. Barry Marshall and Robin Warren proposed that a bacterium was involved. According to the Nobel Assembly, they “challenged prevailing dogmas” [5]. Treatment studies by them and others showed that ulcers were cured only when the bacteria were cleared from the stomach. The two men won the 2005 Nobel Prize in Physiology or Medicine for this discovery.

Asking for a fair test is not the same as rejecting new ideas. A fair test gives a new idea a real chance to prove itself.

How can you check a claim yourself?

When you see a claim about health, food or a product, run through this list:

• Look for a prediction that could fail. If no result could ever count against the claim, it cannot be tested.
• Look for a fair comparison. Did people who got the treatment do better than a similar group who got a placebo?
• Check whether others found the same thing. One study is a start. Several independent studies are much stronger.
• Watch how failures are handled. A good field changes its ideas after a failed test. Supporters of a pseudoscience make excuses for the failure.
• Be careful with stories, tradition and secrets. Testimonials, “ancient wisdom” and “hidden cures” are not evidence.
• Ask what the seller can show. A real product should point to published fair tests, not just customer ratings.

Bottom line

A claim earns trust by making a prediction that could fail, passing a fair test, and holding up when others check it. Astrology and homeopathy have both been tested, and neither passed. People still use both.

The line between science and pseudoscience is not always sharp, and even philosophers disagree about where to draw it. For everyday use, ask two questions. What evidence would prove a claim wrong, and has anyone looked for it?

References

[1] S. O. Hansson, “Science and Pseudo-Science,” in The Stanford Encyclopedia of Philosophy (Fall 2023 Edition), E. N. Zalta and U. Nodelman, Eds. First published Sep. 3, 2008; substantive revision May 20, 2021. [Online]. Available: https://plato.stanford.edu/archives/fall2023/entries/pseudo-science/

[2] S. Carlson, “A double-blind test of astrology,” Nature, vol. 318, no. 6045, pp. 419-425, Dec. 1985, doi: 10.1038/318419a0.

[3] National Health and Medical Research Council, “NHMRC Information Paper: Evidence on the effectiveness of homeopathy for treating health conditions,” NHMRC, Canberra, Australia, Ref. CAM02A, Mar. 2015. [Online]. Available: https://web.archive.org/web/20170129113730/https://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cam02a_information_paper.pdf

[4] National Health and Medical Research Council, “NHMRC Statement: Statement on Homeopathy,” NHMRC, Canberra, Australia, Ref. CAM02, Mar. 2015. [Online]. Available: https://web.archive.org/web/20170129113730/https://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cam02_nhmrc_statement_homeopathy.pdf

[5] The Nobel Assembly at Karolinska Institutet, “Press release: The Nobel Prize in Physiology or Medicine 2005,” NobelPrize.org, Oct. 3, 2005. [Online]. Available: https://www.nobelprize.org/prizes/medicine/2005/press-release/