Supplement Studies: How to Read One Without Getting Fooled

Why this matters

Almost every supplement says it is “clinically proven” or “science-backed.” Often that means one tiny, industry-funded study - or an experiment in rats. A few simple questions separate real evidence from marketing.

The stakes are not only financial. US emergency departments handle an estimated 23,000 visits a year for adverse events linked to dietary supplements, about 2,150 of them ending in hospital admission, with weight-loss and “energy” products commonly implicated, particularly among young adults (Geller 2015). Between 2007 and 2016 the FDA identified 776 supplements adulterated with undeclared drugs, mostly sold for sexual enhancement (hidden sildenafil), weight loss (sibutramine, a withdrawn drug) or muscle building (steroids), and some were found adulterated again after a warning (Tucker 2018). Reading the evidence is how you avoid paying for nothing, or worse.

The checklist

1. Was it done in humans? Test-tube and animal results are a starting point, not proof for people. “Shown to fight cancer cells in a dish” tells you almost nothing about your body.

The track record is sobering. A systematic review compared animal experiments with the human trials that followed: corticosteroids helped animals with head injury but not people; the stroke drug tirilazad shrank infarcts in animal models and made patients worse (Perel 2007). A mouse dose scaled to a human is often impossible to eat, and a cell in a dish is bathed in concentrations no supplement reaches in the blood.

2. Was it a randomized controlled trial (RCT)? RCTs - where people are randomly assigned to the supplement or a placebo - are the gold standard. Observational studies (which track what people already do) can only show association, not cause. Most “superfood” headlines come from observational data.

Randomization matters because it spreads the things you cannot measure (motivation, wealth, undiagnosed illness) evenly across groups. Even within RCTs, quality matters: in an analysis of 250 controlled trials, those with inadequate or unclear concealment of the allocation sequence exaggerated treatment effects by 41 and 30 percent, and trials that were not double-blind exaggerated them by 17 percent (Schulz 1995). Look for the words “double-blind” and “placebo-controlled”, and for a description of how people were assigned. For how the observational-versus-trial gap plays out with berries and greens, see superfoods decoded.

3. How many people, and for how long? A 12-person, two-week study is a hint, not a conclusion. Look for larger samples and durations that match the claim. The strongest evidence is a systematic review or meta-analysis that pools many trials.

Small studies do not just miss real effects; when they do find something, it is more likely to be a fluke, and the effect size is usually overestimated (Button 2013). A widely cited analysis laid out when a published finding is least likely to be true: small studies, small effects, many outcomes tested with flexible analysis, financial interest, and many teams chasing the same headline (Ioannidis 2005). Supplement research ticks most of those boxes. Meta-analyses help, but only if the trials going in are decent; check how many participants they pool, whether the results are consistent across trials, and who funded the trials inside them. A creatine memory meta-analysis, for example, drew a published statistical critique after publication.

4. Who paid for it? Industry funding does not automatically invalidate a study, but it is a flag to read more carefully, especially if the funder sells the product.

The Cochrane review on this question (Lundh et al. 2017) pooled 75 papers comparing industry-sponsored with independently sponsored drug and device studies. Industry-sponsored studies reported favourable efficacy results 27 percent more often and favourable conclusions 34 percent more often, and their conclusions agreed with their own results less often. Standard risk-of-bias checklists did not explain the gap; the authors describe an industry bias that those tools do not capture, which is why the funding line matters even when a trial looks well run (Lundh 2017). The funding statement and conflict-of-interest declaration are usually at the end of the paper. Read them first.

5. Was it peer-reviewed, and is the effect meaningful? Check that it appeared in a legitimate journal, and ask whether a “statistically significant” result is actually large enough to matter in real life.

Two additions. First, what you can read is not all that was done. When researchers compared 74 FDA-registered antidepressant trials with the published literature, 94 percent of trials looked positive in journals but only 51 percent were positive in the FDA’s files, and published effect sizes ran 32 percent higher than the complete data (Turner 2008). Across research generally, statistically significant outcomes have 2.2 to 4.7 times the odds of being fully reported, and 40 to 62 percent of trials changed, added or dropped a primary outcome between protocol and publication (Dwan 2013). A trial that was registered before it started, with the outcome it reports matching the outcome it registered, deserves more trust.

Second, size. A supplement that lowers a score by 2 points on a 60-point scale can be “significant” with enough participants and still be unnoticeable to the person taking it. Ask for the absolute change, not the relative one (“halves your risk” may mean from 2 in 1,000 to 1 in 1,000), and compare it with what the same trial reports for placebo. Legitimacy: a journal indexed in PubMed is a reasonable minimum; pay-to-publish outlets with no real review will accept almost anything.

Quick red flags

• A “proprietary blend” that hides doses
• A single study cited as definitive
• Claims to treat or cure a disease (supplements legally cannot)
• Testimonials standing in for trials
• A “clinically studied ingredient” tested at a dose the product does not contain
• Relative percentages with no absolute numbers, or a surrogate marker (a blood value) presented as a health outcome
• A press release or influencer summary standing in for the paper itself

A 60-second read of any study

Find the abstract on PubMed and check five things in order: species (humans?), design (randomized and blinded?), size and length (how many, how long?), funding (who paid?), and the result in absolute terms (how big, on what measure?). If the answers are humans, randomized, hundreds, months, independent, and a change you would notice, the study is worth your attention. If the answers are mice, uncontrolled, twelve, two weeks, the manufacturer, and a 4 percent change in a blood marker, it is marketing with a reference number. The berberine coverage on this site is an example of that filter applied to a supplement with real but overstated evidence.

Where to look

Reliable, plain-language sources include the NIH’s National Center for Complementary and Integrative Health and the FDA’s dietary supplement pages, plus independent reviewers like Cochrane and Examine. The NIH Office of Dietary Supplements publishes fact sheets on individual nutrients and botanicals, and PubMed is free to search for the original papers. And remember: in the U.S., the FDA does not evaluate supplements for safety or effectiveness before they are sold - the burden of proof is on you.

When to involve a clinician

Reading well protects your wallet; it does not replace a conversation with someone who knows your medications. Talk to a clinician or pharmacist before adding a supplement if you take prescription medicines (interactions are common and rarely listed on labels), are pregnant or breastfeeding, have liver or kidney disease, or are considering anything sold for sexual enhancement, weight loss or muscle building, the three categories most often found to contain hidden drugs.

Bottom line

You don’t need a science degree to be a smart consumer - just the habit of asking: human trial or not, randomized or not, how many people, who paid, and is the effect big enough to care about. That filter protects your wallet and your health. Add one more habit, checking whether the trial was registered and reported what it promised, and you are reading more carefully than most headlines are written.