“Evidence-based.” “Clinically-proven.” “Science-backed.” Oh, how our industry loves to chant these mantras.
 
And with good reason. Solid science engenders trust and signifies credibility, both of which are vital to the value propositions forwarded by branded ingredient companies and finished product marketers alike.
 
But just as integrity of the raw materials supply chain depends on the right people doing the right things the right way at each point along the path, so too does the scientific value chain depend on unwavering integrity of all participants. From the lead investigators who design the trials, to the research assistants and post-doctoral fellows who crunch the data, to the practitioners who develop therapeutic plans for patients, and marketing directors who craft ingredient or product claims, honesty and competence are essential.
 
We put our trust in the premise that most clinical researchers run their studies ethically and honestly, and that the editors and peer-review boards of the medical and scientific journals carefully scrutinize the thousands of papers they receive, that they’re sifting out the garbage and only publishing the reports that pass statistical and ethical muster. 
 
We’d like to believe that people within the medical and scientific communities are able to balance critical thinking with open-mindedness, and that they’re able to account for their own biases when assessing published research and formulating clinical practice guidelines. Likewise, we hope that marketers are able to balance their drive for media impact and sales growth with a respect for truth in advertising.
 
That’s how it would be in an ideal world. But let’s face it: this is not an ideal world.

Fraud on the Rise

It’s an open secret within healthcare that research fraud is rampant. This is true in many areas of medicine, but it is especially problematic in the field of nutrition.
 
Alan Gaby, MD, is a holistic physician and author of the massive textbook, Nutritional Medicine. The book contains nearly 17,000 research citations, covering the use of herbs and nutraceuticals for more than 400 specific health conditions. Nutritional Medicine is the most comprehensive compendium on the subject currently available. Suffice to say, Dr. Gaby has probably read more clinical research papers than anyone on the planet.
 
He contends that the problem of research fraud has reached epidemic proportions.
 
“Over the past 50 years, I’ve probably reviewed about 50,000 papers. And about 15 years ago, I became aware of some irregularities in a lot of the research. A growing number of published papers left me wondering if the research had actually been done at all, or if the data were simply fabricated.”
 
Gaby said the number of suspicious­—or at least highly questionable—papers has surged dramatically in recent years, in part due to the growth of the open access publishing model and the proliferation of small, poorly refereed open-access journals and websites, some of which operate on a pay-for-play basis. But open-access is only part of the problem. Gaby said he’s seen numerous instances in which very questionable studies have been published in venerable, “high impact,” (i.e., widely-cited) conventional medical journals.
 
“Several hundred papers per year, in my view, raise questions about whether the research is legitimate,” said Gaby. He added that it is difficult—and time consuming—to prove definitively that a scientific report is fraudulent. But there are an alarming number of studies that simply do not hold up to careful scrutiny. When this happens in respected peer-reviewed journals, as it sometimes does, it suggests that peer reviewers are asleep at the wheel, or that they’re turning a blind eye to shoddy methodology and statistical flim-flam.
 
Gaby published his concerns in an excellent 2022 article in Integrative Medicine: A Clinician’s Journal, and in lectures and webinars.

A Legacy of Deceit

He is by no means the only physician ringing alarm bells about fraudulent research. A team of researchers based in New Zealand called out two prominent Japanese investigators—Yoshihiro Sato and Jun Iwamoto—claiming that nearly 300 papers published by the pair in 78 medical journals had major methodological flaws, signs of fabrication, and ethical lapses.
 
Sato, who died in 2017, and Iwamoto both held prominent academic positions at Japanese universities, and their work was primarily focused on bone metabolism. They published many studies looking at the effects of vitamin D, vitamin K, and folate. They also studied prescription therapies like methylprednisolone, hormone replacement therapy, and valproic acid, and some of their research extended into fields like neurology and gastroenterology.
 
Over the decades, their papers appeared in some of the world’s leading journals, including the Journal of the American Medical Association (JAMA), Neurology, and the Journal of Bone and Mineral Metabolism.
 
Biochemist Alison Avenell,the chair of health services research at the University of Aberdeen, Scotland, was the first to smell a rat in the research published by Sato. Back in 2006, Avenell was delving into the question of whether vitamin D could reduce the risk of bone fractures. While plumbing the medical literature, she came across two studies by Sato. One involved a cohort of stroke patients, and the other involved patients with Parkinson’s.
 
In reading the papers, Avenell noticed that in both studies, the patient populations had exactly the same mean body mass indexes. That, she thought, was highly unlikely from a statistical viewpoint. She started digging more deeply, and the more she looked, the more anomalies she found: unreasonably large treatment effects, unusually large patient populations relative to the number of investigators and clinic sites, plagiarized text, numbers that simply didn’t add up.
 
Soon after, Avenell teamed up with Andrew Grey, Mark Bolland, and Greg Gamble of the University of Auckland, New Zealand. The team undertook an exhaustive review of 292 papers published by Sato over many years, some of which were collaborations with Iwamoto.
 
In 2016, this intrepid team published an in-depth takedown of 33 studies by Sato, Iwamoto, or both. They notified 78 journals that most if not all of the nearly 300 papers published by these two researchers were flawed at best, utterly fraudulent at worst.
 
The saga, well chronicled by the American Association for the Advancement of Science’s journal, Science, ultimately resulted in retractions of 121 studies, three corrections, and 12 “editorial expressions of concern”—a polite term journal editors use when they question the validity of a study but don’t feel they have a solid enough case to retract it.
 
Not long before he died, Yoshihiro Sato admitted that he falsified his research, and absolved Jun Iwamoto of any direct responsibility.
 
The “Sato Affair” has been called one of the biggest legacies of medical research fraud in history. According to Retraction Watch, a website founded by former Medscape VP Ivan Oransky and science writer Adam Marcus that monitors retractions of research papers across a vast range of scientific disciplines, Sato and Iwamoto hold the fourth and sixth places for highest number of papers retracted worldwide.

Who’s number one? That dubious honor belongs to Joachim Boldt, a German anesthesiologist and intensive care specialist, who had a stunning 194 of his published papers retracted for data fabrication and lack of ethics board approval for his studies.

Critical Thinking and Red Flags

With fraudulent research on the rise across the medical landscape, and peer review boards apparently unable or unwilling to detect it, all of us who read clinical studies need to sharpen our critical thinking skills. But you don’t have to become a statistician.
 
Over the years, Dr. Gaby has identified 10 red-flag warning signs that should raise one’s index of suspicion about scientific misconduct or outright fraud:
 

• Implausibly prolific research output by a single researcher: A good clinical researcher can complete and publish 3-4 randomized, double-blind, placebo-controlled trials in a period of 5 or 10 years. Yet a careful look at the nutrition and herbal medicine literature shows that some researchers publish 10, 20, or even 30 papers in that time span. “Whenever you see implausibly large research output, it makes you wonder how could they have possibly done all of that research.”

• Implausibly large number of study participants: Gaby said that over time, people who read a lot of clinical studies develop a good sense of how many people could be reasonably enrolled in a given trial. This is based in part on the number of researchers and clinics involved in the trial, the size of those clinics and their catchment areas, the general prevalence of the disease or healthcare condition in question, and the stringency of the patient inclusion/exclusion criteria. In some nutrition studies, a lone researcher claims to have a trial population far larger than one could reasonably expect even in a multi-center study, let alone a study done at a single clinic.

• Unusually short recruitment time: Recruiting patients for legitimate clinical studies is not easy, nor is it swift. It takes a lot of outreach, a lot of effort, and a lot of resources. If a study claims to have recruited 100 people with a particular disorder in a 3-month period, and they all met a set of stringent inclusion criteria, you should be suspicious.

• A paper rapidly submitted for publication: Dr. Gaby said he sometimes sees studies in the nutritional literature that were submitted for publication very soon after completion—or even before it would be possible to complete the trial based on the schedule described in the paper. Most studies disclose what the time period was for patient recruitment, and how long the study lasted. From that, you can estimate the earliest possible date of completion. “In some cases, we’ve seen papers that were submitted for publication before it was possible to have completed the trial; in many other cases, only a few weeks to a month after the trial could have been completed. That’s also implausible, because in a real study it takes a very long time to analyze data, to write the paper, and then to submit it.”

• An RCT done before there is preliminary evidence of efficacy in humans: Double-blind, randomized trials are costly. Few funding sources are likely to take a chance and underwrite the cost of a trial without some compelling preliminary evidence from case reports, open-label uncontrolled trials, or pilot studies to suggest that the intervention in question may be beneficial. Yet, in the nutrition and botanical literature, there are many alleged RCTs conducted without any such preliminary evidence.

• Effect sizes larger than one would expect from nutrients: “If you read a lot of medical literature, you start to get a general idea of how effective nutrients are. Sometimes it’s dramatic, but most of the time it’s not. Usually, it’s a combination of nutrients producing a moderate effect,” said Dr. Gaby. Yet, “in many of the studies I’ve looked at, there were much larger effects ... effect sizes you usually only get from drugs. So that raises eyebrows.”

• No funding source is listed, or the study is described as “self-funded”: This is particularly important when researchers describe their study as RCTs. Real randomized controlled trials are very expensive. If nobody is funding it, one has to wonder how it was possible to do the study. And if funding sources are not openly stated, one needs to wonder why.

• The trial design raises ethical issues: If a study involves subjects who are very sick with advanced disease, and they are randomized to receive either a nutrient or a placebo, there’s likely an ethical problem. That’s because clinical research is still in the domain of patient care, and it is incumbent upon the doctors to treat people with the best available known therapies. Whether natural medicine advocates like it or not—the “best available” treatment for serious diseases—are usually prescription drugs. If clinical researchers are intentionally withholding a viable drug treatment option in order to test a particular nutrient against a placebo, they’re treading on shaky ethical ground. That should raise the index of suspicion in anyone reading and reviewing the study.

• Implausible patient baseline characteristics: Pay close attention to the stated inclusion and exclusion criteria, especially the age range of the patients and their baseline characteristics. Dr. Gaby said he sometimes sees papers that indicate a particular age range for inclusion, but then when he looks at the mean ages and the standard deviations in the results data, it would be mathematically impossible that all the study participants actually met the stated age criteria for inclusion.

• A large study—especially an RCT—conducted by a graduate student: Grad students are the unsung workhorses of clinical research. While they certainly do deserve credit for their efforts, the reality of academic hierarchies is that they are seldom lead investigators, especially on big clinical trials. Yet in the nutrition literature, it is not uncommon to see big studies authored by a grad student or junior researcher, sometimes as the sole investigator. While this is not a universally damning indicator, it should raise the index of suspicion a bit, especially if there are other red flags. 

Country of Origin

Dr. Gaby said there’s another important indicator of potential scientific fraud: geography. “The most common country of origin, by far, for questionable papers is Iran—to a lesser extent, Egypt and China, then India, Pakistan, Japan, and Italy.”
 
Though a German holds first place for total number of retractions, and several Japanese researchers aside from Sato and Iwamoto also rank high on Retraction Watch, as a nation Iran is now the world’s leader in terms of the sheer volume of questionable papers now flooding the medical literature, said Dr. Gaby.
 
“It’s gotten to the point that if something comes out of Iran, I’m inclined to not bother even reading it. Which is too bad, because probably some of the studies are legitimate. But my estimate is that at least three-quarters and probably more of the studies coming out from Iran these days, and to a lesser extent from Egypt, Japan, Italy, and others, raise serious concerns about whether the studies were actually done.”
 
Iran has an advanced healthcare system—a mix of public, private, and non-governmental organization (NGO) payers. On some public health metrics, it ranks higher than the system here in the U.S. Roughly 90% of all citizens there have some form of healthcare insurance.
 
Unfortunately, the country also has a highly competitive market for well-paying, high-prestige jobs that require advanced degrees and scientific prominence. That, along with a totally unregulated cottage industry of for-hire study writers, is a major driver of the surge in fraudulent research from Iran.
 
It is not a new problem. In 2016, Richard Stonethe international news editor for Science magazine authored an article called, “In Iran, a Shady Market for Papers Flourishes.” In it, he details a thriving business in Iran, centered around fabricating research and getting it placed in the international clinical literature.
 
For the equivalent of around $600 (1.8M Iranian Tomans), scientific aspirants can commission a research paper or thesis that “doesn’t need lab work.” An additional $400 increases the odds that the paper will be published “under your own name” in a “reputable” journal. That means, a journal published by a well-established, internationally recognized publisher like Springer or Elsevier.
 
Stone noted there were several thousand of these “paper mills” in operation throughout Iran, mostly centered around major cities with prominent academic institutions. He cited a prominent member of Iran’s Academy of Sciences who, in 2014, estimated that roughly 10% of all masters and PhD theses awarded in the country—amounting to about 5,000 theses per year—are “bought,” and based on work that the candidates never actually did.
 
This is completely legal. There are no laws in Iran—or many other countries for that matter—against the fabrication of scientific data or publishing research one did not actually do. Stone noted that in 2016, a group of Iranian scientists concerned about scientific integrity and the reputation of their nation’s research institutions, proposed a law that would criminalize—at least partially—the selling of concocted science. It never saw the light of day.
 
In his review in Integrative Medicine, and in his lectures, Dr. Gaby drew attention to several Iranian researchers who published prolifically on nutritional topics, and whose work is very likely fraudulent.
 
Among them, Zatollah Asemi,a metabolic disease specialist at the Kashan University, who published more than 191 “RCTs” in the field of nutrition over his career, including 148 studies published between January 2016 and March 2019.
 
“That’s almost 50 papers per year,” said Dr. Gaby. Just on face value, that level of ‘productivity’ should raise a red flag.”
 
Further, Asemi’s published output indicates that he was simultaneously running as many as five different RCTs investigating five different treatments for the same health condition, concurrently. “That is unprecedented. In my decades of reviewing scientific papers, I’ve never come across anyone as remotely prolific as this.”
 
Asemi’s numerous citations cover a wide range of nutrients and herbs including quercetin, ginger, probiotics, magnesium, zinc, vitamin D, berberine, and melatonin. He and his colleagues have used these to treat an equally wide range of medical conditions including metabolic syndrome, diabetes, depression, leukemia, osteosarcoma, and polycystic ovary syndrome (PCOS).
 
Papers by Asemi and colleagues have found their way not only to obscure open-access journals, but into some well-reputed high-impact ones like the American Journal of Clinical Nutrition, the Canadian Journal of Diabetes, and the British Journal of Nutrition.
 
Dr. Gaby said the alarm bells about Asemi’s research are loud and numerous.
 
Beyond the implausibly prolific output, nearly all of his 191 published trials show unequivocally positive, “statistically significant” outcomes for the nutritional interventions being tested. Often the effect sizes are large—larger than one usually sees in legit nutrient studies.
 
“Having reviewed medical literature for many, many years, I can say that it is nearly impossible that all treatments studied always work, regardless of what conditions they’re studied in. Asemi has 191 studies, all positive,” Dr. Gaby said.
 
Further, Asemi’s trials often have implausible timelines. “At least 12 of his papers were submitted to journals before it was possible to have completed the trials. That’s easy to determine because he says exactly when he started them, and how long they lasted.”
 
For example, a 2018 study of magnesium and zinc for women with PCOS, published in the journal Biological Trace Elements Research, stated that the recruitment period was from June to August 2017, and that the treatment was administered for 12 weeks. If the patient recruitment ended on Aug. 1, 2017, the earliest that a 12-week trial could have been completed would be Oct. 24, 2017. Yet the paper was received by the journal on Sept. 27, 2017—weeks before the treatment protocols were completed.
 
Another of Asemi’s studies—this one looking at the effect of multi-mineral plus vitamin D supplementation in women with gestational diabetes—claimed to have recruited 60 women with this condition, who were between 24 and 28 weeks’ gestation, at a single clinic, within a 3-week period. Dr. Gaby said this is highly unlikely.
 
“I looked up some data to see if that was even possible. The region where this study was conducted (a city called Arak), has a population of about 500,000. And I looked up the prevalence of gestational diabetes, and the birth rates for this area. What I calculated was that during any given 3-week period, only about 36 women in the entire city would have had gestational diabetes between 24 and 28 weeks’ gestation. Yet Asemi claimed to have recruited 60 such women at just one clinic.”
 
Dr. Gaby shared his concerns about Asemi with the New Zealand team that ultimately took down Sato and Iwamoto. The latter group was able to obtain a grant to undertake an exhaustive review of 172 studies published by Asemi and colleagues. The result? A comprehensive 115-page dossier which the Auckland group sent to editors at 65 journals that had published Asemi’s trials. It details the myriad inaccuracies, implausibilities, discrepancies, and ethical breaches spanning Asemi’s career.
 
Progress has been slow, but as of now 12 of Asemi’s papers have been retracted, and editors have issued 85 Expressions of Concern.
 

---

“Most nutrition research is believable, and the incidents of fraud do not change my observation that nutritional medicine is highly beneficial for prevention and treatment of a wide range of health conditions. But this is a stain on scientific integrity.” — Alan R. Gaby, MD

---

Dr. Gaby said there are dozens of other Iranian researchers whose work is just as questionable as Asemi’s. There’s Reza Safarinejad, an internationally known urologist, whose main area of interest is male infertility. He’s published numerous studies on the impact of coenzyme Q10 (ubiquinol) on semen parameters, sperm function, and pregnancy rates. He’s also done studies on omega-3s, selenium, and N-acetyl cysteine in the male fertility context. Nearly all of Safarinejad’s studies are problematic.
 
Dr. Gaby pointed to one in particular: a 2009 paper in the Journal of Urologylooking at the effect of CoQ10 on sperm parameters and hormone levels in 212 infertile men.
 
Safarinejad is the only author of this paper, and claims to be the sole physician treating all of these patients. Dr. Gaby said 212 is an implausibly large number of subjects for a standalone urology practice doing its own non-funded research (no funding source is listed in the study). “A single investigator does not have the time or resources to conduct such a large trial by himself.”
 
The study protocol was equally implausible. It claimed that all 212 men visited the clinic 12 times over a period of 13 months, and gave two semen samples at baseline, and two samples within a 1-2 week period around the time of each visit. Further, the semen was collected after 3 days of recommended abstinence.
 
“That’s 24 semen samples per subject, with a total of at least 72 days of abstinence over a 13-month period. I don’t know anybody who would do that. If somebody is infertile and wants to have a pregnancy, he’s going to want to have sexual intercourse and have a baby. The idea that anyone would sign up for this (protocol) is crazy,” said Dr. Gaby.
 
There are also big logistical questions about the protocol, like the process for collecting the semen samples. “While half the 24 semen samples could potentially have been collected during the 12 clinic visits, the other half (12 samples) would have to be collected between visits ... that is, at home. The subjects would have to deliver the semen samples to the clinic within 1 hour of ejaculation, because sperm cells start to die off after an hour. So, they would have to get to the clinic within 1 hour, on 12 different occasions. The paper claimed that 194 of the 212 men completed the trial and provided all the required 24 samples. That defies belief.”
 
Safarinejad claimed that he and a lab tech did all the semen analyses. Doing the math, that’s 4,650 sets of lab tests, all of which had to be done within hours of ejaculation. That’s a heavy workload even for someone not running a busy urology clinic, let alone as a side project for a practicing clinician.
 
Further, the study’s inclusion criteria stated that men were eligible to participate only if they had “normal” fertile female partners “according to investigation,” which meant the participants’ partners had undergone a complete medical history, physical exam, lab testing, and hysterosalpingogram (a test to make sure the fallopian tubes are open).
 
“That’s a lot of work to do,” said Dr. Gaby. “Doing this on 212 women would be very expensive and time consuming. It is not something a urologist would do; it would be done by a gynecologist. But the paper does not specify who conducted these fertility evaluations and who paid for them. And since Safarinejad is the sole author, and there’s no indication of funding source, it defies the imagination that 212 women would have had salpingograms just so their husbands could participate in a study.”
 
Dr. Gaby sent a letter to the editors of the Journal of Urology detailing all of his concerns. “They wrote back by email within 3 hours saying they were going to investigate this. It took about 6 months, but a couple months ago they issued an expression of concern about all 14 papers that Safarinejad had published in their journal.” Slow progress, but several steps in the right direction.
 
The examples cited here are but a few. Dr. Gaby said he’s identified many more problematic studies by researchers from Iran and other countries. Keep in mind that the U.S. is definitely not immune to bad research. 

‘A Stain on Scientific Integrity’

Fraudulent medical research is incredibly corrosive. It erodes public trust in the scientific process; it misleads doctors and other health professionals; and it skews meta-analyses and systematic reviews that aggregate and analyze multiple different studies on a given nutrient or drug. Once marketers and salespeople get hold of it, dodgy data easily transform into exaggerated or outrightly dishonest product claims. At minimum, that means people get ripped off. At the extreme, it means they get hurt.
 
Though there’s no lack of bogus research on pharmaceuticals, the problem is especially damaging to the field of natural medicine, which is continually fighting an uphill battle for credibility in the eyes of the medical community, the public, and the regulators. Fraudulent studies like those described in this article only serve to strengthen the critics who want to paint the entire supplement industry as dishonest and maleficent.
 
As can be easily seen from Dr. Gaby’s experience, and the New Zealand group’s efforts to censure Sato and Asemi, medical journals are very reluctant to retract studies once they’re published. Retractions make journals—and their editors—look bad. There could be potential accusations of libel, even lawsuits. Plus, the work of definitively establishing fraud is long and tedious.
 
Even if papers are retracted, there can be lingering damage, especially if they’d been in the literature for a long time, they appeared in high-impact journals, and they were included in meta-analyses.
 
Despite his extensive experience exposing fraudulent research, Dr. Gaby said he’s still able to put things in proper perspective.
 
“Most nutrition research is believable, and the incidents of fraud do not change my observation that nutritional medicine is highly beneficial for prevention and treatment of a wide range of health conditions. But this is a stain on scientific integrity.” 

Call to Action

What can we do to improve the situation and minimize the spread of questionable research? Dr. Gaby has a few ideas:
 

• Read and analyze all research with a healthy degree of skepticism.
• Contact journal editors if you discover evidence suggestive of scientific misconduct. Encourage them to investigate.
• Whenever possible, insist on better peer review by medical journals. When there are strong reasons to question the legitimacy of a study, editors should require investigators to submit the raw data used to support the paper in question.
• Bar individuals found, beyond a reasonable doubt, to have engaged in scientific misconduct from publishing future papers. “There should be penalties,” said Dr. Gaby. “Currently there are no penalties, and few consequences.”

---

About the Author: Erik Goldman is co-founder and editor of Holistic Primary Care: News for Health & Healing, a quarterly medical publication reaching about 60,000 physicians and other healthcare professionals nationwide.