Annette Dickinson, Ph.D.10.01.02
Products such as functional foods, nutraceuticals and dietary supplements live or die by the science supporting the safety and benefit of the ingredients they contain. Thus, any new study casting doubt on safety or benefit can spell disaster, especially given the tendency of the media to present each newly emerging research finding as the final word on the subject.
Science is a process for discovering the truth about the world around us through systematic observation and experimentation. It proceeds in a stepwise fashion, and individual steps may be in a forward, backward or sideways direction. At any point in time, it is the cumulative knowledge about any given subject that is important. The weight of the evidence, and not merely the most recent finding, should determine whether a new product gets introduced and whether an old product gets withdrawn or reformulated. The question is, how can the weight of the evidence exercise its rightful sway and hold its own against the furor that is whipped up around each new bit of data?
Upon instant impact, the new data is likely to trump the old, no matter how positive or how persuasive the body of prior evidence may be. Over a decade ago, when the enthusiasm for oats and oat bran was at fever peak, a couple of new studies threw cold water on the well-founded notion that oat fiber reduced cholesterol levels and therefore was likely to reduce the risk of heart disease. The press trumpeted the supposed "fact" that oat fiber was after all worthless, and the impact on sales of oat products was immediate and negative. In the long run, the strength of the overall evidence was recognized and led to FDA approval of a health claim for oat products containing a threshold amount of the soluble fiber beta-glucan, but the blow struck by the media's excessive hyping of the negative studies remains implanted in public memory.
More recently, a research study published in JAMA reported no benefit of ginkgo biloba in improving normal cognitive function among healthy older adults, and the results were trumpeted in the media as proof positive that ginkgo "doesn't work." Press reports almost completely ignored the fact that numerous clinical trials of ginkgo have been published, with most of them showing benefit. Ironically, one of those was published just a month or two before the one that appeared in JAMA. It showed a significant benefit, but was not picked up by the media.
This illustrates further the point that some scientific journals are more equal than others, in that they generate their own press materials and actively seek coverage of certain articles they publish. While this practice serves the institutional need to promote the particular journal involved, it does not necessarily contribute to a balanced perspective on the overall state of the evidence. The basis underlying the publisher's selection of particular articles to be publicized is also an issue.
The only way to avoid placing unwarranted emphasis on a single research report would be for the entire science publication system to reconsider the appropriateness of its current practices-an eventuality that seems highly unlikely. The very journals that are considered the repository of scientific wisdom are among those responsible for the rush to judgment that occurs in the wake of new articles. The major science journals treat each issue as a national secret, share it with the media a few days in advance of publication and issue press releases touting the hottest new articles about to appear. As expected, the media responds in a feeding frenzy, gobbling up the news and scrambling to get a handle on it, complete with quotes from the researchers and rapid responses, if possible, from other scientists, industry sources and consumer groups.
This is a system that could not be better constructed to suppress a reasoned evaluation of the new study's significance in the context of the existing body of evidence. At every stage, the premium is on the instant response, not the thoughtful perspective. Nobody has an adequate amount of time to truly consider the findings.
For example, the reporter who suddenly finds a hot story on her desk the day before an important family event on the other side of the country is forced into a rapid response mode in order to craft an insightful, if sensationalized, story in a relatively short timeframe. The industry spokesman contacted late on a Sunday evening as he returned home from a weeklong sales conference in Japan is pressed for a quick comment on a barely readable study faxed to him on the spot, often without time to consult with his scientific staff. The researchers who published earlier positive studies are hit with the news that someone has come up with differing results and are asked to respond, without time to fully evaluate the new work and its methodology, to compare it to previous studies, or to engage in the kind of collaborative discussion that sheds light rather than heat on any given topic.
The International Food Information Council (IFIC) has published comprehensive "Guidelines for Communicating Emerging Science on Nutrition, Food Safety and Health," which was developed in cooperation with an advisory group convened by the Harvard School of Public Health. The guidelines emphasize the need to carefully analyze the study design and methodology as well as the results, and they also highlight the importance of putting the findings into context. It would be a major step forward if all parties involved in communicating new research followed the IFIC guidelines.
After the initial impact, there is time for fuller evaluation and response, but unfortunately it is difficult, if not impossible, to get anybody to pay attention to the broader analysis. The story that was so hot last week is suddenly old news, and it is difficult to get the space and time to present the facts in context.
In this phase, the views of the experts in the field can be mobilized to present and discuss the balance of the evidence in letters to the editor for the scientific journals, in review articles for later publication, in seminars and workshops at professional meetings and in meetings or discussions with regulatory agencies. These avenues for fuller consideration among scientists do not generally reach the consumer level, however. Public relations efforts may provide balance for consumers in the form of articles in popular magazines or experts interviewed on radio or TV programs, for example, and industry advertising may also play a role. However, it will always be an uphill battle to correct mistaken impressions already forcefully conveyed to the public through the media.
It may sometimes occur that disappointing results emerge, not just from an isolated study, but from a series of reports. This has been the case in recent years with regard to studies of vitamin E and its potential benefit in reducing the risk of coronary heart disease. Several major clinical trials have shown benefit from vitamin E when used by people with known heart disease or strong risk factors (secondary prevention), and several have failed to find any benefit. Despite the mixed results of the studies, many of the experts in the field continue to be steadfastly supportive of the antioxidant hypothesis of cardiovascular disease. These experts have sponsored and participated in seminars on vitamin E and other antioxidants, have written innumerable letters to the editor, have published review articles and books on the subject and have continued to urge additional research. They are firmly convinced that true primary prevention trials will prove the hypothesis valid. The experts emphasize that secondary prevention trials do not necessarily provide an appropriate test of the hypothesis, and they believe it likely that vitamin E will be shown to have protective effects in primary prevention (that is, prevention of heart disease among people who are currently free of disease symptoms).
The foregoing comments have focused primarily on the events that may unfold when a new study shows harm or lack of benefit. If, instead, the new study demonstrates a positive result, the same guidelines should apply.
The new results should be put into the context of the existing body of evidence, so that the overall strength of the science can be assessed. At some point, the weight of the evidence may become sufficient to support an NLEA health claim, and at that point the industry may wish to submit a petition to FDA for approval of such a claim. Even without a health claim, however, there will be abundant opportunities to use the strong science base in formulating educational messages for the public.
There is a need for manufacturers and marketers of health-oriented foods and dietary supplements to improve the public understanding of the evidence supporting particular ingredients and/or products. Recognizing that every new study represents an opportunity for reviewing the knowledge base, whether or not the media normally takes advantage of this opportunity, industry would be well advised to develop and maintain an ongoing "scorecard" that provides a snapshot of the evidence to date. Such a scorecard should of course be promptly updated to show where any new study fits into the overall picture. This relatively simple device could provide a tool that could be used to illustrate the balance of the evidence at any given time, and to help put any emerging research into an appropriate perspective.NW
About the author: Dr. Annette Dickinson, is vice president of scientific and regulatory affairs at the Council for Responsible Nutrition, Washington, D.C. She can be reached at 202-263-1021; Fax: 202-872-9594; E-mail: annette@crnusa.org.
Science is a process for discovering the truth about the world around us through systematic observation and experimentation. It proceeds in a stepwise fashion, and individual steps may be in a forward, backward or sideways direction. At any point in time, it is the cumulative knowledge about any given subject that is important. The weight of the evidence, and not merely the most recent finding, should determine whether a new product gets introduced and whether an old product gets withdrawn or reformulated. The question is, how can the weight of the evidence exercise its rightful sway and hold its own against the furor that is whipped up around each new bit of data?
The Instant Impact
Upon instant impact, the new data is likely to trump the old, no matter how positive or how persuasive the body of prior evidence may be. Over a decade ago, when the enthusiasm for oats and oat bran was at fever peak, a couple of new studies threw cold water on the well-founded notion that oat fiber reduced cholesterol levels and therefore was likely to reduce the risk of heart disease. The press trumpeted the supposed "fact" that oat fiber was after all worthless, and the impact on sales of oat products was immediate and negative. In the long run, the strength of the overall evidence was recognized and led to FDA approval of a health claim for oat products containing a threshold amount of the soluble fiber beta-glucan, but the blow struck by the media's excessive hyping of the negative studies remains implanted in public memory.
More recently, a research study published in JAMA reported no benefit of ginkgo biloba in improving normal cognitive function among healthy older adults, and the results were trumpeted in the media as proof positive that ginkgo "doesn't work." Press reports almost completely ignored the fact that numerous clinical trials of ginkgo have been published, with most of them showing benefit. Ironically, one of those was published just a month or two before the one that appeared in JAMA. It showed a significant benefit, but was not picked up by the media.
This illustrates further the point that some scientific journals are more equal than others, in that they generate their own press materials and actively seek coverage of certain articles they publish. While this practice serves the institutional need to promote the particular journal involved, it does not necessarily contribute to a balanced perspective on the overall state of the evidence. The basis underlying the publisher's selection of particular articles to be publicized is also an issue.
The only way to avoid placing unwarranted emphasis on a single research report would be for the entire science publication system to reconsider the appropriateness of its current practices-an eventuality that seems highly unlikely. The very journals that are considered the repository of scientific wisdom are among those responsible for the rush to judgment that occurs in the wake of new articles. The major science journals treat each issue as a national secret, share it with the media a few days in advance of publication and issue press releases touting the hottest new articles about to appear. As expected, the media responds in a feeding frenzy, gobbling up the news and scrambling to get a handle on it, complete with quotes from the researchers and rapid responses, if possible, from other scientists, industry sources and consumer groups.
This is a system that could not be better constructed to suppress a reasoned evaluation of the new study's significance in the context of the existing body of evidence. At every stage, the premium is on the instant response, not the thoughtful perspective. Nobody has an adequate amount of time to truly consider the findings.
For example, the reporter who suddenly finds a hot story on her desk the day before an important family event on the other side of the country is forced into a rapid response mode in order to craft an insightful, if sensationalized, story in a relatively short timeframe. The industry spokesman contacted late on a Sunday evening as he returned home from a weeklong sales conference in Japan is pressed for a quick comment on a barely readable study faxed to him on the spot, often without time to consult with his scientific staff. The researchers who published earlier positive studies are hit with the news that someone has come up with differing results and are asked to respond, without time to fully evaluate the new work and its methodology, to compare it to previous studies, or to engage in the kind of collaborative discussion that sheds light rather than heat on any given topic.
The International Food Information Council (IFIC) has published comprehensive "Guidelines for Communicating Emerging Science on Nutrition, Food Safety and Health," which was developed in cooperation with an advisory group convened by the Harvard School of Public Health. The guidelines emphasize the need to carefully analyze the study design and methodology as well as the results, and they also highlight the importance of putting the findings into context. It would be a major step forward if all parties involved in communicating new research followed the IFIC guidelines.
The Middle Term
After the initial impact, there is time for fuller evaluation and response, but unfortunately it is difficult, if not impossible, to get anybody to pay attention to the broader analysis. The story that was so hot last week is suddenly old news, and it is difficult to get the space and time to present the facts in context.
In this phase, the views of the experts in the field can be mobilized to present and discuss the balance of the evidence in letters to the editor for the scientific journals, in review articles for later publication, in seminars and workshops at professional meetings and in meetings or discussions with regulatory agencies. These avenues for fuller consideration among scientists do not generally reach the consumer level, however. Public relations efforts may provide balance for consumers in the form of articles in popular magazines or experts interviewed on radio or TV programs, for example, and industry advertising may also play a role. However, it will always be an uphill battle to correct mistaken impressions already forcefully conveyed to the public through the media.
It may sometimes occur that disappointing results emerge, not just from an isolated study, but from a series of reports. This has been the case in recent years with regard to studies of vitamin E and its potential benefit in reducing the risk of coronary heart disease. Several major clinical trials have shown benefit from vitamin E when used by people with known heart disease or strong risk factors (secondary prevention), and several have failed to find any benefit. Despite the mixed results of the studies, many of the experts in the field continue to be steadfastly supportive of the antioxidant hypothesis of cardiovascular disease. These experts have sponsored and participated in seminars on vitamin E and other antioxidants, have written innumerable letters to the editor, have published review articles and books on the subject and have continued to urge additional research. They are firmly convinced that true primary prevention trials will prove the hypothesis valid. The experts emphasize that secondary prevention trials do not necessarily provide an appropriate test of the hypothesis, and they believe it likely that vitamin E will be shown to have protective effects in primary prevention (that is, prevention of heart disease among people who are currently free of disease symptoms).
What if a New Study Shows Benefit?
The foregoing comments have focused primarily on the events that may unfold when a new study shows harm or lack of benefit. If, instead, the new study demonstrates a positive result, the same guidelines should apply.
The new results should be put into the context of the existing body of evidence, so that the overall strength of the science can be assessed. At some point, the weight of the evidence may become sufficient to support an NLEA health claim, and at that point the industry may wish to submit a petition to FDA for approval of such a claim. Even without a health claim, however, there will be abundant opportunities to use the strong science base in formulating educational messages for the public.
Preparing for the Future
There is a need for manufacturers and marketers of health-oriented foods and dietary supplements to improve the public understanding of the evidence supporting particular ingredients and/or products. Recognizing that every new study represents an opportunity for reviewing the knowledge base, whether or not the media normally takes advantage of this opportunity, industry would be well advised to develop and maintain an ongoing "scorecard" that provides a snapshot of the evidence to date. Such a scorecard should of course be promptly updated to show where any new study fits into the overall picture. This relatively simple device could provide a tool that could be used to illustrate the balance of the evidence at any given time, and to help put any emerging research into an appropriate perspective.NW
About the author: Dr. Annette Dickinson, is vice president of scientific and regulatory affairs at the Council for Responsible Nutrition, Washington, D.C. She can be reached at 202-263-1021; Fax: 202-872-9594; E-mail: annette@crnusa.org.