Since starting this blog I’ve posted a few times (here, here, and here) on the credibility or lack thereof in medical and scientific journal articles. Today’s New York Times features a long article on this very subject that is well worth reading in its entirety. It’s one of the few articles from the New York Times with which I agree completely.
I’ll excerpt the article and let it speak for itself. Click here, though, and read it completely.
Recent disclosures of fraudulent or flawed studies in medical and scientific journals have called into question as never before the merits of their peer-review system.
Because findings published in peer-reviewed journals affect patient care, public policy and the authors’ academic promotions, journal editors contend that new scientific information should be published in a peer-reviewed journal before it is presented to doctors and the public.
That message, however, has created a widespread misimpression that passing peer review is the scientific equivalent of the Good Housekeeping seal of approval.
The release of news about scientific and medical findings is among the most tightly managed in country. Journals control when the public learns about findings from taxpayer-supported research by setting dates when the research can be published.
Here the author points out the same case I have made many times:
Increasingly, journals and authors’ institutions also send out news releases ahead of time about a peer-reviewed discovery so that reports from news organizations coincide with a journal’s date of issue.
A barrage of news reports can follow. But often the news release is sent without the full paper, so reports may be based only on the spin created by a journal or an institution.
Even the system’s most ardent supporters acknowledge that peer review does not eliminate mediocre and inferior papers and has never passed the very test for which it is used. Studies have found that journals publish findings based on sloppy statistics. If peer review were a drug, it would never be marketed, say critics, including journal editors.
A widespread belief among nonscientists is that journal editors and their reviewers check authors’ research firsthand and even repeat the research. In fact, journal editors do not routinely examine authors’ scientific notebooks. Instead, they rely on peer reviewers’ criticisms, which are based on the information submitted by the authors.
Editors say they have accepted a number of papers that reviewers have harshly criticized as unworthy of publication and have rejected many that received high plaudits.
I have found this last to be half true in my own situation. I have served as a ‘peer’ in the review process for a scientific journal, and of all the papers I have savaged, none were published. But several that I thought were terrific or needed only slight changes never saw the light of day.
Many nonscientists perceive reviewers to be impartial. But the reviewers, called independent experts, in fact are often competitors of the authors of the papers they scrutinize, raising potential conflicts of interest.
The analogy is often made of Ford being in the position of ”reviewing’ GM’s cars and deciding if they should be allowed to be made and sold.
Journals have devolved into information-laundering operations for the pharmaceutical industry, say Dr. Richard Smith, the former editor of BMJ, the British medical journal, and Dr. Richard Horton, the editor of The Lancet, also based in Britain.
The journals rely on revenues from industry advertisements. But because journals also profit handsomely by selling drug companies reprints of articles reporting findings from large clinical trials involving their products, editors may “face a frighteningly stark conflict of interest” in deciding whether to publish such a study, Dr. Smith said.
These last charges are the most serious as far as I’m concerned. I posted earlier about how many medical journals had become publishing arms of the pharmaceutical industry.
All the above makes me look at each article I see with a jaundiced eye and makes me pour over the thing hypercritically. I read scientific/medical articles constantly, and I can tell you, well written, well argued papers that include decent, statistically relavent data are scarce as hen’s teeth. This terrific New York Times piece tells us why.