Positive Drug Studies Get all the Glory

This week's New England Journal of Medicine study about selective publication of antidepressant studies reminded me of the importance of grey literature.  "Grey literature" refers to working papers, rough drafts, notes from half-finished studies...really, anything that doesn't make it into the peer-reviewed publication stream (some people might disagree with this very broad definition).

The NEJM found that a full accounting of the effects of popular drugs like Prozac and Paxil would show that they are barely more effective than placebos.  Unfortunately, medical journals are much more likely to report positive findings than neutral or negative findings.  So that full account does not exist.

Grey literature can correct this skew, and there are ways to find it.  But it's much harder to find a working paper than a published paper, so grey lit will always be the underdog. 

Moral of the story: If the proclaimed benefit of a new drug or treatment seems too good to be true, it probably is.

Comments

[MarkD]

The problem is a result of the funding mechanism of research. Most research, contrary to the beliefs of the medical library community, is not publicly funded but privately funded by commercial interests. Let us accept that commercial interests are going to spend their research money searching for positive results. There is nothing wrong with that – so long has the industry doesn’t deliberately hide negative outcomes when they inadvertently stumble upon those outcomes.

However, there is a gap in research protocols. For the moment little money is spent on research designed to look for negative outcomes regarding efficacy, dosage and most importantly adverse reactions and side-affects. The problem rests in human nature. Researchers naturally want to look for medical breakthroughs. There is no glory in discovering that something doesn’t work or doesn’t work as well as was first thought. Everyone wants to find the cure for cancer. No one wants to discover the potential negatives of NSAIDS (for example).

This bias toward positive research can be easily overcome however by implementing a simple change in philosophy in private foundation and government funded research. A larger percentage of the NIH (and foreign government) research grants should be dedicated to focusing on looking for the negative outcomes in the existing drug compendium rather than looking for new cures.