It’s a no brainer that evidence based medicine is a good idea. Who doesn’t want to know that there is is some form of testing going on to find out if your treatment is likely to work? The medical profession, or most of it, confidently asserts that this is the way medicine is done these days. Without a garland of clinical trials you can’t be a player at the therapeutic table. But who’s job is it to take action when it looks as if evidence based medicine is more of a hope than a reality?
A big study just published in JAMA came to a damming conclusion that because most clinical trials were so poor, very few of the guidelines used by doctors to decide how to treat patients in an evidence based way. “Less than 15 percent of major guideline recommendations,” write the authors “are based on high-quality evidence.”
The study was an analysis of a big publicly available database called ClinicalTrials.gov containing nearly 100,000 clinical trials dating back to 2005. The team looked at human trials, nearly all designed to test drugs, run between 2007 and 2010 and found that the great majority “weren’t good enough to produce the high quality evidence needed to guide medical decision-making.”
Critics of non-drug treatments regularly point out that they cannot be considered to be scientifically based without evidence from large-scale, randomised – i.e. half the participants get the treatment and half get something else, usually a placebo – and double blinded – no one knows who is getting the treatment.
The trials need to be big to flag up problems could emerge when they are given to, maybe, millions. They need to be randomised to show they don’t just rely on the placebo effect and they need to be blinded to cancel out experimenter bias.
So it is shocking to discover that from this data base containing the sort of trials used to get a licence or to guide prescribing, 96 per cent contained fewer than 1000 people and 62 percent had fewer than 100, which is small. Even worse while 70% was randomised and only 33 percent was double blinded.
The researchers looked at trials in three areas – cancer, cardiovascular disease and psychiatric conditions. Those for cancer would seem to have the poorest evidence base, even though there were more cancer trials than for the other two combined. Only nine percent of them were double blinded and just 36 percent were randomised.
And there was something else from the analysis that, while not surprising, is surely a cause for concern. Everyone agrees on two facts about our health care at the moment. With more and more people living longer it is going to become impossibly expensive. There needs to be a much greater emphasis on prevention. Half or more cases of cancer and heart disease are often estimated to be preventable.
So what percentage of clinical trials are designed to test different prevention strategies? Just 11%.
The tone of the report is very low-key but its conclusion is stark. There needs to be “consideration of a comprehensive redesign of the clinical trial enterprise.”
And this comes hot on the heels of a study in the BMJ warning that: “a large proportion of evidence from human trials is unreported, and much of what is reported is done so inadequately.” See also the post about the influenza drug Tamiflu and the battle of availability of evidence.
Any comprehensive redesign needs to boost adherence to the principles of evidence based medicine and I’d suggest three other elements:
Run many more trials of promising prevention strategies
Set up a public/private body such as the one suggested for antibiotics to oversee them
Consider whether the types of trials designed to test drugs are appropriate for interventions involving lifestyle changes.
Body of Evidence 
Leave a reply