How much evidence is enough?

Yesterday I posted a link on twitter to a recently updated Cochrane analysis on the role of probiotics in preventing VAP critically ill patients.

I was gently chided by an acquaintance of mine in the UK because the strength of the literature reviewed is very weak.  He's right, by the way, it is.  The reason for me posting it because I think its important for people to understand just how strong (or not) the evidence is for the things we do in the ICU.

And his response got me thinking.  What level of evidence is enough for us to change our practice?

The issue of TRIPS (translation of research into practice) is far from a new one.  Its well recognised that the weight of evidence is not the only factor in changing our practice, and there are some profound examples of therapies where the evidence is strong and uptake is poor (just as the SDD enthusiasts).   Converting quality research into improved patient outcomes is a frustrating problem for many in clinical medicine.

My colleague made the the point that the cash strapped health system had little capacity to implement costly therapies without evidence of impact.  I totally agree with him, yet the paradox is that this same argument is not applied across the board.  Applying the same principles we'd be doing without oxygen, bolus IV fluids, blood transfusion and many antibiotics.  You can forget the fancy new modes on your ventilators.  You might be blowing thousands of dollars starting renal replacement too early.  Don't get me started on ECMO.  Adrenaline in resuscitation algorithms.  We've abandoned the pulmonary artery catheter, but the ECG and full blood count are not held to the same standards.  Clearly, evidence is not applied in a consistent fashion.

On the flip side, over the past 2 decades we've watched practices change on the basis of early work, only for them to be abandoned when the "proper" trial comes out. Tight glucose control.  Steroids in sepsis.  Activate protein C.  Vasopressin.  Dopamine for kidneys.  Activated Factor 7.  Nasojejunal feeding.  Aprotinin.  Early goal directed therapies in sepsis.  Desperate for an intervention that makes a difference, we've latched onto this early with gusto, only to be disappointed.

Was it wrong for us to move to these practices before the confirmatory evidence became available?

So, to come back to my initial post, the evidence is far from conclusive.  Its entirely possible that a well conducted, large RCT may demonstrate no impact, or even harm.  But that's yet to be confirmed so we're left with the best available evidence, which is that it might help and seems to have little potential for harm, albeit with a cost (and I'm not trivialising that).

I'm not for a moment suggesting that we should implement all these half baked ideas on the basis of rubbish information.

But it does make you ask the question - should we be practicing based on the best available information?  Because the fact is, we're doing this all the time.