JC: A decade of reversal: an analysis of 146 contradicted medical practices.

Untitled design(5) copyThis week the blog, the podcast, SMACC and twitter have been ‘off on one‘ about belief, evidence, adoption, change and practice.

It has been great to tackle one of the trickiest decisions we have to make as clinicians with some fantastic people chipping in from across the world. What do we truly believe in? If you haven’t already done so please head over to my talk on belief, and Louise’s talk on evidence to get a feel for what people have been saying. You can also follow the storify record relating to some of the traffic on social media.

In summary it’s been a great week for meta-cognition!

This week’s journal paper is not our typical fare. JC papers on St.Emlyn’s focus on bringing you papers that might change your practice, but this week it’s different. This week we are going to look at a landmark paper from last year on medical evidence reversal. Both Louise and I used the findings in this study when preparing our SMACC talks and I really think that you should read it too. I think it’s #FOAMed and that you can find it by clicking on the abstract below.

Screenshot 2014-06-27 11.01.07 This paper looks back over 10 years of papers in a high impact journal to see what gets in and perhaps a little at why. In particular it focuses on medical reversal, a term used to describe when a previously evidence based practice is proven to be ineffective or harmful.

The bottom line is that there are many (in this case 146) cases where medical reversal takes place and that’s a lesson to us all about how we adopt new ideas.

The publishing game is an interesting one and I remember going on an editors course for the BMJ many years ago when I suddenly realised that publishing is only partly about the science, but also about the interest a paper will generate. It’s a tricky business filled with bias and flaws which, until the advent of #FOAMed was the best we had. That’s also important when interpreting this paper as the decision to publish is not entirely rational, it might appear. Arguably decisions on positive vs negative vs reversal papers are not the same and so proportions of paper types are sometimes tricky to interpret.

Oh, whilst you’re at it, go and have a listen to Tony Brown on publishing. Another talk from SMACC and arguably favourite thus far (after Iain Beardsell). Tony takes us through many of the concerns about medical publishing in a really passionate and convincing talk. Seriously, pause this blog now and go over to have a listen.

I digress. This paper challenges our beliefs about evidence in 363 papers (out of 1344 concerning medical practice) where a study tested an established practice.

The headline is that more papers reversed a previous therapy (40.8%) than reaffirmed it (38%), with the remainder being inconclusive.

WOW!

Just over a third are confirmed in subsequent trials which is rather amazing. It really does take me back to the old adage that half of what we teach you will be wrong, we just don’t know which half…..

Not all of these reversals were relevant to emergency/critical care clinicians, but it is likely that it this study focused on our practice then it will be the same for our evidence base too. Arguably it may be worse owing to the paucity of data for many of our beliefs. If our current evidence base is flaky (seriously it often is – go to BestBets for loads of examples) then overturning past evidence is even more likely.

At this point I must declare a personal conflict of interest as one of the principle examples of medical reversal is the use of aprotonin in cardiac surgery. This is a personal disaster for me as the following abstract from my first ever publication will reveal. My very first article advocated something that we now know to be harmful. The drug was in use at the time with cardiac surgeons advocating it as a future life saver, and now it is no more which makes me both sad and also rather excited about how we must manage our practice beliefs.

Screenshot 2014-06-27 11.26.28

Screenshot 2014-06-27 11.53.01

QUICK #FOAMed post publication review…… @iceman_ex tells me aprotonin is making a comback (Ed – you might have thought you were right but you’re weren’t!).

 

Where does this leave us? Well, we must be cautious as this is only one journal analysis and the NEJM has unique characteristics but I doubt that we would find a completely different story or conclusion in other journals, or if we looked at this across topic groupings rather than journal groupings (Ed – though that could be reversed in another paper couldn’t it 😉 ).  I would strongly recommend that you also read the editorial that accompanied this paper as it brings out many of the points I could also raise here.

Arguably we are left in a tricky position as we must accept that we are constantly learning, constantly changing and that we must remain vigilant not just to what lies in the future but also to what we think we know from the past. I do believe that social media and #FOAMed will help us understand and interpret our evidence base in a way that we never previously could, but even now, we must look out for the next reversal that’s just around the corner. I can guarantee that in the next year you will stop doing something in your clinical practice that at this very moment in time you truly believe to be effective. I’ve no idea what it will be, but it will happen. That may scare you, make you feel uncomfortable, or even question your reason for being here, but don’t let those negative thoughts take over. Embrace change, embrace learning and, as Maya Angelou so wisely said  ‘do the best that you can until you know better, then do better’.

vb

S

1 Comment

  1. MIke Charlesworth

    Aprotinin is very much in use upstairs in cardiac theatres in Virchester…saving lives albeit off license. I think the American studies claiming harm from ~2006 were found to be flawed but this is still a good example of reversal.

    Reply

Thanks so much for following. Viva la #FOAMed

Translate »