Major Trauma Protocols for transfusion in the emergency department are a good thing aren’t they? I certainly think so, it’s something we have instituted in both paediatric and adult practice. Surely there must be robust evidence to back them up, and if not then surely it would be easy to create….. or maybe not?
I recently came across an interesting paper in the Australian and New Zealand journal of surgery this month, it’s an e-publication ahead of print which is why it may not have hit the airwaves as yet. This post started out as a quicky, a short paper to bring us all up to speed on the latest research…., I apologise for turning it into something a little longer, a but whinier, but hopefully a little more informative. As ever, I’d love to hear your views.
Back to the paper then…., in essence this is a systematic review and meta-analysis of the use of major transfusion protocols in the management of major trauma.
As with all critical appraisals at St.Emlyn’s you really should read the paper in full, or at the very least the abstract. It’s quite difficult to track down the full paper as there is no link from PubMed.
However, you can access it (sadly there is a paywall) through the Wiley online interface using this link.
Major transfusion protocols have been around for a while now and there is a great deal of interest borne out of the military experiences in Afghanistan in their use. We have certainly adopted MTPs in Virchester for both adults and children. We’ve had great support from the haematology/transfusion teams to get it set up and riunning and in general I would say that things work well and appear to make a difference.
Having said that as evidence based emergency physicians we are always interested in the evidence behind what we believe to be a good idea. Although we love faith based medicine (FBM) we do strive for a bit of EBM as well.
(Ed – if that doesn’t work then ABM is best when I’m around……that’s Authority Based Medicine, or what I say goes).
We digress, so let’s get back to the paper. This is a review article looking at papers that compare outcomes from MTP protocols in major trauma, Interestingly the authors conclude that the benefit of MTPs is unclear which was a little surprising to the Virchester team, but that’s not a problem. JC at St_E loves it when a paper contradicts what we think, it makes us look harder and deeper at the evidence and in particular the methodology so let’s do that.
[DDET Are we clear in what we mean by Major Transfusion Protocol?] This is clearly important and in fact there is some variability. What we are talking about are protocols that combine red cells, FFP and possibly platelets to replace volume whilst also attempting to maintain the ability of the blood to clot (and thus preventing the doom associated with transfusion related coagulopathy). More details here, or from a variety of websites around the world.
In this study the definition of >10 units of blood or packed red cells reasonably.[/DDET]
[DDET Who (what) was studied?] In a systematic review +/- a meta-analysis the subjects of the study are the papers themselves. It’s essential that the authors define and clearly categorise what is and what is not relevant to the question. In this study the authors included papers that reported patient outcome data from single institutions both before and after the institution of a major transfusion protocol.
That is interesting as before/after studies have their difficulties. If we consider the MTP to be an intervention then as purist EBM docs then surely we would be looking for randomized controlled trials to tell us the magnitude of any effect. Before and after trials suffer from many biases because many things change over time, and in particular we must consider how the management of major trauma has changed in recent years. We have changed many things in addition to the adoption of MTPs and all or none of these may also have had an influence on overall mortality. (Ed – to be fair the authors do admit this in the discussion).
As an extreme example we have seen that the wearing of hats by doctors has decreased over the last century, and survival from trauma has also increased. However, hat wearing does not influence trauma, they just happen to have taken place over similar time periods. An extreme and silly example, but you get the picture.
An RCT is something that we would ‘like’ to see but there are potential difficulties. Pragmatically it might be tricky to run an RCT in the same institution comparing an MTP with no MTP. Would and could that work in reality with critically ill patients? Perhaps not and therefore we might have to look at before and after studies for some time to come.
Perhaps we might even look at performing cluster randomization where MTPs were randomized between institutions, but again the practicalities of this would be challenging and although before/after studies have flaws as decribed above it may be the best evidence available and thus it’s what we have to work with
So. The authors have searched appropriately for papers with real patient outcomes and we are left with 8 papers to consider that include 1586 patients in total.[/DDET]
[DDET What about quality?] Systematic reviews and meta-analyses rely on good quality papers if their conclusions are to be valid. Simply put you cannot amalgamate several bad papers in the hope that you will create a great summary just as you cannot make a silk purse from a sow’s ear.
All good SRs will carefully consider the quality of included papers and in this case the authors have assessed quality as described in table 1. However, the detailed assessment of quality is unclear and it is clear that the retrospective nature of much of the data collection may lead to bias (a problem outlined in the Cochrane instructions outlined earlier).
The overall quality is relatively poor on methodological grounds which raises concerns about the validity of any amalgamated findings. [/DDET]
[DDET Is it valid to amalgamate the studies into a meta-analysis?] This is a tricky question and to some extent relies on the judgement of the authors, though we as readers must question this as a key marker of a quality meta-analysis.
Ideally if studies are going to be combined through meta-analysis then they should be as similar as possible, and by similar we mean similar in methodology, intervention, outcome and event rate. The greater the difference between studies then the greater the risk of differences other than the intervention in question will creep into the overall picture and give us an erroneous result. We can get an idea about similarity from looking at the individual papers and asking oursleves whether they seem similar enough to be justifiably combined. It’s all there in table 2 and it looks fairly clear to me that these are quite disparate protocols and analyses. Although all involve an MTP and all look at mortality there is significant variation between studies.
So, whilst the decision as to whether papers can and should be combined is firstly a judgement, that may be open to bias so to help us in cases where we might be unsure there are statistical tests that can assist us in deciding how heterogeneous they are. The most common one is the I squared statistic which 63.8% suggesting that nearly 3/4 of the value of the odds ratio was due to differences between the studies. . In this study the authors found an I2 of 63.8% which is high and therefore a concern.
In summary we have quite diverse study designs and quite variable results which raise concerns for me about the validity of combining the results. [/DDET]
[DDET What were the main findings?] Well, of the 8 studies identified only 2 showed individual benefit to MTPs in terms of before and after mortality.
After meta-analysis the combined picture showed an Odds Ratio of 0.73 (95% CI 0.48-1.11) which crosses a value of 1 meaning that the result is not statistically significant.
Personally I’m not a big fan of odds ratios and much prefer outcomes to be explained to me in natural frequencies but that’s rather difficult to fathom in this paper. The overall pooled mortality pre-MTP use is described as 41.3% but the overall pooled mortality post introduction is not described. We have to work this out which is a bit of a pain to be honest, but there are shortcuts such as using a nomogram like this one.
This gives us a post MTP mortality of (with an OR of 0.73) about 31%.
Back to natural frequencies though – overall the pooled NNT for MTPs is roughly 41-31/100 or in other words an NNT of ’10’. (Ed – Wowsers, that would be huge if true!!!) Before we go mad though, let’s remember that the OR passes 1 and so this is a statistically non-significant result. In fact we can calculate NNTs for the confidence intervals and that gives us the following result expressed as a natural frequency…. The NNT is 10 (95% CI of NNT-5 to NNH-100). That’s a huge range, but with the overall result apparently trending towards benefit with a small (i.e. good) NNT. BUT – the disparity between the trials means that we may be justifiably sceptical about whether these studies should have been combined in the first place. [/DDET]
[DDET So where does this leave us?] Interesting isn’t it. We have an intervention that makes pathophysiological sense, that has seen widespread implementation but which at first glance does not seem to be backed up by high quality evidence. We must also face the fact that it is unlikely that we will see a large RCT of MTPs anytime soon and so before/after studies may be what we have to live with.
So should we conclude (like the authors) that there is no convincing evidence to support MTPs? I’m not so sure. The design of the papers and the variability of quality makes me sceptical that the overall figure given here represents reality and I’m left at a bit of a loss as to whether this non-significant result means much to those of us involved in trauma care. We must remember that an absence of evidence is not evidence of absence and I would be perhaps be a little more optimistic than the authors in their conclusions……., but that’s probably because I want to believe that MTPs work, (@EMManchester has been reading this and is therefore experiencing quite a lot of cognitive dissonance at the moment!!) [/DDET]