The MACS Rule: Immediate ‘rule in’ and ‘rule out’ for suspected cardiac chest pain

This post is about a paper we published today in Heart – the result of more than 9 years of work – phew! Here’s a link to the paper (open access), which reports how we derived and externally validated the Manchester Acute Coronary Syndromes (MACS) decision rule. The MACS rule may help us to exclude ACS using a single blood test in the ED, while also ‘ruling in’ some patients and risk stratifying those who can’t yet have the diagnosis ‘ruled out’.

The MACS Rule

A long, long time ago I took a really important decision in my life. 2 days before Christmas in 2004 I had an interview for a job in the lovely city of Lancaster. The job was my way into higher specialist training in Emergency Medicine and I was the only person being interviewed. This was my ticket to becoming a consultant in Emergency Medicine within 4 years – what an opportunity! Just before that interview, I met with Kevin Mackway-Jones for some career advice. He looked at my CV and asked why I wasn’t doing a PhD instead. My first thought was something along the lines of: you’re crazy, why would I spend 3 years doing research in central Manchester when I could work in leafy Lancaster and be a consultant in 4 years?

Lancaster in the snow

Lancaster in the snow

But after a couple of sleepless nights and a couple of hours spent thinking during a great drive to Lancaster and back in the snow, I took his advice and cancelled my interview. This was, after all, my chance to really make a difference to Emergency Medicine. And so I worked in central Manchester and did a PhD deriving clinical decision rules for patients with suspected acute coronary syndromes. (I’d recommend this route to anyone working in EM – academic EM is an awesome career)

The bright lights of Manchester

The bright lights of Manchester

That was a long time ago and, after investing a lot of blood, sweat and tears and learning an awful lot, the Manchester Acute Coronary Syndromes decision rule – or MACS rule for short – is finally published in Heart today. Many things have changed since we started out on this project – notably the biomarkers – so it was lucky that we made plans to freeze plasma and serum samples, which were later tested for newer markers like high sensitivity troponin T.

[DDET So what was the aim here?]

We (Kevin, Simon, Garry McDowell and Chris Wibberley and I) set out with an idea to derive a clinical decision rule that would allow ACS to be immediately excluded in the ED based on information that was immediately available at the time of arrival, including the results of a single blood test. We wanted to combine clinical features with biomarkers, since neither of these are ever really used alone. What’s more, we wanted to not only identify patients who could immediately go home from the ED, but to risk stratify the others so that we can make more judicious use of inpatient resources and ‘rule in’ the highest risk patients so that they can benefit from early treatment.

Ultimately, we aimed to derive and then (in a separate study) externally validate the decision rule.

[/DDET]

[DDET What did we actually do?]

In the derivation study we recruited 804 patients with suspected cardiac chest pain (of whom 698 could be included in this final analysis), recorded their clinical details using a proforma, and drew blood for biomarker testing. As a reference standard for AMI, all patients had a troponin test at least 12 hours after symptom onset. We also followed them up after 30 days for our primary outcome – MACE (AMI, death or revascularisation including new coronary stenoses on angiography). We repeated the same method in the validation study, at a different centre (Stockport).

MACS rule  derivation & validation: The Methods

MACS rule derivation & validation: The Methods

We derived the decision rule by logistic regression, including all the variables that (by themselves) were both reliable (kappa score >0.6) and made MACE either significantly more or less likely. One novel aspect of the decision rule is that biomarker levels are treated as continuous variables – i.e. there’s no cut-off. The higher the level, the greater the risk. As cut-offs are essentially arbitrary, isn’t that how we should be using them, after all?!

[/DDET]

[DDET What did we find?]

The final decision rule contains 8 things, as shown below.

Slide5

You basically just use a computer calculator, enter values for each of these, and the computer will calculate the probability of MACE. Based on that, it will put patients into one of 4 risk groups:

Very low risk – Go straight home
Low risk – Go to ED observation ward
Moderate risk – Go to Acute Medical Ward
High risk – Go to high dependency/specialist area, e.g. CCU

Here’s how it worked…

MACS rule performance

So, the rate of MACE in the ‘very low risk’ group is pretty low and over a quarter of patients could go straight home just after a single blood test at the time of arrival. Meanwhile, just under 10% of patients were ‘high risk’ and would go to CCU – making the best use of those resources.

[/DDET]

[DDET What is H-FABP?]

H-FABP, or heart-type fatty acid binding protein, is an early marker of myocardial ischaemia/necrosis. It rises as early as half an hour after the onset of myocardial necrosis so is theoretically ideal for our purposes in the ED. This link will take you to a manufacturer’s website – so you’ll expect that it won’t be impartial – but it actually gives you a good summary of what H-FABP is and its potential value.

[/DDET]

[DDET How does the MACS rule make the best use of hs-troponin?]

Every week we see more and more papers published telling us of the potential benefits of using hs-troponin. The assays have some real advantages – they give more reliable results (especially at lower levels) and they can detect smaller concentrations of troponin – including levels in >50% of healthy individuals. With a hs-troponin assay we can also detect 90% of AMIs using a single test at the time of arrival, which is much better than standard assays.

One of the limitations for us in the ED is that a single hs-troponin level can’t ‘rule out’ ACS and, unless the level is very high, it can’t ‘rule in’ ACS either. The MACS rule handles hs-troponin as a continuous variable – so it takes account of all of the richness of the diagnostic information it brings, rather than dichotomising the result at what’s essentially an arbitrary cut-off (the 99th percentile). As such, the lower your hs-troponin level the less risk you are at. This makes perfect sense – and it seems to allow hs-troponin to really flourish as a biomarker, even before you undertake serial sampling. The MACS rule takes account of other information too – such as the ECG, the patient’s symptoms and other biomarkers (we found H-FABP to be the most useful additional marker of those we looked at) – to really get the best out of the biomarkers.

Of course, as I’m the first author, this is a biased opinion – but you can see from the results that it appears to work well!

[/DDET]

[DDET What does this mean?]

Potentially you could use the decision rule now. It’s been derived and externally validated. But one or two things will stop us. First, we need to know how it actually works in practice. Will docs actually be happy to send patients home? Will patients want to go home? Will it perform as well when doctors have to use the rule in practice to guide decisions? These questions can only be answered in an RCT, and that’s what we’ve already started – as you can see at this link and from Shweta Gidwani’s tweet below…

Second, this report tells us how the MACS rule works with a semi-automated ELISA assay for H-FABP. You might be able to get results in a couple of hours but it would be a lot of hard work – and a real push (I ran some of the tests myself!). The test might work slightly differently when we use a fully automated assay that can get us a result fast enough to influence care in the ED. We need to prove that the rule will work with such an assay – and we’ll have to watch this space for those results.

Getting my hands dirty for the sake of the MACS rule!

Getting my hands dirty for the sake of the MACS rule!

[/DDET]

[DDET What other questions remain unanswered?]

Will the rule work with other troponin assays? We need to know – because not all assays are the same. How essential is H-FABP to the rule? How does it compare to the HEART score? And what happens when we add in serial sampling over 1 to 2 hours? All these questions now need to be answered.

[/DDET]

Thanks for reading! Please get in touch if you want to know more about the MACS rule or if you’re interested in getting involved with our future work. We’re on the look out for potential sites for the next phase of our research!

Rick

15 Comments

  1. Andywebster (@Andywebster)

    Richard,
    Great work. In Leeds we have been looking at HFAB as well, and a low initial FAB we have found with a normal ECG to be a useful biomarker for excluding adverse events.

    Even though you are using a program to work out risk….do you really need to use 8 variables to risk stratify?

    I only have one comment and that is your title of the blog post. Immediate rule in and rule out of chest pain…..there is more to chest pain assessment than cardiac causes. Your rule only excludes acute coronary syndromes….Maybe the blog title should change to reflect this i.e. rule in rule out of suspected cardiac chest pain. An emergency physician still needs to consider other significant causes such pulmonary embolism and aortic dissection.

    Reply
    1. richardbody (Post author)

      Thanks a lot for your comments and questions, Andy – great to hear from you!

      In answer to your first question about the title, as you can see I’ve changed it in line with your suggestion. Thanks for pointing this out. As you rightly say, the MACS rule is only for patients with suspected cardiac chest pain.

      To answer your second question about whether you need all 8 variables, I can answer that one by talking through some of the statistical analyses. I know you’re extremely literate in this area but I’ll explain this as simply as I can for everyone’s benefit! The MACS rule was derived by forward stepwise logistic regression. This means that the computer basically identified the strongest predictor of MACE first and put this into a preliminary rule. Then, it tested to see if the performance of that preliminary rule could be improved (with p<0.05) by adding any of the other variables in there. It repeated this over and over until the model was built and no further improvements could be made. At each step, the computer also tested to see if anything it had already put in the rule could be removed without affecting overall performance.

      What this means is that everything in the final rule is there because it adds (with statistical significance) to the performance of the rule. You can take variables out but performance suffers. I'm hoping that I might get time to run some additional analyses and publish them in the near future - to illustrate some of these principles (and their implications) a bit more clearly. I'll keep you posted about that!

      I know that you've done a lot of work in this area in Leeds. I've spoken with Julian Barth about it and I know all about Alistair Hall's work. I'll definitely be following what you guys publish with a lot of interest!

      Rick

      Reply
  2. Henry Hart

    Hi Rick
    Great work
    Just wondering which features BEST help to differentiate the 4.2 /5.8 AMI/MACE (1.2 on validation) for observation ward discharge
    Are they differentiated precisely by a repeat HS Trop and if so what gradient change on your continuous variable
    e.g. 14-16 sufficient?
    If not (ie due to renal disease…seems to be the default explanation) what was the pecking order for the other variables, are some stronger than others in prediction at that point?
    H

    Reply
    1. richardbody (Post author)

      Hi Henry,

      Thanks a lot for your comment. It’s great to hear from you and to get your perspective as an expert in observational medicine. Actually, for the later troponin levels we need to go back to considering troponin as a dichotomous variable – because AMI is still defined as a rise and/or fall of troponin with at least one level above the 99th percentile (14ng/L for the Roche hs-cTnT assay) in the appropriate clinical context.

      Because of the way the MACS rule works, if a patient has a hs-cTnT >14ng/L at the start they definitely won’t be in the ‘very low risk’ group. Patients could potentially be in the ‘low risk’ group despite having a hs-cTnT level slightly above 14ng/L. However, as you can see from the stats in the paper the diagnosis didn’t turn out to be AMI in any of these patients when we validated the rule. Perhaps this means that the MACS rule will help us to tighten up our specificity and avoid the plague of troponinitis that we currently have! Or perhaps clinicians won’t be comfortable investigating such patients in ED observation wards. (It would be quite a change to our culture, after all).

      The RCT we’re proceeding to will tell us. If it happens that doctors aren’t comfortable having patients with hs-cTnT levels of anything above 14ng/L in the observation ward, it will still be possible to use the MACS rule. At our 2nd trial site for the pilot RCT of the MACS rule, all of the patients in the ‘low’ and ‘moderate’ risk groups will be admitted under Acute Medicine (because they have no protocol to investigate suspected cardiac chest pain in the ED observation ward).

      Lastly, given your interest in observational medicine, you’ll probably be most interested in finding out how you interpret the later hs-cTnT level. By the time the patient gets their 2nd troponin level back in the observation ward, you can pretty much ignore the MACS rule outcome (unless we get more evidence to suggest that isolated H-FABP elevations should be acted upon). Interpret the troponin as you normally would – i.e. you need a level >14ng/L in the appropriate clinical context with a rise and/or fall on serial sampling (>9.2ng/L for hs-cTnT) to diagnose AMI.

      Thanks again,

      Rick

      Reply
  3. Dewald Behrens

    Richard

    Thank you for the excellent article – I will cascade this for local input. South Australia is still working on an agreed low risk chest pain pathway.
    Looking at the MACS variables, I remember in an EMJ article emanating from Taunton being first alluded to the fact that chest pain spreading to the right arm is the single highest cardiac risk pain symptom.
    I presume that ultimately, employing this rule in totalis in my practice would require a computer calculator (a possibility for MDCalc ?) and the availability of H- FABP. I see there is a near patient testing apparatus available.
    The elephant in the room for me is as you have aptly termed troponinitis in your last reply. Although the majority of international articles and guidance clearly state for use in cardiac sounding chest pain, or presumed ACS/AMI only, it has become a ‘standard’ work up test (collapse query cause, elderly, short of breath, abdo pain etc etc.) with professionals then unable to interpret the result, a bit like building a machine without a users manual. It contributes to impressive extra cost and time wasting all round.
    D dimer testing has benefited from clinical prediction rules. If one looks at best practice in clinical prediction rules, the PE ones (esp. the PERC) do not fare too well, and CTPA is now often being over utilised without the insight of its issues – but at least there is process that can be honed further.
    Do you know of any work that has been done on this aspect?

    Lastly, surely it is only coincidence that the MACS score is a potential abbreviation of the great MACkway-joneS?

    Many Thanks

    Dewald

    Reply
    1. richardbody (Post author)

      Thanks a lot for your comment, Dewald! I think you’re right about the pain radiating to right arm issue: it was Steve Goodacre who wrote the paper in the EMJ in 2009 (although Cliff Mann from Taunton was also an author, as you say). Here’s the link: http://emj.bmj.com/content/26/12/866.long Interestingly, they also found that pain radiating to the right arm was an independent predictor of ACS on multivariate analysis. I think Panju et al included it in their meta-analysis from 1998 too – http://www.ncbi.nlm.nih.gov/pubmed/9786377. When you read about findings like that it really does change your thinking.

      We reported the value of pain radiating to the right arm or shoulder too (shortly after Steve, Cliff and co.) as part of our univariate analyses for the decision rule derivation. Here’s the link to that one: http://www.resuscitationjournal.com/article/S0300-9572(09)00595-4/abstract

      In answer to your question about the potential for rebound overinvestigation, it is a possibility – and that’s why it’s so important that we don’t stop here. We’ll run an RCT to evaluate effectiveness, and we should continue to audit use in practice even if the RCT demonstrates benefit. If it does lead to rebound over-investigation, the next question will be whether we need a decision rule to tell us who we should use the MACS decision rule in!

      Lastly, the great Kevin Mackway-Jones supervised my initial PhD that ultimately led to the derivation of the MACS rule in its present form. It’s a fitting tribute to the great man that the rule’s name sound like his own – but actually it is a coincidence! It’s not Kevin’s vanity that led to the name. It was my own imagination. I was originally going to call it the ‘Early Vascular Markers of Acute Coronary Syndromes’ rule because I imagined that markers of plaque rupture would be in there. As it happened, they weren’t particularly useful and we moved on to look at markers of ischaemia and necrosis that have better analytical performance than the previous generations (e.g. hs-troponin). Therefore, I renamed the rule ‘MACS’ – and I chose a name that paid tribute to the city I was born in, grew up in and (having declined that option of moving to Lancaster over a decade ago) continue to work in.

      I’ve led an interesting life. 🙂

      Reply
  4. Pingback: Cardiology Case 04: An unusual ECG - St Emlyns

  5. richardbody (Post author)

    You probably already know about the fantastic new critical appraisal site, The Bottom Line. If not, check it out. They’ve covered the MACS rule here… http://www.wessexics.com/The_Bottom_Line/Review/index.php?id=576604263220757137

    Reply
    1. The Bottom Line (@WICSBottomLine)

      Thanks a lot. Looking forward to reviewing your RCT!

      Reply
  6. richardbody (Post author)

    Ryan Radecki has also appraised the MACS rule at his awesome website, EMLitofNote. You can find his appraisal, and my reply, at this link… http://www.emlitofnote.com/2014/05/the-unusable-manchester-chest-pain.html

    Rick

    Reply
  7. richardbody (Post author)

    Julio Marchini (@jfmarchini), a cardiologist working in the ED in Brazil, has commented on the MACS rule via Twitter….

    That’s a great question, and something that caused me to think for a very long time at the start of this work.

    First and most important, we recognised that it was important to incorporate major adverse cardiac events at follow up in the primary outcome. If patients are discharged early but a significant proportion develops a major adverse cardiac event soon afterwards, the decision rule isn’t safe or effective. Then we needed to consider which events are important. Acute myocardial infarction is clearly important and relevant. Revascularisation is arguably important too – although there are issues to discuss about that outcome.

    Death is clearly an extremely important outcome. Discharging a patient who dies unexpectedly soon afterwards is one of the worst things that can happen to anyone – on every side. But this is a decision rule for suspected acute coronary syndromes, so shouldn’t we simply look at cardiovascular mortality? If a patient dies in a road traffic collision, that’s not relevant to the MACS rule so should it really be considered an outcome?

    On the other hand, the probability that well patients will die in a road traffic collision within 30 days of discharge is vanishingly small. If they did (and died within, say, 2 days of admission) you could argue that they are at the very least lost to follow up – because they had no opportunity to develop a major adverse cardiac event for the remaining 28 days. Another consideration was that patients could die of a massive gastrointestinal haemorrhage within 24 hours and still be considered as an appropriate and safe discharge according to this research. I thought that was an unreasonable assumption. If a patient dies soon after discharge, it’s a disastrous outcome and (even though the MACS rule is designed for use in patients who only have suspected cardiac chest pain) we should expect that the rule is designed to give us sufficient confidence that such things won’t happen when the rule is used in practice with reasonable judgement.

    Thanks for a superb question, Julio. I hope that answers it but feel free to add more – and indeed to critique this. Publicising our research via social media is an opportunity to have an ongoing, open peer review – and that means we welcome both positive and negative feedback for the benefit of everyone.

    Rick

    Reply
  8. Pingback: FOAM Eye-Catchers 8: New ACS rule out rule, Tamiflu Scandal, ACEP Seizure Policy & more | Emergucate

  9. Pingback: The LITFL Review 135 - LITFL

  10. Pingback: Is severe chest pain more likely to be a heart attack? - St Emlyns

  11. Pingback: The MACS rule: a new user-friendly version - St Emlyns

Thanks so much for following. Viva la #FOAMed

Translate »