How accurate is clinical judgement for acute coronary syndromes?

This blog post is based on the paper published in the EMJ this week – Body R et al. Can emergency physicians rule in or rule out acute coronary syndromes using clinical judgement? EMJ 2014 [Online first].

Anyone who’s worked in Emergency Medicine for any length of time will appreciate that an acute coronary syndrome (ACS) is one of the most difficult diagnoses to rule in or rule out in the Emergency Department. We’ve probably all seen the old stat that 2% of all acute myocardial infractions (AMIs) are missed in the ED. You might also have seen the stat that up to 7% of patients discharged from the ED have prognostically important myocardial damage, which by today’s standards would be considered as acute myocardial infarction. Perhaps that’s why missed ACS is one of the leading causes of medical litigation.

Image from Wikipedia Commons

What William Osler, King of Gestalt, had to say about clinical judgement for acute coronary syndromes

Maybe we struggle because, as doctors, we like to use our clinical judgement. We like to think that we can use the clinical information we have to make diagnoses without having to rely on tests. Perhaps we also feel like we lose face if we admit patients for investigation when patients don’t actually have the diagnosis we investigated them for. Don’t you hate it when someone from an admitting team catches you in the corridor and says something like, “Hey, you remember that patient you referred to me yesterday with query ACS? We sent him home – it was just gastritis”? It makes you feel inferior – like you should have been able to know that if your gestalt had been a bit better or if you’d have been just a bit bolder.

However, the literature is quite clear about this. If the patient’s symptoms consist of acute discomfort or pain in the chest, epigastrium, jaw, neck, throat or arms and you haven’t otherwise explained it, you’re right to consider ACS – even if the symptoms may not seem so convincing. This is based on good evidence that patients with ACS often have atypical symptoms. Taking individual symptoms, for example, the character of the pain, the radiation of the pain and the number of risk factors a patient has don’t affect the probability of ACS to any significant extent, as you can see from these slides taken from my talk at SMACC Gold (based on our own research – which you can find here).

image

Percentage of patients with AMI stratified by pain character

image

Even grouping symptoms together as ‘typical’ or ‘atypical’, the patients with atypical symptoms are no less likely to have ACS than those with typical symptoms – as shown this great study from Louise Cullen’s group, which I was honoured to be involved with.

image

We ran a cohort study. The main aim of the study was to validate the MACS decision rule but we also ran several substudies using the data collected. In this one, we wanted to know about the diagnostic accuracy of emergency physicians’ clinical judgement for acute coronary syndromes – both alone and in combination with the tests available on arrival – troponin and the ECG. To be honest, we didn’t expect to find a useful ‘rule out’ strategy – we simply wanted to quantify the accuracy to better inform our practice and to find out whether we might be able to refine and improve the MACS rule by incorporating gestalt.

In this study we asked emergency physicians to record their ‘gestalt’ or overall clinical judgement for ACS using a 5 point Likert scale (from ‘definitely ACS’ to ‘definitely not’ ACS. We then cross-tabulated this with the patients’ outcomes – and the outcomes we were interested in were a diagnosis of AMI and the occurrence of major adverse cardiac events (MACE) within 30 days.

The bottom line is that, for patients in whom we’ve already considered the possibility of ACS as a diagnosis, our ‘gestalt’ regarding the overall probability of that diagnosis can’t be relied upon by itself to rule out that diagnosis. Nor can gestalt be used to ‘rule in’ the diagnosis – only half of those whom clinicians felt ‘definitely’ had ACS actually had AMI or developed MACE within 30 days. Here’s another slide from my SMACC Gold talk to illustrate that.

image

No, wait, there’s more. And here’s where the story gets really interesting. Nobody in their right mind would consider the possibility of ACS without doing an ECG, right? So it’s probably unfair to assess clinical judgement alone without at least taking account of the ECG. We’re unlikely to want to discharge these patients if they have ischaemic ECGs, even if we do think that ACS is still unlikely. What’s more, we have another amazing test that we’d always run in these patients – troponin. We run that test on arrival too, and again we wouldn’t send the patients home if they had a positive troponin on admission – even if we did think that ACS was unlikely. So we looked at the overall diagnostic value of gestalt combined with the ECG and initial troponin level.

What we found may seem surprising. If we discharged patients who had a normal ECG, a normal troponin and we felt that the diagnosis was ‘probably not’ ACS, then we wouldn’t have missed a single case in this cohort – the sensitivity was 100%.  Almost a quarter of patients could have been discharged using that strategy.

If we used a high sensitivity troponin assay rather than a standard contemporary one, we could also have discharged patients in whom we felt the diagnosis ‘could be’ ACS without missing any cases – 100% sensitivity. Using this strategy, over 40% of patients could potentially have been discharged – without missing a single AMI in this cohort.
And one of the really great things about this strategy is that it didn’t seem to matter if it was an experienced consultant or a junior doctor giving their gestalt – we still achieved 100% sensitivity.

Maybe emergency physicians are as good as House after all

Maybe emergency physicians are as good as House after all

Before anyone gets carried away by these promising findings, it’s important to recognise that there are some limitations to this work. The 95% confidence intervals for sensitivity extend down to 95.6% and the first reports of new diagnostic technology often overestimate performance (commonly due to reporting and publication bias). Therefore we need to validate these findings – first in observational studies and then, if they still show promise, in interventional trials. Of course, doctors might be slightly less bold when it comes to stating their gestalt if they knew that they’d have to send patients home based on their estimates. We can only find that out by evaluating the strategy in practice.

This means that you shouldn’t use gestalt to rule out ACS right now, even in combination with the ECG and troponin. But you can rest assured that the probability of ACS in patients with normal troponin and ECG on arrival is extremely low if your clinical judgement suggests that the diagnosis is unlikely. This can affect your practice – because we often treat patients with possible ACS on the assumption that they have that diagnosis, before further tests can confirm or refute it. Those treatments have risks (especially bleeding). If the diagnosis of ACS is extremely unlikely, the patients really aren’t going to benefit from early treatment overall. So hang fire with your prescribing pens – and rest assured that your judgement is probably right, even though you do still need to rely on those serial troponins (at least for now!)

Rick

We recently had an offline request from Anand Swaminathan (@EMSwami) for some extra data from this study.  Swami was keen to know how much high sensitivity troponin really adds once the physician’s gestalt and the ECG findings are taken into account.  He asked if we can also report the sensitivity and specificity of the gestalt + ECG – i.e. without high sensitivity troponin.  So here we go, in what I believe could be a first in the #FOAMed world – new data being presented as #FOAMed outside of a traditional medical journal.  Thanks for the request, Swami!

The table below shows the diagnostic accuracy of the combination of gestalt + the ECG (i.e. the presence or absence of acute ischaemia on the initial ECG in the treating physician’s opinion) for the adjudicated diagnosis of AMI (as described in the paper).

Strategy

Sensitivity

Specificity PPV

NPV

AMI ‘ruled out’ if no ECG ischaemia and ‘probably not’ or ‘definitely not’ ACS

95.1

(87.8 – 98.6)

30.0

(25.4 – 34.9)

22.6

(18.3 – 27.4)

96.6

(91.5 – 99.1)

AMI ‘ruled out’ if no ECG ischaemia and ‘could be’, ‘probably not’ or ‘definitely not’ ACS

82.7

(72.7 – 90.2)

61.8

(56.7 – 66.7)

31.8

(25.5 – 38.5)

94.3

(90.7 – 96.9)

AMI ‘ruled out’ if no ECG ischaemia and anything other than ‘definitely ACS’

56.8

(45.3 – 67.8)

79.1

(74.6 – 83.0)

36.8

(28.4 – 45.9)

89.5

(85.7 – 92.6)

The table shows that this approach isn’t sufficient to rule out AMI.  The bottom line is that clinical judgement alone isn’t sufficient to rule out AMI (as reported in the main paper).  Even if you combine that with ECG findings, you still can’t rule out AMI (or rule it in).  It’s only when you start to incorporate troponins that you potentially get the rule out (pending validation, of course).

So, is troponin a friend or a foe in the Emergency Department?  We’ll be posting more on that (from SMACC Gold) imminently, but here’s some good evidence to suggest that it may well be a pretty good friend.

Thanks again for a great request, Swami!

 

Comments

  1. says

    But is a troponin change 100% specific for an AMI? (see Third Universal Definition of Myocardial Infarction)

    Maybe we ought to rephrase the title as “Clinician gestalt is inadequate to anticipate troponin rises”

    • says

      Thanks a lot for the comment, Derek. You’re absolutely right about the difference between a troponin rise and an AMI and it’s an important point. Actually, in this study the outcome was an adjudicated diagnosis of AMI as per the universal definition. I.e. a troponin rise by itself wasn’t sufficient to make the diagnosis. We also excluded patients with other reasons for hospital admission, which excludes most if not all type 2 AMIs.

      Thanks again for reading and commenting,

      Rick

      • says

        My concern is that using ‘adjudicated’ clinical diagnosis as the gold standard is somewhat circular. Were the adjudicators blinded to the troponin rise?

        The objective definitions of new changes in ECG, echo motion or acute angiographic thrombus would seem to more robust versions of that definition.

        Nonetheless, the rapid rule out with hsTrop has great promise. But will it become the next d-dimer leading to increasing over-investigation of improbable ACS?

  2. says

    Thanks again, Derek. The universal definition requires a rise and/or fall of troponin in order to satisfy the diagnosis. Therefore, adjudicators weren’t blinded to troponin levels. It’s a good question about hs-troponin. We’re planning to record some podcasts on it very soon.

    • says

      Retrospective diagnosis (and clinical decisions) can be coloured by the results of subsequent investigations. But that doesn’t mean the conclusion (AMI) or the proposed intervention (revascularisation of the ‘culprit’ vessel) was ultimately correct.

    • says

      Thanks a lot, Simon! I think the prevalence is fairly typical for the undifferentiated population. The cohorts that you see a lower prevalence in tend to have excluded patients with ECG changes – but we kept them in. Some of the US cohorts also seem to have a lower prevalence. (E.g. Erik Hess’ North American decision rule derivation, where the prevalence was more like 5%). In the UK and Australasia, most papers with undifferentiated patients seem to report a prevalence of 15-20%. Our cohorts from Manchester have consistently found a prevalence of 17-18%, despite a changing definition of AMI over the years, which is interesting!

    • says

      Hi Corey,

      Thanks for your comment and I agree that this was really interesting. However, we’re only talking about 1 patient with MACE out of a total of 11 for whom the clinicians checked ‘definitely not’ ACS. With such small numbers, the confidence intervals around that proportion will be quite wide. I’d like to know what would happen to that proportion in a larger sample.

      I was actually surprised to find any patients with ‘Definitely not ACS’ given that the inclusion criteria were ‘suspected cardiac chest pain’! However, although the clinicians checked ‘definitely not ACS’ they still proceeded to test for cardiac biomarkers – so I wonder how certain they really were.

      The NPV of ‘def not’ + negative troponin + negative ECG is 100%. Essentially we dichotomised gestalt for the analysis – so in the main analysis those who either had ‘probably not’ or ‘definitely not’ ACS recorded were considered to have a ‘negative test’. Of course we then combined that with the troponin and ECG.

      Rick

  3. Victoria Brazil says

    Brilliant work (again)
    This approach probably is ‘enough’ if we were able to combine it with an ‘acceptable miss rate ‘
    Ie that % of ACS identified only by (over) investigating a huge population and for whom we cause more harm than benefit at a population level….

    Some relevance of the ‘over diagnosis’ movement…?

    Once again – well done

    Victoria

    • says

      Hi Victoria,

      Thanks a lot for the comment – I’m very glad you like it! I totally agree. Martin Than looked at what emergency physicians perceive to be an acceptable risk of major adverse cardiac events in an international survey. It seemed that emergency physicians would find a rate of somewhere between 0.1% and 1% acceptable. (http://www.internationaljournalofcardiology.com/article/S0167-5273(12)01292-2/abstract)

      That’s quite arbitrary though. It would be interesting to take a ‘test threshold’ approach as in PE. I bet Jeff Kline would have some interesting thoughts about that.

      Rick

  4. says

    Rick – fantastic work. Were you able to break down sens/spec in each category of gestalt when ECG also unremarkable? Curious to see how much troponin added after gestalt + ECG.

    • says

      Hi Swami,

      Thanks a lot for this request. I’ve (finally!) made time to run these additional analyses and I’ve added them to the bottom of the blog post in an additional drop down. I think it makes interesting reading. See what you think!

      Thanks again,

      Rick

  5. Dr Cynthia Papendick says

    Thank you so much for this Dr Brody.
    I am presenting a talk at my institution and would love to use some of your slides from this recent study and your 2010 study from Resuscitation. I would of course credit you and give a shout out to St Emlyns if you are okay with me to use them.
    Regards,
    Dr Cynthia Papendick FACEM
    Royal Adelaide Emergency Department
    Royal Adelaide Hospital
    South Australia, Australia

  6. says

    Rick this is a great read, and thank you for putting this together. I have also put a post together on the value of taking a good history on patients presenting to the ED for chest pain: http://rebelem.com/chest-pain-value-good-history/

    I am pretty sure your 2009 publication was used as part of this post.

    My Clinical Bottom Line: NO historical elements alone or in combination can reliably help us rule in or rule out ACS or AMI, but instead give us some level of risk stratification in conjunction with ECG and Cardiac Biomarkers.

    Salim

  7. says

    Fantastic work. We have noticed in Fiji that those of Indian decent so often have atypical symptoms and +ve troponins! Gestalt alone definately doesn’t work well for that group. This is a great article.

  8. Pendell Meyers says

    Dr. Body, thank you for your excellent data! I have a question for you regarding the following passage from your article:

    “To achieve a sensitivity of 100% with a standard troponin
    assay in combination with the ECG and clinician gestalt, only
    those in whom the clinician felt the diagnosis was either ‘definitely
    not’ or ‘probably not’ ACS could be discharged. This strategy
    would have enabled 106 (23.1%) patients to be immediately
    discharged with no missed AMIs. One (0.9%) of these patients
    developed MACE within 30 days.”

    My question is: can you tell me what that one MACE in the 30-day follow up was? I would specifically like to know the age ( 40 years?) and outcome (death, AMI, revascularization?) if possible.

    Thank you,

    Pendell

  9. Nigel Moore says

    Hi Richard,

    Great post and article

    I agree that the prevalence of AMI (as opposed to ACS) does seem quite high at 17% or so, but as you say depends on the population being studied – i.e. these patients didn’t necessarily have a normal ECG.

    My question is for the patients sent home by the clinicians (or those who would have been sent home) with a negative troponin and ECG along with a low clinical likelihood of ACS, what was the 100% sensitivity quoted referring to – ACS (which includes the nebulous unstable angina diagnosis) or AMI? It looks like it was AMI, with MACE at 30 days being used as the other outcome measure to reflect possible missed ACS on the index visit (0.9% or 1 out of 106)

    I couldn’t tell whether these patients were all getting provocative testing after their serial troponins (i.e. treadmill test or stress echo) as is usual practice here in Australia to try and identify the patients who have ACS (but not AMI), and am interested in your thoughts as to whether you think this is necessary in a very low risk cohort (normal ECG, normal serial troponins)

    Thanks, Nigel

Thanks so much for following. Viva la #FOAMed