Behind the Scenes at St Emlyn’s

Hello again :-)

We’d like to think you might have missed us a little in the last ten days or so when the St Emlyn’s blog has been offline (if you hadn’t noticed our absence, please nod and smile – it’s what we British people do).

As you can see we’re back and more than a little relieved – and we are really sorry for the brief hiatus BUT it does give us a great opportunity to provide you with a little more information about the inner workings at this, Virchester’s finest healthcare institution.

St Emlyn’s: An International Hospital

St Emlyn’s is truly a virtual hospital; the medical staff come from all over the UK to work here, most without ever actually leaving their houses. But there’s a little more to it than that. The physical virtual space in which St Emlyn’s exists is a real place – in Western Australia.

The Father of FOAM: Behind the Scenes

As you probably know, FOAM was conceptualised at ICEM 2012 in Dublin over a pint of Guinness – but for more than five years before ICEM 2012, Life in the Fast Lane had been doing its thing; sailing the as yet unnamed ship of free, open access medical education into uncharted waters. And at its helm – the inimitable Mike Cadogan.

Image courtesy of Domnhall Brannigan

Mike is key to the success of St Emlyn’s. When the virtual hospital opened its doors shortly after ICEM, Mike was on hand to guide and advise. Since those early days Mike has not only been our go-to guy for the more technical aspects of social media, coding, search engine optimisation and blogging (if it sounds like I know what I’m talking about here I apologise because I really don’t) but he also helped us to migrate the whole site over to a server he runs in Western Australia (together with a collection of other FOAM blogs).

In the last few weeks the servers have been under attack from hackers to an unprecedented degree, taking most of the co-hosted blogs offline. Thanks to Mike’s hard work it looks like most are back up and running.

Why are we telling you this?

Well, obviously we didn’t enjoy our time offline – but mainly we wanted to use our first opportunity back online to express our collective wholehearted love and gratitude towards Mike for all of his hard work.

Mike, we wouldn’t be where we are without your input. We certainly wouldn’t be back online again. We – and I’m sure I speak on behalf of the whole FOAM community here – are so grateful for all your help and hard work! And we want you to know – YOU’RE AWESOME.

JC: Getting Chilly Quickly 4. Doing It For The Kids

DOING IT FOR THE KIDS

We have looked at therapeutic hypothermia not once, not twice, but three times already here at St Emlyn’s; most of our ED consultants also work, at least some of the time, down the long corridor in our paediatric ED – so our interest in how we can best look after our post-cardiac arrest patients extends to children as well.

By the wonder that is twitter, it was a matter of moments after the publication of this particular paper that it was tweeted to me as something I might be interested in reading (thank you so much, James Tooley!)

Of course, Cliff Reid was first off the starting blocks with his appraisal over at Resus.me while Salim Rezaie has written a review at Academic Life in EM which I have peer-reviewed.

That said, before I had a look at either of their appraisals I did read the paper myself (something I’d recommend to all FOAM colleagues) and we plan to look at the paper in the real life journal club as well as here in the virtual one; my thoughts are below and also over at PEMLit.org.

NEJM

Kudos to NEJM for making the paper open access – please go and read through it yourself before reading reviews here or on any other sites. Click the abstract above which will take you to the NEJM site and the full paper.

What kind of study was this?

This was a single-blinded, multicentre randomised controlled trial of therapeutic hypothermia versus therapeutic normothermia for paediatric out-of-hospital cardiac arrest. Patients who presented after requiring at least 2mins of chest compressions and with an ongoing requirement for mechanical ventilation after return of circulation were randomised in a ratio of 1:1 by permuted blocks stratified by age into either hypothermia (targeted temperature of 33deg for 48h, then targeted temperature of 36.8deg for another 72h) or normothermia (targeted temperature of 36.8deg for 120h).

It’s important to note that this was therapeutic normothermia, so even in the control group there was an active intervention to tightly control the temperature between 36.0-37.5 degrees rather than just allowing the temperature to react spontaneously.

Who was studied?

The subjects were recruited over just more than three years from 36 sites (there were 38 sites involved but two did not recruit any patients).  The patients were aged between 48 hours and 18 years and the trial itself was conductive in the Paediatric Intensive Care Unit.

Patients were included if they had received two minutes or more of chest compressions with a subsequent return of circulation but ongoing need for mechanical ventilation. They were then excluded if they could not be randomised within 6 hours of return of circulation, if they scored 5 or 6 for the motor component of the Glasgow Coma Score (implying rapid return to normal conscious level and presumably short anticipated need for mechanical ventilation), a decision by the treating team to withhold aggressive therapy or major trauma as the cause for the cardiac arrest (this seems reasonable as traumatic cardiac arrest has different aetiology).

The primary outcome of interest was survival at 12 months with a “good functional outcome” and the authors were looking for an absolute effect size of 15-20%.

Patients were also excluded if their baseline performance score (VABS-II) was <70 since this was determined to be the marker of good neurological function used as the primary outcome measure at 12 months.

There were 1355 patients meeting the inclusion criteria but after initial exclusions 295 were randomised, 155 to therapeutic hypothermia and 140 to therapeutic normothermia. Results were analysed using an intention-to-treat protocol; this means that patients were analysed according to the group to which they were initially randomised, irrespective of whether they received the intended intervention, the alternative intervention or withdrew from the study altogether after randomisation.

However, 25 patients were found after randomisation to have a baseline performance score <70 so these patients were not included in the analysis of the primary outcome, leaving 138 in the hypothermia group and 122 in the normothermia group for analysis.

The study also explored some secondary outcomes; survival at 12 months and change in neurobehavioural function (defined as the difference in VABS-II score at 12 months from pre-cardiac arrest baseline).

What did they find?

For the primary outcome there was no significant difference between the two groups in survivors with VABS-II score at 12 months;

27/138 in the hypothermia group had a VABS-II score >70 at 12 months

15/122 in the normothermia group had a VABS-II score >70 at 12 months.

The authors calculated a risk difference of 7.3 percentage points (95% confidence interval -1.5 to 16.1) and a relative likelihood of 1.54 (95% confidence interval 0.86 to 2.76).

Although that sounds like a significant difference, the confidence intervals are really important here.

For the risk difference, the 95% confidence interval crosses zero; this means that the “true” value could be 0% difference between the groups. For the relative likelihood, although the confidence interval doesn’t cross zero, remember this is a likelihood: a ratio, where we equivalence occurs at 1 rather than at zero, so it is the fact that the confidence interval crosses 1 which indicates no significant difference between the groups.

In the secondary outcomes (table 2) there was a similar result of no difference in survival at 12 months between the groups, irrespective of neurological function. The authors note that survival over time was significantly longer within the therapeutic hypothermia group (mean survival for hypothermia group 149±14 days compared with 119±14 days) and there is a Kaplan-Meier curve on page 22 of the supplementary material which demonstrates this nicely. This rather leads us to question what we are trying to achieve; survival without the functional score criteria implies survival with poor quality of life. It is not my place to argue here what constitutes meaningful quality of life and I would be interested to hear the thoughts of the parents of these children surviving with poor functional scores, especially if they ultimately die later. There are some big moral and ethical questions here but if anything they do not encourage me to pursue hypothermia in these patients.

Interestingly, the Kaplan-Meier curve shows for both groups the majority of deaths occurred within the first 28 days; the authors tell us this was 87/153 (57%) in the hypothermia group and 93/139 (67%) in the normothermia group with no statistically significant difference between them (P=0.08).

Anything else interesting about this study?

There’s loads of other really interesting data in this study. Aside from the outcome measures, it’s useful to know more about the aetiology and prognosis of paediatric out-of-hospital cardiac arrest. Firstly we can see from table 1 that the majority of cardiac arrests occured due to respiratory causes – it’s good to know that what we teach on APLS is based on evidence! The rates were 111/75 (72%) in the hypothermia group and 102/140 (73%) in the normothermia group.

I was surprised that the rhythm was VF or VT in as many as 23 patients (14 in the hypothermia group, 9 in the normothermia group). Although the paper doesn’t give p values for the baseline characteristics in table 1 the authors have flagged those baseline characteristics with a P<0.05 (number of pre-hospital adrenaline doses and in-hospital adrenaline doses), suggesting this was not a significant difference between the groups even if it does seem higher than I would expect.

I was equally surprised to see that the median time to CPR from cardiac arrest was so low; 3 mins for the hypothermia group and 2 mins for the normothermia group. It’s not clear whether this represents rapid EMS response times or bystander CPR (which occured in 68% of the hypothermia group and 63% of the normothermia group)  but I can’t help but wonder whether how well we could achieve that in the UK.

The median time to ROSC is interesting too, 23 mins in the hypothermia group and 28 mins in the normothermia group. We tend to spend longer in our resuscitation attempts for children in cardiac arrest than we do in adults and previous studies have looked at outcomes with resuscitation attempts over 20 minutes in children, showing that good neurological outcomes can occur even after prolonged resuscitation.

Problems with the paper and areas for future work

There are a few sticking points here; there was no reasonable way to use blinding to prevent bias in the intervention groups and although the authors have sensibly used an independent observer to collect the outcome data without knowing which group each subject was randomised to, the data was collected by telephone interview and there is nothing to stop a parent asking whether the researcher thought that their child being cooled had made a difference, for example.

The authors themselves acknowledge that children in the hypothermia group might have been alive longer due to the confounding effect of the hypothermia on the clinical team’s ability to determine futility.

We must also remember that these were out-of-hospital arrests; what should we do with those patients who have a cardiac arrest in-hospital? Is there a benefit from hypothermia for that group of patients?

What does it mean for us in practice?

Remembering that in this study the null hypothesis was no difference between the groups, this study doesn’t prove that hypothermia is harmful or not beneficial; there is simply insufficient evidence to reject the null hypothesis of no difference, based on this study.

So: should we cool our paediatric out-of-hospital cardiac arrest patients? At present, there is no evidence of benefit from this paper – but that doesn’t mean that the benefit doesn’t exist. For now I think we have to follow local protocols and hope for more evidence from larger trials, difficult as that might be to collect. That said, it’s still an important trial because it gives clinicians greater freedom to deviate from protocols when necessary if the evidence of benefit is difficult to prove.

 

Natalie May

 

FCEM-style Questions

  1. What is a likelihood ratio?
  2. What is meant by the sensitivity analysis mentioned in the first paragraph of the Outcomes section and why is it performed? (If you need help, there’s a nice open access article here)
  3. Why is Intention-To-Treat analysis important in a study like this one?

See also this FCEM style review over at PEMLit.

 

Neurocritical care in Manchester NASGBI and St.Emlyn’s

Screenshot 2015-05-21 06.58.25

Neurocritical care debate, discussion and overall goodness this month, in the form of a conference. This time at least it’s local and the St Emlyns team are off to Manchester for the 50th Annual Scientific Conference of the NeuroAnaesthesia Society of Great Britain and Ireland (NASGBI). If you thought that was a mouthful then during the meeting they voted themselves to become the NeuroAnaesthesia and Critical Care Society of Great Britain and Ireland (NACCSGBI). Long live a good acronym……

Anyway, a fantastic couple of days with great keynote speakers, invigorating debates and a good display of national work. The local organisers did a fantastic job with the venue and main track, but more importantly if this wasn’t enough they also ran a parallel day for non-anaesthetists celebrating all things traumatic and neurosurgical. Less important than the content here is the coming together of tribes. We all know that multidisciplinary input is key to the journey of traumatic brain injury and this felt like real recognition and understanding of the additional clinical roles outside of theatre. Also a great networking opportunity for emergency physicians, intensivists, neurosurgeons and anaesthetists; altogether a very valuable exercise.

No conference can be summed up in a single post, but there were a few particularly interesting topics that found their way to the twittersphere and garnered attention. These were our favourites:

Oli Harrison presented a short survey of consultant anaesthetists on choice of induction agents and post intubation physiological targets for a standardised single hypothetical scenario of a young patient with isolated traumatic brain injury. The results were not quite as cohesive as may have been expected. Propofol topped the chart at 51% for induction, thiopentone 33% and only 2% choosing ketamine. Post induction targets were even more interesting – less than 50% were interested in a strict ETCo2 of 4 to 4.5KpA and just over 50% wanted to keep the MAP >80. This being despite recent NICE guidance. Tentative suggestions as per the conclusion that standardised operating procedures for RSI in the head injured patient may be of benefit sparked debate on twitter:

This discussion concluded with the idea of a standard operating ‘guideline’ depicting safest practice while recognising and encouraging expert adaptation on a per patient basis. However, I’m still not sure this sits right with me. I recognise that education and individual clinical assessments are key to delivery of safe practical care to the critically ill patient. But I think there is also little merit in having a guideline that offers unclear and limited guidance. Local attempts to discuss standardised induction regimes have met with harsh criticism recently (Ed- and not just in your world Dan, we’ve experienced this too) so people obviously feel strongly about this. But I think the debate needs to continue – education and adoption of best practice will take time; until then, should those of us with more critical care experience not be collaborating to propose a baseline ‘safest level of care’ through induction algorithims?

My next favourite was a combination of talks on fatigue and situational awareness, facilitated by an air traffic controller (have a look at this if you’re wondering what air traffic control has to do with cognitive overload) and a member of the RCOA working party on fatigue in anaesthesia. A good document and worth a read on your next night shift. We were encouraged to identify the difference between system 1 and system 2 by a simple maths puzzle, with a bat and a ball costing £1.10. If the bat costs £1 more than the ball, how much does the ball cost? System 1 took hold of me – 10p 10p 10p 10p 10p 10p 10p. After wrestling control of my own brain back I worked it out eventually but this always highlights a point for me. The benefits of understanding fatigue and situational awareness are primarily about understanding what these situations will do to you personally – such that you can recognise, adapt and engage others when you feel you are not firing on all cylinders. And did you know that if you are awake and operational for 21 hours it is comparable to being over the drink drive limit for alcohol in the UK? The speakers urged us to all to end the ‘macho’ culture of back to back shifts/opting out of the European working time directive. There is no doubt that we may sometimes do our patients a disservice by trying to persevere in exhaustion. This martydom should be discouraged at the top, and from the top down.

After this, a debate about specialist neurointensive care units versus neuro patients on a general unit. Interesting arguments. The proposer for specialist care cited disease specific knowledge of highly trained neurosciences nursing staff as the key to benefit in these complex patients. Maybe a subtle way of celebrating national nursing day? The opposer had slyly performed a round telephone survey of all UK specialist neurocritical care units previous to the debate and had current data on the general ICU measures that they could and could not perform. There was some quibbling in the questions, but the take home from this was powerful – On site ability to deliver polytrauma care, critical care echocardiography and citrate based renal replacement therapy were rare within specialist centres. Very rare. My take home? Stick me on a good general unit with neurosurgical input any day of the week please…..

Brendan McGrath from the Global tracheostomy initiative talked about the big fuss over a small hole – I cannot commend this work highly enough and would urge you to look at the resources and training packages provided online if you have not already. It will save your (patients) life one day…. James Palmer talked about NAP5 and relevance to neuroanaesthesia/critical care. One important message I got from this was about the real terror of awareness being the fact that no-one is talking to you or understands that there is a problem. There were compelling anecdotal stories of patients accidently paralysed who had little psychological sequelae, attributed largely to the fact that the physician responsible had immediately reacted – “I know what is happening and I can fix it immediately”. Sobering stuff, but a good reminder of the fact that all our patients need kindness, humanity and understanding; even the ones who are sedated, supposedly comatose or deemed to be unaware.

Too much other great stuff to fit on the page. But suffice to say it was an excellent event and a great collaborative event. Have you considered putting on a local multi-tribal meeting? You may find it is to everyone’s benefit. Including your patients.

Best wishes all

 

Dan

 

PS – The Ed enjoyed Steve Peters talk from Chimp Management. I now understand the St.Emlyn’s team to resemble a troop (a rather nice troop of course).

JC: An International Clinical Trials Day Special

International (2)

International Clinical Trials Day 2015 is on 20th May.  To mark the occasion we’re going to run an extra special critical appraisal.  This time, let’s look at an awesome piece of work I came across in the literature.  It’s not exactly hot off the press – but it’s an inspirational piece of work that I’m sure will influence medicine for many years.  The topic is also a little off piste but bear with me – I think this one’s a game changer, one way or another…

You can find a summary of this trial at this link.  Or you can read the full text here.

[DDET What question did they answer?]

Dr James Lind Wikimedia commons
Dr James Lind
Wikimedia commons

In this work, the lone author, Dr. Lind – who was from Scotland – noticed that he had a problem with sailors on his Royal Navy ship (HMS Salisbury) coming down with scurvy and many of them died.  He wasn’t sure how best to treat them so he ran a trial.

So, let’s critically appraise this for a minute…

‘Was there a sound rationale for the trial?’

Well, yes, because they had a big problem with scurvy.  But I’m not so sure that Lind did everything he ought to in order to justify the position of equipoise.  Where’s the systematic review?  Where was the early phase work for pharmacodynamics and pharmacokinetics of the interventions?  What’s the justification for choosing the interventions he went for?  I have some concerns about this.

‘Were the objectives clearly stated?’

It seems to be obvious what Lind was trying to achieve but – no – his objectives were not stated at all, either a priori or in the manuscript.  And there’s simply no evidence of a null hypothesis.  It looks like Lind wanted to compare six treatments for the treatment of scurvy, which is pretty ambitious.

Was this a superiority trial, a non-inferiority trial, an equivalence trial?  He doesn’t tell us.  However, I think we’ll let him off as it seems to be obvious that this is, in reality, a superiority trial.  The null hypothesis was that there was no difference between the treatments.

[/DDET]

[DDET What did they do?]

This was a prospective controlled trial at a single centre (well, a single ship) coming from the UK.  They (well, Lind) included 12 patients with symptoms of scurvy.  The inclusion criteria aren’t very clearly stated but Lind tells us that “in general” they all had “putrid gums, the spots and lassitude, with weakness of their knees”.  No tests were done to confirm the diagnosis of scurvy.

Page from the journal of Henry Walsh Mahon showing the effects of scurvy, from his time aboard HM Convict Ship Barrosa. 1841/2 via Wikipedia
Page from the journal of Henry Walsh Mahon showing the effects of scurvy, from his time aboard HM Convict Ship Barrosa. 1841/2 via Wikipedia

I’m getting concerned here.  The inclusion criteria are pretty vague.  None of the diagnoses were confirmed and there could have been a huge variation in the severity of disease among participants.  In fact, there are no baseline characteristics reported at all – so we can’t tell.

The patients were assigned to one of six treatment groups, as in the pic below.  They took their treatment for six days and then outcomes were assessed.

interventions (1)

How were patients randomised?

Hmmm.  There was no randomisation.  They were just assigned to different groups – by Lind.  We’ve got another problem here.  There’s no allocation concealment so there’s a possibility that the milder cases were selected to go into Lind’s chosen treatment groups.  The patients with more severe disease might have been given less promising treatments.  Without randomisation, we’ll never know, but we do know that the patients who got sea water (poor things) were “two of the worst patients, with the tendons in the ham rigid (a symptom that none of the rest had)”.  Clearly this introduces a huge bias.

[/DDET]

[DDET So what was the primary outcome and what were the key findings?]

Well, the primary outcome isn’t actually stated.  Not at all.  We assume that Lind was looking for recovery but he didn’t define this.  We do know that the two patients assigned to eat oranges and lemons recovered quite well – one of them fully recovered, the other was well enough to resume duties after 6 days (whatever that means!).  It seems that the outcomes of the patients in the cider group were fairly good.  Lind reports, “Next to the oranges, I thought the cyder had the best effects”.  Objective?  Hmmm.  The outcomes of the other groups don’t seem to have been quite as good although the putridness of the mouths of patients in the sulphuric acid group seems to have been reduced.

Scorbotic gums via Wikipedia
Scorbotic gums via Wikipedia

Without a primary outcome that’s clearly defined (including when and how to measure it), how do we know what success looks like in a trial?  Without comparing the groups statistically, how do we know that the findings aren’t just down to chance?

That brings us on to the crux of the whole thing… Lind concludes that oranges and lemon were superior to the other groups for treating scurvy.  But let’s take a look at the analysis.  There are six groups so I assume that Lind really wanted to do pairwise comparisons – i.e. to compare one group to another and run multiple comparisons.   When you do lots of comparisons, you need to be aware that sometimes the p value might be <0.05 just by chance.  Sometimes you might choose to adjust for that – and only accept a lower p value.  Even without that, what was the p value here?

Assuming that none of the patients in the other groups did well, this is our 2×2 table…

Patient did well Patient didn’t do well
Patient took oranges and lemons 2 0
Patient took sulphuric acid 0 2

 

It looks good, right?  The absolute risk reduction is 100% – number needed to treat is just ONE!  Comparing those two proportions tells us that the absolute difference was 100% BUT the 95% confidence intervals extend from -19% to 100% with a p value of 0.32.  So this trial didn’t show any statistically significant difference at all!  Clearly a 100% difference is pretty clinically significant – so that means the trial was underpowered.  Tut tut.

[/DDET]

[DDET What about the way they conducted this trial?]

I think the MHRA and FDA would have something to say about this.  There’s no mention of any ethical approval.  It doesn’t look like they took written consent from the patients.  They might have consented verbally – we don’t know – but did they know what they were letting themselves in for?  Were they given a patient information sheet and the opportunity to ask questions?  We don’t know.  The trial certainly wasn’t registered a priori.  And there’s no mention of any declarations, financial relationships with fruitmongers or ownership or orchards.  We don’t know which organisation sponsored this trial – or even if the Royal Navy knew about it and gave R&D permission.

[/DDET]

[DDET So what are the take home messages?]

All in all, it’s a pretty lousy example of a trial.  I don’t think we can take anything meaningful away from this.  Should oranges and lemons be used to treat scurvy?  Who knows?  If I get scurvy, I’ll probably give the oranges a try as I quite like them anyway – and there aren’t too many side effects.  I’d quite like to try the cider too, mind you.

Obviously, there is a serious take home message from all this too.  This trial is widely held to be the ‘first clinical trial’ by James Lind, reported in 1753.  It’s not bad for it’s time – but clinical trials have come a long way since – and just think how much better off we are for it.  Happy International Clinical Trials Day!  I wonder what clinical trials will be like in another 250 years?

[/DDET]

Be sure to check the full text out by hitting this link – it’s an interesting read!

Rick

 

JC: Are typical chest pain symptoms predictive of outcome? St.Emlyn’s

Chest pain

We love a bit of meta-cognition here at St.Emlyn’s. As time passes and we all become a little longer in the tooth it becomes increasingly apparent that it’s not just ‘what’ we know it’s how we use it. In diagnostic testing in the ED (something that we are supposed to be awesome at) we must understand how we use tests and how our clinical judgement affects the use and subsequent performance of those tests.

We are always on the look out for papers that examine how we think and how we make decision and so I was intrigued to look at this paper on the ability of clinicians to determine the probability of underlying MI/IHD in patients presenting to the emergency department. If you’re a regular reader you will know that Rick Body has a great deal of experience in this area and has published widely on clinician ability to identify patient risk. You might want to look at these blog posts where we have explored judgement and metacognition in the past.

How good is clinical judgement in assessing chest pain?

Is severe chest pain more likely to be a heart attack?

What is Gestalt?

The paper is from Bristol UK and the abstract is below. At the moment it is open access on the website so please read the abstract, but as always, if you can find the time please read the full paper before going further.

Screenshot 2015-05-09 06.30.27

What kind of paper is this?

This study was part of a larger study at the use of chest pain pathways and high sensitivity troponins. This is a pre-planned study within that larger study and it’s essentially an observational cohort.

Tell me about the patients

They have looked at 912 patients of whom 12.5% had an AMI. This is a fairly low event rate but not dissimilar to other studies in this area as they have only looked at patients with a non diagnostic ECG. Interestingly they excluded patients who had no diagnostic testing which may have missed a small number of patients. The study is looking at judgement and part of that judgement is whether or not to test a chest pain patient for AMI. It’s tricky as the alternative would be to test everyone with chest pain according to predetermined criteria. This way is more pragmatic but it may have missed some patients. This is a single centre UK medium sized hospital, so limited because single site, but not untypical of many UK departments.

What did they do?

In the larger study they are following patients through to a definitive diagnosis of AMI/not AMI and IHD/not IHD using troponin testing and angiography. Since that will give a pretty robust gold standard outcome they have collected data on physician impression of pain as patients presented in the emergency department. They have asked clinicians to state whether the pain is typical or not then followed the patients through to see if they are right. They have also compared experienced vs inexperienced clinicians to see if seniority improves performance.

AMI was determined using the third universal definition of MI and IHD determined in some patients undergoing angiography although that was quite a small percentage (17.2%).

What did they find?

The key findings from the authors are that fewer than half of patients were characterised as having typical chest pain and that experienced clinicians. There was a small difference in classifying patients as ‘typical’ between experienced and non-experienced clinicians (35.2 v 45.8%).

The ROC curves are interesting and reminiscent of work we did on risk factors in cardiac disease. In other words the performance of doctors in assessing risk is rubbish! Hang on though. This is a little bit unusual as chest pain typicality is a dichotomous variable (the chest pain can either be typical or atypical – there are only two possibilities).  You usually use an ROC curve to plot sensitivity versus specificity for something that could have several possible cut-offs (an ordinal or continuous variable).  However, they have plotted a single value on these curves and then drawn lines to the corners.  If you only have one data point then it’s really just a dot and therefore you shouldn’t really use an AUC (Area under the curve) analysis. Have a look at the graphs below for interest, but bear in mind that the most sensible way to look at this data is to consider the specificity of the opinion of the clinician.

It’s also worth asking what we mean by sensitivity in this study. Arguably it’s irrelevant as all patients are at suspected disease (that was one of the inclusion criteria) but in order to plot a ROC curve you need a sensitivity and a specificity so it appears that they have plotted the sensitivity at the point of describing the pain as ‘typical’.

So, the graphs are interesting but I would not focus on them. The key finding is the specificity.

 Discriminatory ability of the typicality of chest pain for either acute myocardial  infarction, or significant coronary artery disease with and without high sensitivity  troponin T elevation.

The authors have looked at experience and in the paper discuss the fact that experienced clinicians have slightly better specificity than non-experienced. In terms of specificity experienced clinicians had a specificity of 65.8% vs non-experienced at 55.4%.  However, I’m not sure how much use that is. This data tells me that they are both rubbish at it. Sorry folks, but I suspect that you are just as poor at determining this as other clinicians around the world (including me). Although there is a difference it’s clinically unimportant and I’m not sure that we can use experience as a determinant of probability in clinical practice.

It’s also interesting to see that 685/912 patients in this study were seen and assessed by doctors with less than 2 years of emergency medicine experience. International readers may find this surprising but it’s typical of many UK EDs and I would argue that in many hospitals the proportion of patients seen by juniors is even higher. On a marginally political note and with no wish to offend any hard working junior docs this study does show that UK EDs have a seniority staffing issue even when assessing chest pain patients who are clearly a high risk group.

So in summary?

The bottom line from me is close to that of the authors. You’re a bit rubbish at determining whether your patient has AMI or IHD on the basis of gestalt, clinical judgement or gut feeling. Don’t guess – test.

 

vb

S

Questions for FRCEMers

1. On the ROC curve looking at the diagnosis of Acute Myocardial Infarction place a cross at the point where the test performs best (in statistical terms).

2. If 1000 patients were assessed by senior emergency physicians on the basis of this study and 300 of those patients were selected as having typical pain then how many of the 300 identified as having typical pain would have a final diagnosis of AMI assuming a specificity of 65.8%.

 

Before you go please don’t forget to…

Meducation in Virchester #FOAMed