JC: Duel of the Rules: Which decision rule for head injury in children?

Ladies and Gentlemen!

Dual of the Rules

The assessment and investigation of paediatric head injury is a case study in clinical risk management. Similar to chest pain in grown up medicine, it’s a frequent presentation, with a rare but potentially devastating outcome. It’s these kind of patients where we as ED docs come into our own as diagnosticians (or do I mean  probablesticians? 😉 ). Head injury in children is also something of an interest of mine; both as a doc, and as the parent of a rambunctious, seemingly fearless, 6 year old boy.

While we all like to feel that our razor sharp clinical acumen is sufficient to pick out the badness, it is  nice to have some evidence based back up for our hunches and ‘gut feelings’. So in order to help guide us through choppy diagnostic waters, a great deal of time and effort has been expended to develop clinical decision rules (CDR’s) (or instruments, if you prefer). For chest pain we have TIMI, PERC, HEART to name just a few. Unsurprisingly, paediatric head injuries also have a selection of decision rules to choose from. Of course, this brings with it yet another question; which rule to use? Which is best? This is the question the authors of this study have tried to answer. They have taken their own cohort, and pitted three decision rules against each other in a Battle Royale!

[DDET Contenders…. ready! ]

PECARN

PECARN

In the blue corner, all the way from the US of A, weighing in with a super heavy weight derivation cohort of 42000 patients…..It’s PECARN!

In the red corner, it’s our local boy, from right here in Virchester, and basis of the NICE guidance on paediatric head injury….. It’s CHALICE!

And in the um, other corner, last but certainly not least, …. from Pediatric Emergency Research Canada, it’s CATCH!

[/DDET]

[DDET We’ve got three fine decision rules here, what kind of form are they in? ]

These rules have been compared to each other before. Pickering et al compared the three rules and concluded that PECARN was the most sensitive with an acceptable specificity. This excellent article by Lyttle et al compared the three decision rules and the methods by which they were derived in great detail, and is well worth a read for a textbook example of how to assess a CDR. It is also worth mentioning that PECARN has been validated in a

CHALICE

CHALICE

separate (but demographically identical cohort) as part of the same study in which it was derived in. CHALICE has not been externally validated, but has been extensively implemented and retrospectively analysed since its inclusion in the NICE guidance.

[/DDET]

[DDET So how’s this match shaping up? ]

This is the first paper applying the rules to a new, prospectively collected cohort. Their patients were recruited from the ED at a level 2 paediatric trauma centre in the US. 1009 patients were assessed aged less than 18yrs, who had sustained a head injury within 24 hours of presentation.

 

F1.large_

CATCH

Patients with a GCS <13, and those with a coagulopathy, were excluded. There were some eligible patients missed, but they seemed to be demographically similar to those included. While not identical to any of the derivation cohorts from the original studies, this population is probably similar to what we all see as ‘minor’ paediatric head injuries, and therefore the population we would want to apply a CDR to. The outcomes looked at were death from TBI, neurosurgical intervention, ventilation for >24 hours due to TBI, which all seem reasonable and appropriately patient orientated to me – focusing on what happens to the patients rather than findings on CT. By the authors own power calculation, a cohort of 1000 patients would be sufficient, so we’re happy there.

[/DDET]

[DDET Leeeeeeeeeeeet’s get ready to rumble! ]

Diagnostic performance of the PECARN, CHALICE and CATCH rules

Diagnostic performance of the PECARN, CHALICE and CATCH rules

As you can see, PECARN comes out fighting with no misses and 100% sensitivity. CHALICE put up a valiant defence with a 85% specificity but falters with a surprisingly low 84% sensitivity. CATCH puts in a solid bout, but is unable to come out on top. So no knock-outs today, so were going to have to ask the judges to make a decision.

[/DDET]

[DDET Judges’ verdict]

 

Kids and radiation don't mix.

Kids and radiation shouldn’t mix unless they really have to

One hundred percent sensitivity – now that’s the headline figure right there. When we’re dealing with kids and brain injuries and neurosurgery, we want as close to perfect as we can get, and PECARN seems to deliver. This is consistent with the comparison by Pickering et al that also put PECARN on top. Of course, it’s not all about the offence, a CDR needs a good defensive technique too. It’s all well and good to go all out with the sensitivity, but if you haven’t got the specificity, you’re gonna have a problem. Here, CHALICE performs the best, 85% is a impressive specificity for any decision rule, especially if we are concerned about zapping kids brains with CT scans.

The 84% sensitivity of PECARN is a surprise, this is lower than we would have thought. Even more worrying is the fact that the updated NICE guidance has relaxed some of the criteria, so we are not CT’ing all of the children who are ‘CHALICE positive’. This is a concern, and it’s set me off searching for an explanation for this surprising under-performance. The first thing that catches the eye is those confidence intervals, which for the sensitivity results are pretty big. For CHALICE, the sensitivity is 84%, but the 95% CI is 60-97%. The authors themselves point out that as their cohort is far smaller than that of any of the derivation studies, we should be wary about making absolute conclusions. Could it merely be a lack of power that has resulted in CHALICE’s poor performance? That said, even the top end of the confidence interval, a sensitivity of 97%, may not be sufficient for our needs.

[/DDET]

[DDET Let’s see the replay…  What did CHALICE do wrong?]

CHALICE ‘missed’ three patients, and the authors give us the information to look at. The first is a 17 year old who was hit by a rock. He was picked up by PECARN and CATCH with a severe headache, which CHALICE doesn’t include. Can’t argue with that one. The second was a 16 year old, who scored PECARN and CATCH positive with a GCS <15. He was not picked up by CHALICE, presumably as his GCS was 14 (CHALICE would CT at a GCS of <14). The third was a 15 yr old who fell 2.5 metres. This counts a significant mechanism in PECARN and CATCH, but not in CHALICE. Why not? Because CHALICE would CT after a fall of 3 metres. For me, those last two are pretty subjective, and easy to imagine that they could just have easily have been assessed as being CHALICE positive.

[/DDET]

[DDET Last minute allegations of fight fixing! ]

If you take a look at the PECARN decision rule – it has three final outcomes. A low risk, ‘CT not recommended’ group, a high risk ‘CT recommended group’ and also a moderate ‘observation vs CT’ group. The authors of this comparison treated both the moderate and high risk patients as ‘postive’. This gives PECARN an advantage when calculating its sensitivity, and may not reflect how the rule would be used in the real world, where some of those children categorised as moderate risk may have been missed.

[/DDET]

[DDET And the paediatric head injury CDR champion of the world is…..]

I was tempted to cop out and call it a draw, but… Sorry Manchester, but the title has gone overseas. PECARN has taken it. While the specificity is not as good as some, they edge it with an impressive zero miss rate.  The sensitivity of PECARN may have gotten a boost from the way the authors applied the rule. In real life, we may not send every PECARN positive kid to CT, but we would be at least observing them. CHALICE may have had some bad luck with some decisions not going their way, but on the day, its the numbers that count. 

[/DDET]

6 Comments

  1. Simon carley

    Hang on a minute! I will declare a Virchester bias about CHALICE here but let’s stop and think about why it missed three ‘kids’.

    The chalice study http://m.adc.bmj.com/content/91/11/885.full derived a CDR in children <16 and yet it is declared a failure here because it failed to spot a head injury in three patients aged 15,16,17. The rule should only have applied to one of these patients.

    This is arguably a problem of misapplication of a good rule rather than a bad rule.

    First define what a child is, and in tes of chalice those patients should not have had the rule applied.

    Now who's going to reanalyse the data with those patients excluded?

    S

    Reply
    1. drgdh (Post author)

      Unbelievable…. Even when he’s on holiday Prof is spotting stuff I’ve missed…. 😉

      Yes indeed, CHALICE was derived in kids <16 years old. Two of the patients it has 'missed' in this cohort are older than that. Is PECARN exonerated?

      On the face of it, excluding these patients would improve the sensitivity. Without seeing the whole data set I can't say what the effect this would have on the rest of the numbers though.

      The authors have addressed this issue though. In the limitations section they point out that their cohort does not match the derivation cohorts for any of the three rules. They do say that when they re-analysed the rules using restricted cohorts to 'match' the derivation cohorts, the results were much the same. No numbers provided though.

      Reply
      1. Simon Carley

        Hi Gareth,

        Sorry it took so long to get your comment on the site, it went to the spam queue!

        As for the authors assertions that it would make no difference then this simply cannot be true can it? If two of the three missed neurosurgical patients were ineligible for the CHALICE rule then the sensitivity moves up considerably.

        We would indeed need to see their data, but I do not believe that mathematically it would not make a difference – it simply must!

        Fancy writing a letter to the journal?

        Thailand was wonderful by the way.

        vb

        S

  2. Charlotte

    Are the ‘missed’ injuries clinically significant? We had a kid pecarn positive (severe headache), nice etc. Negative. CT showed extradural. Neurosurgeons observed for three days and discharged. Was the extra radiation clinically beneficial despite the positive result?

    Reply
  3. Pingback: The LITFL Review 134 - LITFL

  4. Pingback: Orbital Floor Fractures | Don't Forget the Bubbles

Thanks so much for following. Viva la #FOAMed

Translate »