Duel of the Rules: Which decision rule for head injury in children?

Ladies and Gentlemen!

Dual of the Rules

The assessment and investigation of paediatric head injury is a case study in clinical risk management. Similar to chest pain in grown up medicine, it’s a frequent presentation, with a rare but potentially devastating outcome. It’s these kind of patients where we as ED docs come into our own as diagnosticians (or do I mean  probablesticians? ;) ). Head injury in children is also something of an interest of mine; both as a doc, and as the parent of a rambunctious, seemingly fearless, 6 year old boy.

While we all like to feel that our razor sharp clinical acumen is sufficient to pick out the badness, it is  nice to have some evidence based back up for our hunches and ‘gut feelings’. So in order to help guide us through choppy diagnostic waters, a great deal of time and effort has been expended to develop clinical decision rules (CDR’s) (or instruments, if you prefer). For chest pain we have TIMI, PERC, HEART to name just a few. Unsurprisingly, paediatric head injuries also have a selection of decision rules to choose from. Of course, this brings with it yet another question; which rule to use? Which is best? This is the question the authors of this study have tried to answer. They have taken their own cohort, and pitted three decision rules against each other in a Battle Royale!

PECARN

PECARN

In the blue corner, all the way from the US of A, weighing in with a super heavy weight derivation cohort of 42000 patients…..It’s PECARN!

In the red corner, it’s our local boy, from right here in Virchester, and basis of the NICE guidance on paediatric head injury….. It’s CHALICE!

And in the um, other corner, last but certainly not least, …. from Pediatric Emergency Research Canada, it’s CATCH!

These rules have been compared to each other before. Pickering et al compared the three rules and concluded that PECARN was the most sensitive with an acceptable specificity. This excellent article by Lyttle et al compared the three decision rules and the methods by which they were derived in great detail, and is well worth a read for a textbook example of how to assess a CDR. It is also worth mentioning that PECARN has been validated in a

CHALICE

CHALICE

separate (but demographically identical cohort) as part of the same study in which it was derived in. CHALICE has not been externally validated, but has been extensively implemented and retrospectively analysed since its inclusion in the NICE guidance.

This is the first paper applying the rules to a new, prospectively collected cohort. Their patients were recruited from the ED at a level 2 paediatric trauma centre in the US. 1009 patients were assessed aged less than 18yrs, who had sustained a head injury within 24 hours of presentation.

 

F1.large_

CATCH

Patients with a GCS <13, and those with a coagulopathy, were excluded. There were some eligible patients missed, but they seemed to be demographically similar to those included. While not identical to any of the derivation cohorts from the original studies, this population is probably similar to what we all see as ‘minor’ paediatric head injuries, and therefore the population we would want to apply a CDR to. The outcomes looked at were death from TBI, neurosurgical intervention, ventilation for >24 hours due to TBI, which all seem reasonable and appropriately patient orientated to me – focusing on what happens to the patients rather than findings on CT. By the authors own power calculation, a cohort of 1000 patients would be sufficient, so we’re happy there.

Diagnostic performance of the PECARN, CHALICE and CATCH rules

Diagnostic performance of the PECARN, CHALICE and CATCH rules

As you can see, PECARN comes out fighting with no misses and 100% sensitivity. CHALICE put up a valiant defence with a 85% specificity but falters with a surprisingly low 84% sensitivity. CATCH puts in a solid bout, but is unable to come out on top. So no knock-outs today, so were going to have to ask the judges to make a decision.

 

Kids and radiation don't mix.

Kids and radiation shouldn’t mix unless they really have to

One hundred percent sensitivity – now that’s the headline figure right there. When we’re dealing with kids and brain injuries and neurosurgery, we want as close to perfect as we can get, and PECARN seems to deliver. This is consistent with the comparison by Pickering et al that also put PECARN on top. Of course, it’s not all about the offence, a CDR needs a good defensive technique too. It’s all well and good to go all out with the sensitivity, but if you haven’t got the specificity, you’re gonna have a problem. Here, CHALICE performs the best, 85% is a impressive specificity for any decision rule, especially if we are concerned about zapping kids brains with CT scans.

The 84% sensitivity of PECARN is a surprise, this is lower than we would have thought. Even more worrying is the fact that the updated NICE guidance has relaxed some of the criteria, so we are not CT’ing all of the children who are ‘CHALICE positive’. This is a concern, and it’s set me off searching for an explanation for this surprising under-performance. The first thing that catches the eye is those confidence intervals, which for the sensitivity results are pretty big. For CHALICE, the sensitivity is 84%, but the 95% CI is 60-97%. The authors themselves point out that as their cohort is far smaller than that of any of the derivation studies, we should be wary about making absolute conclusions. Could it merely be a lack of power that has resulted in CHALICE’s poor performance? That said, even the top end of the confidence interval, a sensitivity of 97%, may not be sufficient for our needs.

CHALICE ‘missed’ three patients, and the authors give us the information to look at. The first is a 17 year old who was hit by a rock. He was picked up by PECARN and CATCH with a severe headache, which CHALICE doesn’t include. Can’t argue with that one. The second was a 16 year old, who scored PECARN and CATCH positive with a GCS <15. He was not picked up by CHALICE, presumably as his GCS was 14 (CHALICE would CT at a GCS of <14). The third was a 15 yr old who fell 2.5 metres. This counts a significant mechanism in PECARN and CATCH, but not in CHALICE. Why not? Because CHALICE would CT after a fall of 3 metres. For me, those last two are pretty subjective, and easy to imagine that they could just have easily have been assessed as being CHALICE positive.

If you take a look at the PECARN decision rule – it has three final outcomes. A low risk, ‘CT not recommended’ group, a high risk ‘CT recommended group’ and also a moderate ‘observation vs CT’ group. The authors of this comparison treated both the moderate and high risk patients as ‘postive’. This gives PECARN an advantage when calculating its sensitivity, and may not reflect how the rule would be used in the real world, where some of those children categorised as moderate risk may have been missed.

I was tempted to cop out and call it a draw, but… Sorry Manchester, but the title has gone overseas. PECARN has taken it. While the specificity is not as good as some, they edge it with an impressive zero miss rate.  The sensitivity of PECARN may have gotten a boost from the way the authors applied the rule. In real life, we may not send every PECARN positive kid to CT, but we would be at least observing them. CHALICE may have had some bad luck with some decisions not going their way, but on the day, its the numbers that count. 

 

When Sick Means Sick: Emesemantics and Vomiting in Kids

Image from Healthysimulation.com

When I was preparing for my talk at smaccGOLD I contacted the wonderful Ross Fisher, a Paediatric Surgeon, to ask him for some tips; what did he wish all ED docs and GPs knew about paediatric surgical patients? He was kind enough to provide me … [Continue reading]

Who’s That Girl? Impostor Syndrome at #SMACCGold

1795558_808058995888848_1527192652_n

I want to let you into a secret. I’m an impostor.   OK, that’s not quite true. I’m a 34-year-old (shock!) Emergency Medicine trainee with a penchant for karaoke and a passion for education, but I’m really not anybody special. I play the … [Continue reading]