Headache is a pretty common reason for presentation to the Emergency Department – so common, in fact, that it has two curriculum sections of its own (and we’ve already produced an induction post about headaches that you can find here). It can be tricky to find the risk/benefit balance for these patients given the host of possible nasty underlying pathologies headaches might represent, particularly when arranging MR scans can be difficult and/or delayed and CT scans mean a hefty radiation dose. Subarachnoid haemorrhage is always high on the list of differentials and something we don’t want to miss – Chris Gray’s post covers the basics of SAH management in excellent detail.
Research to help us risk-stratify our patients is very welcome and that’s the subject of the paper this JC post is concerned with. Perry et al from the Ottawa group have previously derived a Subarachnoid Haemorrhage Rule and earlier this month the validation of their rule was published in the Canadian Medical Association Journal (CMAJ). The paper is open access and, as ever, we encourage you to read the paper itself and draw your own conclusions before reading further.
What is this paper about?
This is a validation of a decision rule: the authors had previously derived a decision rule from a number of other SAH decision rules in a multi-centre cohort study and they believed that their new rule was the most useful for excluding SAH. The original JAMA paper is also open access; you can find it here.
In their derived rule, they described a particular patient set (age > 15years, new severe non-traumatic headache reaching maximum intensity within one hour), described some rule-out criteria (new neurology, previous SAH, brain tumours or history of recurrent headache – all sensible criteria for patients you definitely would or definitely would not want to investigate for potential SAH) and then suggested some high-risk variables. Their premise was that if one or more of these variables is present, these patients with the new severe non-traumatic headache reaching maximum intensity within one hour should be investigated.
- Age 40 years or older
- Neck pain or stiffness
- Witnessed loss of consciousness
- Exertional onset
- Thunderclap headache (instantly peaking pain)
- Limited neck flexion on examination
In this most recent paper, the authors are looking at a new cohort of patients and applying this rule in order to understand better how well it performs in helping us decide who we should and should not investigate with CT scan and, perhaps, lumbar puncture.
If you get in a bit of a muddle over sensitivity and specificity, you might want to start by listening to this podcast from a few years ago:
What did they do?
This was a prospective, multi-centre cohort study involving six Canadian Emergency Departments. Consecutive patients with headache were considered for enrolment; in addition to the Rule Criteria, the authors also excluded patients with GCS <15 and those presenting > 14 days since headache onset, both of which seem reasonable. They mention exclusion of patients with head trauma in the preceding 7 days but I thought this was already included in their initial Rule Criteria (severe non-traumatic headache). I suppose it does make sense to put a timescale on the trauma aspect for the sake of clarifying decision-making.
There were some additional exclusions: transferred patients with known SAH (fair enough); 3 or more similar headaches in the past 6/12; patients reattending within the same clinical episode if they had already had CT and LP assessment; patients with papilloedema on fundoscopy; patients with new neurological deficits; patients with previous SAH, known aneurysm, brain tumours, VP shunts or hydrocephalus. Again, these criteria seem sensible as any of these history or examination findings should prompt us to think differently about the likelihood of intracerebral pathology and our diagnostic decision-making.
For the remaining patients, a standardised form was used to record the presence or absence of the Rule Criteria. Clinicians participating were asked to answer two questions after completing the standardised form: does the rule suggest this patient should be investigated [yes/no]; how comfortable would you be in using the rule for this patient [5 point Likert scale]?
The outcome of interest was subarachnoid haemorrhage (unsurprising!): the authors defined this as blood present on unenhanced CT brain (from final radiology report), xanthochromia on CSF analysis, presence of red blood cells in final LP CSF sample PLUS an aneurysm or AVM on CT angiography. These definitions seem appropriate – I would definitely consider these criteria diagnostic of SAH. The reason for the composite outcome is the current state of practice: in most institutions, CT is performed as first line if the patient within 6h of headache onset and considered sensitive enough for rule-out of SAH (provided the CT scanner gives high resolution images – this comes from the original Perry et al paper back in 2011). In patients presenting >6h from headache onset, or if CT cannot be performed in this timeframe, LP is traditionally performed at >24h from onset for the presence of xanthochromia. As such, the study in question could not demand that all patients were investigated by CT scan and LP as this would not reflect current practice (and would be unlikely to obtain ethics approval). The authors added an additional telephone follow-up at 1 and 6 months for participants not undergoing CT or LP to try to capture all possible outcomes for these patients, and undertook medical record review for similar reasons.
The researchers then set out to calculate test characteristics (sensitivity, specificity and likelihood ratios) for the Ottawa SAH Rule, aspiring to sensitivity as close to 100% as possible and predetermining a target sample size of 1200 to achieve this (expecting 75 diagnoses of SAH). This is important in the context of ruling out SAH but we must remember that high sensitivity comes at a cost to our specificity and that a test performing well on a rule-out basis might rule-in too many patients to be of clinical use in that regard (think of the many reasons for troponin elevation).
What did they find?
The first thing to note is that they didn’t quite achieve their recruitment aim, managing to include 1153 patients including 67 with SAH. For this cohort the sensitivity of the rule was 100% (95% confidence interval 94.6%-100%) with a specificity of 13.6% (95% confidence interval 13.1%-15.8%) . The authors propose adding in data from their original cohort (which I personally think is a little cheeky) to improve the sensitivity to 100% (95% confidence interval 98.4%-100%).
There was some disagreement between the clinician assessment of the patient criteria according to the rule when compared with the researchers’ own assessment, in 5.1% of subjects the clinicians had determined that the rule was “negative” while the researchers found it “positive”, although there were no cases of SAH in this subgroup.
In terms of clinician satisfaction with the rule, 9% of treating clinicians were either uncomfortable or very uncomfortable with using it for their particular patient.
The authors have also looked at investigation rates in this group of patients, finding that 88% underwent neuroimaging with a further 1% having an LP. They have calculated that, had the SAH rule been followed, the rate of investigation overall would have dropped to 84.3%. Hmmmmm. That’s not particularly impressive…
Issues and challenges with this paper?
Ryan Radecki at EM Literature of Note has summarised the specificity issue in a very eloquent and not-very-British way (which is why I’m directly quoting him here – sometimes our American colleagues are just better at getting to the nitty gritty):
…Their route to 100% sensitivity is, essentially: everyone needs evaluation. This can be reasonable when the disease is life-threatening, such as this, but the specificity is so poor in a population with such a low prevalence the rate of evaluation becomes absurd.
Ryan is reminding us here that while prevalence does not affect the sensitivity and specificity (the performance of the test, which in this case is the Ottawa SAH Rule), it does impact the positive predictive value – so at a very low prevalence, the positive predictive value (that is, the likelihood that a person with a positive test result actually has the disease) becomes ridiculously small because there are so many patients testing positive who do not actually have the disease. There’s a great explanation of this point here.
There are issues around how meaningful the outcome diagnosis was too, as it includes some small non-aneurysmal SAHs which did not require any treatment – is this level of sensitivity worth the amount of testing we would undertake for ultimately disease-negative patients? Let’s remember that in the paper there were only 67 patients with SAH while 937 had CT scans without SAH and 444 underwent lumbar puncture with an alternative ultimate diagnosis. That means we are scanning and lumbar puncturing a lot of people – does this reflect our current practice? Of course with serious conditions we expect to have a degree of over-testing – if everyone we scan has a subarachnoid haemorrhage, we should definitely be scanning more patients! But how far should we take this? I suspect that despite the potential seriousness of a SAH diagnosis, this study does not do much to help us reduce overtesting and potentially exposing patients to test-related harms.
What is interesting is the prevalence data around SAH: we can see from this cohort that the patients who had the outcome of interest were older (mean age 55yrs), with a shorter time from onset of headache to peak (median 30mins vs 60mins in the patients without SAH; the IQR in the SAH group was 3-120mins vs 3-600mins in the non-SAH group) and higher rates of impaired consciousness (9% vs 2.9% in the non-SAH group). Neck stiffness (or pain), vomiting and arrival by ambulance were also significantly more common in the SAH group. So there are some data here to aid us in our clinical assessment of patients, although I suspect we might have guessed these things anyway.
The time from onset of headache to peak is particularly interesting to me. This year I looked at our LP patients, because in Australia these patients are admitted to the ED observation ward and generally the LP is performed by the Emergency Physician (as opposed to the UK, where these patients are usually referred to the inpatient medical team for ongoing assessment). I was looking (retrospectively) at whether the SAH Rule would help us to perform fewer LPs in these patients – and over three months’ admissions for LP in the context of headache and suspected SAH, there were no patients diagnosed with SAH by LP following a normal CT brain (usually performed outside the 6h window). What was particularly interesting was how bad we are at documenting headache onset and time to peak – notes often included the phrase “sudden onset” but without timescales, so it was nearly impossible to work out whether these patients would have warranted testing in accordance with the SAH rule from notes review alone.
What does it mean for us in practice?
SAH remains a low-prevalence, high-significance condition and we don’t yet have a brilliant way of ruling out the condition without subjecting lots of patients to radiation (CT scan) and unpleasant procedures (LP). These tests are not without harm. Like Ryan, I’m not sure this rule actually helps us reduce our testing rates in a meaningful way because although it’s likely the rule will ensure we do identify all the patients who genuinely have had a subarachnoid haemorrhage, it still captures a huge number who don’t have the disease at all.
What I can say is please, please document clearly the time of onset and time to peak for sudden, severe headaches – I have no idea how much this affected outcomes in the paper (maybe not much, due to the Hawthorne effect) but I can tell you it certainly makes it tricky to assess local practice and if we choose not to investigate patients, it might be of significance in the defence of our clinical decision-making.
Kudos, though, to the researchers for trying to find clinical ways to avoid overtesting. It’s an important mission even if we haven’t quite found the definitive answer yet.
Before you go please don’t forget to…
- Subscribe to the blog (look top right for the link)
- Subscribe to our PODCAST on iTunes
- Follow us on twitter @stemlyns
- See our best pics and photos on Instagram
- PLEASE Like us on Facebook
- Find out more about the St.Emlyn’s team
EMLit Of Note – It’s SAH Silly Season Again!