Welcome back to the 4th part of the series looking at understanding risk and diagnosis in Emergency Medicine. In part 3 we looked at how tests with apparently high sensitivities (e.g. 98% sensitivities) actually mean that you miss 1 in 50 patients with disease. Now to be honest, if you just look at that figure in the context of a disease like PE or Acute Myocardial Infarction then it makes for pretty uncomfortable reading. I know of few emergency physicians who are immediately comfortable with that kind of figure, and to be honest even as I write it down it makes me stop and think again (and I know the answer to the question that follows).
So how can it be that we apparently miss 1 in 50 PEs in the ED, yet we are not in court every other week on the back of a legal case, or worse in the coroner’s court as a result of a patient’s death? To explain this we need to start thinking and explaining other factors that eventually affect the performance of a test in the real world that we work in. The best way to do this will be for us to start with an example and for ease of convenience let’s stick with Pulmonary embolus. I’ve chosen PE as it is a topical subject and also the PERC rule to help illustrate the issues around risk. This post is not really about PE, more about how diagnostic test work in practice. It’s just a convenient example….., oh and I hope my maths is right as we go along. If its not I’m sure you will let me know
The exact prevalence of disease will depend on a number of things but we can use data from the literature to help us so let’s take the incidence from Kline’s paper on the investigation of PE (6.9%) and express that as a diagram based on 100 patients who might turf up in your ED.
The white squares represent patients who do not have PE, and the red ones are those that do. Now of course at this point you have no idea of who has and who does not have the disease so you are going to start investigating them as you would not want to send 6.9% of your patients home with an undiagnosed PE!
Patients represented by the yellow circles (roughly 79% % of patients) are those that are PERC positive and will need investigating further. Those in the green circles (roughly 21% of patients) are classified as PERC negative and are the ones that you might consider sending home with no further testing.
So after applying PERC the patients who do have a PE remain predominantly in the group who need further investigation and that’s fine. However, there is a little bit of red left in the green zone, i.e. in the group that we are considering sending home with no further investigation.
That’s just 1 patient missed for every 500 investigated as a low risk PE patient. This is clearly more reassuring. At Virchester we investigate about 700 patients for PE every year so we would expect to be missing 1-2 patients per year if we applied a strategy such as the one described above.
We think that’s pretty good and we will argue that it is an acceptable miss rate. It means that there is a group of patients who do not need to go onto more invasive testing at this stage (so long as you are comfortable with that miss rate). Although we are still missing one in 50 PEs the number of PEs missed per person investigated is actually quite small.
In our case study around the PERC rule if we were to increase the prevalence to 30% then we would end up missing more patients with PE (about 5 per 500 investigated) and at some point you would need to say that this is not an acceptable strategy.
If you want to play around with how prevalence changes how your diagnostic test will work according to the number of people investigated then download this excel file and play around with the numbers on a hypothetical population of 1000 patients in your ED. Just input the sensitivity, specificity and prevalence and you can see how it will perform.
In Virchester we really quite like this calculator as to be honest the sums make our heads hurt and it’s quite useful to see how the performance of a test changes with a change in prevalence. We prefer to see the effect as a natural frequency rather than percentages and statistics. It makes it feel more real to us as clinicians and you can also use it with patients if you are so inclined. If you want to work things round the other way by filling in tthe event rates to see what sens and spec (plus a whole bunch of other stuff as well) then I would suggest using this fantastic online calculator.
This is one of the many reasons why it’s so important to know about the characteristics of your own patients when assessing new diagnostic tests.