You get an email from Sister Astir….
You need to have a word with Dr Dawdling please. He hardly saw any patients yesterday and there was a massive wait in minors. The nurses were tearing their hair out, even Jim, and he’s bald!
Can you have a word and sort him out please!!!
Some questions for you………..
- Are you a slow or fast emergency physician?
- Does speed matter?
- What does work rate tell us?
- Should we publish this data locally?
Why am I asking this? Well for several reasons really. At the moment it is because I have recently had several conversations with colleagues about work rate and work intensity in the ED. Most of these have revolved around the perception that some doctors (usually junior docs) do not work hard when on shift. This is something that has been said for many years and now that we have electronic databases and patient tracking systems it is increasingly easy to pull data out about who saw how many patients, of what type and when. I hear tales of performance league tables being pinned to the door in some departments with those at the bottom of the list being named and shamed, those at the being given special status and in one story even being given ipods for seeing the most number of patients during an attachment.
Does this worry you?
I hope it does, as it worries me a lot so let’s explore why.
This post is written from the perspective of an educational supervisor exploring numbers with trainees, but it is arguably just as valid for any one of us working in Emergency Medicine. It is very much from a UK perspective and I understand that pressure, payment, activity and productivity are considered differently around the world. What applies here may not be relevant to other health economies.
1. Are you slow or fast?
It’s a bit like asking if you are tall or short. Clearly there is going to be a range, with an average and it’s probably going to be roughly normally distributed. In other words to know if you are abnormal you sort of need to know what is normal. So what is normal for a junior doc in a UK ED? Well there is some data out there on this Aruni Sen and colleagues looked at grade and work rate in a North Wales ED and showed (amongst other things) that there is a significant variabilty between the numbers of patients seen within an equivalent grade. That’s also important as if you are going to make comparisons then it seems only fair that comparisons take place between docs at the same stage of training, doing the same shifts, and over a sufficiently long period that variation due to holiday, nights, leave etc. flattens out. That’s one of the reasons why we usually only look at data once a full cycle of shifts (at least 16 weeks, but usually more) has taken place.
So taking all that into account what we are talking about is a variability from the average. So what sort of difference would you expect to see amongst docs of a similar grade over a period of months? 10%, 20%, 50%,…………, 100%.
Have a guess
Well I have looked at data around this for sometime and the variability is pretty high. Typically the difference will be 80-100% if you have a larger enough number of docs and enough time.
Here is sample data from 41 docs at the same grade, same hospital, same time period, same shifts, same holidays. Thoughts???
The average here is about 660 patients but the range is from 467 to 1012 which is interesting. Sure there will be variation around a mean, but that is quite large. Interestingly the majority of docs are fairly consistent and the variability is greater at the high end rather than the low end. This variability is a fairly consistent feature amongst docs in the ED over the last few years that I’ve been able to look at the data.
So, such data (with all the caveats above) can give an idea of whether you see lots of patients. What is clear is that some docs see more patients than others, but is that the full story, and do those at the bottom end need to be told to speed up?
2. Do numbers matter?
Yes and no. I am pretty sure that the next patient you see does not care how many ‘other’ patients you see over a 4-month period and that’s right. What matters to patients is whether you are going to look after them and if you are going to do your best for them. Surely nobody can argue with this…., or can they? Perhaps there is an argument that an EP looks after a population of patients in the department as much as they look after the individual. Whilst Mrs Miggins may love you to spend lots of time discussing her chronic health problems, in a system with finite doctor resource the time spent with Mrs M is inevitably at the expense of the time you could be spending with other patients (who may be more/same/less in need). In other words there is an opportunity cost related to work rate.
There is also the issue of complexity and patient mix. It will come as no surprise that it takes less time to see a broken finger than it does to see a broken heart (physiological or psychological). If you spend all your time in resus then numbers will be small, but you might just be making the real difference to mortality and morbidity. Similarly you could be seeing loads of patients in the minors end doing the work that (some people) think you are paid for, but for which your advanced training may arguably be superfluous. You should be able to look at case mix when analysing the data. That’s actually quite tricky to do in a way that reflects the needs of individual patients, but you can certainly get some idea by looking at triage categories. This will at least give you an idea of whether you and your colleagues are seeing a similar mix of patients. We find this the easiest way to subdivide the categories, but you could use other classifications like discharge diagnosis I guess. Again, you should be looking for variability in these areas to see if some clinicians seem only to be seeing certain types of patients.
Let’s remember that the argument proposed above about Mrs Miggins is closely related to performance targets and in the UK to the 98% target on 4-hour performance. There is therefore a conflict between individual patient care which is a very clinician focused outcome and patient population performance which is a systems outcome. BOTH are patient related outcomes owing to the opportunity cost if a clinician is slow at seeing patients.
3. What does work rate tell us?
In simple terms it tells us how many and if related to triage category the type and range of patients our doctors see. So, what do we typically find? In truth the range of patients seen varies quite a lot and the type of patients seen does not. It just seems that some docs get through the patients faster than others. Difficulties then arise as a leap can be made to describing this as poor performance – which is probably wrong because it is the wrong question. Measuring the number of patients seen tells us how many but it does not tell us why. As educational supervisors we need to challenge ourselves to explore the why question.
quantity AND/OR quality?
Patients seen per hour does not tell us whether the quality of care changes and indeed a recent article looking at Paediatric Emergency Physicians suggests that productivity does not correlate with quality measures such as admission rates and revisit rates. Now we can and should argue whether those are the only quality measures we should consider, but it is at least interesting to see people looking at it.
Back in 2009 ACEP looked at this and the document on benchmarking and productivity is worth a read. It basically highlights the difficulties in setting any particular benchmark and suggests using oneself as a baseline and working from there, we arguably used a similar concept of self-reflection and performance in the personal NNT post here at St.Emlyn’s. Perhaps, but I also think that in many departments the benchmark might better be placed around an average at a particular grade and rota. ACEP have also looked at the concept of RVUs as explained here (sort of as I find this really tricky to get my head around), but basically a measure of the workload required to see certain types of patients. I think this has more of a place in the US system than here in the UK and it’s not really applicable to trainees.
So what of quality as an unmeasurable? Does fast or slow relate to quality of care given to our patients? I suspect not and in my very unscientific experience I see variability at both ends of the scale.
Ed – Er, fine, but I have a slow trainee though….
We all know who the slow docs are right? Well perhaps not. Again this is anecdotal but it’s often quite tricky to get a feel for productivity just by guessing. You do need the numbers to some extent as there are few behaviours you may recognise that illustrate the difference between perception and reality……
- The machine gun: Works in bursts. Attacks the patient queue with gusto, sees a whole bunch of patients then wanders off. Prone to case selection (takes the easy ones). Numbers often appear high, but nurses get frustrated when enters scarlet pimpernel mode (they seek him here, they seek him there) between bursts. Perception – lazy. Reality – numbers often high.
- The circulator: Often found walking purposefully between different areas of the department. Sees patients in all areas, but often waiting for a second (third, fourth) opinion. Perception – busy, hard worker. Reality – numbers average/low.
- The socialite: Always busy, always doing stuff, chatting with colleagues and popular with staff. If asked people rate this doc as a really good clinician and hard worker. Perception – one of us, great, team player. Reality – numbers typically low.
- The plodder: Doesn’t really like EM that much. Here on rotation. Not a team player, heading for a career in non emergency speciality. Sees all patients in order. Just gets on with it, not much chat. Perception – slow. Reality - numbers often high (these are the real surprises).
- The holistic practitioner. Never embraces EM as a concept. Takes full histories and does complete exams. Always does blood tests. Phones relatives, the police, the RSPCA, makes cups of tea for patients and anyone else. Prefers to ask for advice from speciality teams rather than EM seniors. Perception slow. Reality – glacial.
- The superstar. May or may not be career EM. Keen to learn, knows when to ask about decisions. Perception – excellent. Reality – excellent.
So all a bit tongue in cheek, but I suspect you will recognise some of these caricatures.
So should we do anything about work rate?
OK, so this is my opinion only. Firstly, before you embark on a conversation with a trainee about work rate you need to be sure of the facts and ensure that the numbers are real. Such an interest is usually precipitated by complaints from the nursing staff who rapidly form perceptions about whether juniors are working well. So, look at the numbers and then it’s a question of whether it is worth having the conversation? What might you get out of the conversation and how would you approach it?
Before speaking to the trainee I would pull a sample of clinical records from a few shifts. Basically all the records from complete shifts to get an impression of what is being seen, what is being done, when it is being seen and whether the quality of care (record keeping at least) is up to speed. This can give some impression of quality, but with obvious caveats.
In my practice I rarely have conversations around this but if I do they start off something like this..
“I’ve had a look at the numbers of patients the docs at your grade see in the ED and interestingly it appears that you see fewer patients than I would normally expect for someone at your level. Can we talk about this as I think it would help us understand how you are working and there may be aspects of your practice that we might change”
What comes out of that is interesting. Sometimes it is a clinical issue, sometimes about adapting to EM ways, but more often than not there may be personal, health, confidence or skills issues that come out of the conversation and that is something we can work with.
So numbers are not a problem, but they may be a symptom of something else. Any exploration of this with a trainee should not be predicated on making them faster. That should not be a goal of supervision, rather a side effect of investigating what is happening at the moment.
Whilst putting this together I spoke with my colleagues and we came up with the following graphical description of how we sometimes see trainees in terms of speed and safety. Again this is slightly tongue in cheek and we might argue that the Y-axis is better described as competence (which will vary with grade, experience, role etc.).
Whilst we all want fast, clever, safe, super docs, we know that this cannot always happen. Patient safety is the most important aspect of our practice and as supervisors we need to make sure that we are aware of how trainees are performing and ensuring that they are supported in their decision making. My concern around forcing docs to go faster (directly or indirectly through incentives) is that it may push them to not seek advice, discuss and basically maintain good clinical supervision. If that happens then safety is the issue. So the last thing we want to do is drive docs down from a position of safe practice to one of faster, but unsafe practice. Fortunately, the slow doc with concerns around competence will probably be known to you through other methods (at least they should be).
So basically, whilst I’d like all our docs to be fast and safe, I’ll settle for safe and we need to be cautious about how we approach increasing speed as I am worried that we could turn a safe plodder into an unsafe sprinter.
4. Should we publish this data locally?
This is a question that really divides opinions between managers, seniors and juniors. As we have discussed there are many variables in the data such as opportunity, holidays, shift patterns etc. so my personal opinion is that publishing this data on a very frequent basis is a really bad idea. It risks equating speed as the sole measure of junior doc performance in the ED and that’s a very bad message to send out. If used as a target in this way it is likely to lead to abnormal behaviours (e.g. preferring to see easy/quick patients).
I do know of departments that keep a weekly league table of patients seen and as we know there are those that offer prizes to ‘fast’ clinicians. I really don’t think this is a good idea to be honest as whilst it might appear to have benefits in making people work faster, there are concerns that this may impact on safe decision making and clinical practice.
However, we are entering a time when physician performance is increasingly being integrated into appraisal and now revalidation in the UK. A component of that for nearly all doctors will be some element of performance data in terms of activity, complication rates and hopefully outcomes as well. With this in mind I do usually publish the data to junior docs at the end of their attachment so that they get a feel (good and bad) for how it feels to work in a healthcare system that is starting to get to grips with measuring ‘performance’ through activity. So, this may seem harsh to some, but it is arguably the future of physician management in the UK.
The data is there so we cannot ignore the fact that docs and managers will look at it. As trainers it is also potentially useful in giving us objective evidence.
So it you are told that one of your juniors is ‘slow’. Here’s some do’s and dont’s
- Check the data
- Look at the case mix
- Look at a sample of medical records
- Do use the same processes for everyone
- Tell people in advance what you do about records and work rates.
- Don’t equate speed with quality.
- Don’t trust your instincts – check the data.
- Don’t pick on individuals just because you or others may not like them