Last week in the Journal Club we looked at this paper on the introduction of ED scribes from the American Journal of Emergency Medicine http://www.sciencedirect.com/science/article/pii/S0735675713001940
Understanding this paper requires some grasp of the difference in processes between US and UK Emergency Departments: for a basic description click below.
Most American EDs already use an electronic medical record whereas this practice is rare but of increasing interest in the UK. The term covers systems in which health care professionals record attendance and billing data electronically but can also be used to describe electronically recorded information on a patient which may or may not be available between different healthcare providers. In the UK Advantis ED (used in Stockport) describes itself as the first paperless mobile system to track a patient’s journey through the ED.
The ED in the paper was introducing CPOE (computerised physician order entry), an electronic ordering system similar to those used in the UK to order bloods and radiology. They already use an electronic medical record (EMR) but thought that this was going to take doctors away from the bedside to the computer screen for an extended period of time. The department also introduced scribes later – these are students who are paired with the attendings (since resident and ENP consultations were excluded) and document the history and examination findings straight into the EMR. Where scribes aren’t used in the ED, the doctors do this afterwards or via pre-printed tick sheets for specific presenting complaints. There are obvious advantages to the EMR in terms of legibility and traceability as well as the authors’ ideas about reduction in errors although it’s not clear how viable this is and it certainly isn’t what they measured.
The paper reads as twofold; they want to demonstrate the value of scribes, yes, but also to demonstrate the detrimental effects of CPOE on patient throughput.
This is defined in the paper but unusually not until the discussion section. It’s worth reading this bit first;
“The primary role of the scribe is to complete the EMR. This includes accompanying the ECP into each patient’s room, documenting the initial history, review of systems, and physical examination. After the initial evaluation, the scribe then records all procedures, consultations, and re-evaluations. Documentation of electrocardiogram, pulse oximetry, and rhythm strip interpretation as well as critical care time is also provided. Finally, the scribes are responsible for detailing all diagnoses, treatment plans, prescriptions, and discharge/follow-up information for each patient. Scribes further assist the ECP by tracking results from laboratory and imaging tests, keeping a task list, cross-checking consultation, and admission requests with the private attending staff’s preference guide, and ensuring completion of all charts before the end of the shift.”
The authors begin by stating the case for reduction of clinical errors due to documentation, in particular legibility and use of non-standard abbreviations. Although the electronic medical record will address this, that’s not actually what the paper is about. Scribing has obvious face validity in terms of speed of assessment but potentially not for the reduction of errors. The investigators are measuring benefits on patient throughput and this reads as though they are unsure what they are trying to demonstrate – the effect of scribes in the ED or the detrimental effect of the computerised physician order entry (CPOE). Looking at both changes in the same paper opens the research up to a number of confounders – not to mention the correlation/causation argument a before and after study necessitates…!
From a personal point of view, here are our current ED practices:
Nat: I tend to take the notes into the cubicle and document past medical history, immunisation history, drug history and allergies during the consultation (the things I think I’m likely to forget!). After leaving the cubicle I document history, examination and plan afterwards (ideally sitting down at a desk somewhere to maximise the legibility of my handwriting). Do I think a scribe would reduce the time it takes me to see patients? Maybe!
Damian: My personal practice is to explain to the parents that I am going to write down the history as they explain it to me and gain permission for this. I scribble as they talk and then put my notes down to do a full examination. I explain to the patients my plan and leave the consultation room. I write up my examination and plan outside. Not being able to write at all would save significant time however only if the data entry section was as fast as I can write/think.
The before-and-after study allows evaluation of a specific intervention at a set time point across a group of participants. It is possible to have a randomised before-and-after study which involves a control group (Cochrane reviews 2007) . This would have been difficult for this study as a similar hospital about to introduce an EMR may have been difficult to find. It would also have been very difficult to randomise individuals to use scribes or not within the same centre, considering the training and quality control measures required.
Problems with before-and-after studies are well recognised and include maturation, reactive and regressive effects
The population improve in skill and confidence between before and after testing independent of the intervention
Typically this results from pre-testing, especially prior to the intervention, which itself educates the respondents. In this case the effect may arise from any education on the use of the system itself prior to its implementation.
Any post-intervention analysis forces the participant (or system) to reflect on their practice to an extent which changes their attitudes or perceptions.
Despite the authors’ comments in the discussion section we think there are potential issues with maturation (in the context of clinicians getting acquainted with and used to the CPOE, despite their washout period – familiarity with the system is likely to reduce data input time) and regression to the mean in this study.
There’s a nice FOAM paper on before and after studies here.
The results of the study appear positive although the tables must be interpreted with caution as confidence intervals or standard deviations were not supplied. The authors acknowledge because of the large numbers of patients seen significance was easy to achieve. Ideally previous data on length of stay and time to be seen would have been used to determine what a clinically significant time difference would be and power calculations utilised to determine the number of patients needed to demonstrate this change. T-tests to examine the before and after effect are not an unreasonable choice but they would be unpaired in this situation (which was not stated). The distribution of the results may also not have been normal and this should have been acknowledged.
On closer inspection of the results although there does appear to be a big improvement in the time taken between the doctor reviewing and the decision to admit, the actual improvement in the length of stay in the department was only 6 mins in the post scribe group among admitted patients (and 20 minutes for those being discharged from the ED). The clinical relevance of this effect is difficult to determine in the context of being unsure of its true significance.
It’s not clear whether the introduction of scribes actually impacts the true departmental length of stay and it’s impossible to say whether this effect would be seen were scribes introduced without the preceding CPOE introduction. The authors have tried to argue that scribes have an independent positive effect on ED length of stay by comparing with another hospital where CPOE was introduced without scribes also being introduced; the other department showed an ongoing increase in length of stay which the authors attribute to CPOE – but again this is an example of correlation rather than causality. The attribution of this effect as being proof of scribe benefit is easily challenged as a number of other factors such as staff skill mix and education and training that are not controlled for.
Essentially the intervention (scribes following an implementation CFOE) was not clearly demonstrated at the outset to have a causal link to the outcome measures (various time measurements of patient flow) i.e the construct validity of the study is in question. We cannot assume scribes to be the direct cause of the measured results despite their statistical significance. This is not a total condemnation of scribes. They may have been the cause or they may facilitate the introduction of other time saving measures. The actual clinical significance of the 10 minute time saving is in question. With large patient numbers this could result in huge time savings overall but a full economic evaluation would be needed to demonstrate this benefit.
However given the studies face validity (most people would agree they should help) this is clearly an research idea which warrants further work.
Nat: I think, as a senior doctor in the ED, a scribe might yet be a useful addition to my practice; contemporaneous notes are preferable and while I purposefully don’t record history and examination at the bedside as it might disrupt flow of the consultation – including very important non-verbal communication – I know I am often interrupted by emergencies and questions from other staff, meaning I need to return to notes at a later time. A scribe recording during consultation would potentially increase the accuracy of what is recorded; however I have concerns about checking the accuracy of someone else’s transcription and I wonder how much saved time this would add back in.
I think another possibility is that we could consider using juniors to scribe for us (and us for them) – a sort of extended ACAT? This would still have the same ramifications for checking the accuracy of what’s been recorded but might at least provide some great learning experiences on both sides. Just a thought!
Damian: I would love the opportunity to have a scribe as reviews of patients other junior doctors have previously seen are usually very quick; as I am not having to record this information myself. However there are risks to this – I am very careful to check what has been written about what is essentially my management plan. I wonder if not scribing directly may also change my line of questioning and possibly introduce cognitive errors? I suspect these issues can easily be overcome with practice and I would welcome the opportunity to be part of studies in this area.
Wise words from across the pond – kindly contributed by Steve Carroll
“In regards to the accuracy- they are pretty good with everything except the history of presenting illness – that can be a mixed bag. Sometimes I cringe when I see the HPI they write down – either missing major points or writing down bad words like “lethargic” about a child when they look fine. It’s also interesting to see what they think is important vs. what I think is important. Its pretty important to read the scribe’s HPI before signing off on the chart and editing it as appropriate. I sign a statement at the bottom attesting that the scribe accurately transcribed my work.”
More wise words: Read this blog from 2007 by GruntDoc on scribes in the ED
Nat and Damian