A System for the System - System Errors in the ED

The Gist: In addition to cognitive biases, systems errors are ubiquitous in the emergency department (ED).  Free Open Access Medical education (FOAM) has inspired me to realize that one may mitigate some of these errors to improve patient care/outcomes by vigilance.

As a fan of metacognition, I attempt to mitigate my cognitive errors through simple practices. A few months into residency, I occasionally find myself frustrated by systems-based errors.  Finally, something to blame other than myself!  As that attitude is not terribly productive, I've adopted steps to attempt to overcome these errors, something I foresee as an evolving and expanding process.

Systems-based errors: reflect flaws or problems with process that are part and parcel of the health care delivery system.  These errors are often rooted in inefficiencies, issues with coordination of care, and communication [1].

Case #1:  A 54 y/o male presented to Janus General with fever to 38.8C, malaise, and weakness over the past 2 days. BP 126/82, HR 90. Patient had a PICC line to the right arm that appeared clean, status-post left hip wash out for septic arthritis 3 weeks prior.  I ordered labs and fluid, with plans for antibiotics.  Despite repeatedly checking for the lab results, they didn't appear.  Eventually, I called the lab - who reported receiving the specimens just minutes earlier as the hospital's tube delivery system had malfunctioned.  This resulted in another blood draw from the patient for a repeat lactate (which turned out to be 6), as the specimen was too old, a delay in more aggressive care, and a silly/guilty feeling doctor.  Studies demonstrate that delays in antibiotic administration impact mortality, so while the patient didn't appear to be in the sickest group of patients this still could have resulted in a bad outcome [2].  But, it's the system's fault, right?

Things I try to do to mitigate systems errors:

Communicate with nursing and support staff.  Oftentimes, they can help get things done more expeditiously or identify barriers to the proposed treatment plan.

Call the lab, radiology, pharmacy, etc.  Many steps exist between placing an order for a diagnostic evaluation or intervention and the completion of the order and an error can occur at any point. Furthermore, it's anecdotal, but I've been impressed with how face-to-face or verbal discussion of the "why" or need for urgency can expedite care.

Establish a consistent method to reassess patients/labs.  Time can fly in the ED and often our attention is divided by unexpected sick patients.  For example, we can handle a mostly stable GI bleeder, a septic patient, and a chest pain patient.  However, the minute one begins to crash or a code rolls in, our attention becomes divided and non-critical patients may be placed on the back-burner.  As a trainee, it's easy to think that a stable patient will remain stable but this isn't always the case.  Furthermore, interruptions are rampant in the ED and this forced shift in attention may lead to delays in reassessment [3].

• Some keep running list of things in their pocket/workstation that need follow up.  A unique solution offered by Dr. Jeremy Faust - use Siri [4].  "Siri, remind me to re-examine room 4 in 30 minutes." 

Beware of alarm fatigue.  Alarms constantly ring in the ED and pop up in the EMR - but on occasion, they actually mean something.  It's important to catch it when it does.

Approach sign out with caution.  This area of emergency medicine has garnered much attention as it may lead to a hotbed of cognitive errors and the nature of sign-out can vary within the institution. Most of the literature revolves around the inpatient experience, but I think that the vulnerability of this process translates into the ED [5]. Physicians and hospitals approach sign out differently, but there's a call for increased standardization [6].  Consider standardizing your own approach.

• Re-examine the patient, their vital signs, and crucial diagnostic/interventional endeavors.  
• Tip from Dr. Jeremy Faust - start sign-out saying something along the lines of: I'm intentionally going to be a bit annoying, don't take it personally.  Then, aggressively go through the case. Two heads are better than one. 

Reexamine information received from outside physicians/transferring facilities.  Information often gets left out or lost in the series of communications surrounding transfers in care or partial work ups - minimize this by utilizing the patient's data.

• Take a gander at a patient's EKG or diagnostics for yourself.  The "sinus bradycardia" for suspected accidental beta-blocker overdose may actually be a high degree AV block or a radiograph may be revealing.  

Establish a consistent method of follow up.  A myth exists that emergency physicians do not or should not follow up their patients.  As a result of the discontinuity of care, we will not typically see our mistakes unless we look for them. Check out the EM Res podcast on this topic.

• In the EMR, I keep a list, by month, of patients I see in the ED so I can easily check up on patients.  This takes 1 extra click per patient but saves time attempting to recollect the name and has created the expectation within myself that I will follow up on some of those patients.
• Consider a "bounceback" program.  This provides larger buy-in, but provides an invaluable educational opportunity.  At my institution, if a patient returns within 7 days, we receive a notification.  Oftentimes these are unpreventable (the daily drunk patient) or an indication that we provided good discharge instructions; however, more often than not there's a pearl for the future.

Use the EMR or family members to get to know the patient.  In the ED we don't know (most of) our patients, which can create fragmented care and an incomplete picture of our patient.  Valuable information can be found in the EMR (ex: the patient does, in fact, have baseline confusion and left sided weakness) or from family members who may be able to explicitly detail how the patient is different from baseline.

Case #2 (months later): A 51 y/o female presented to Janus General with cough, tachypnea, decreased oral intake.  Temperature 37.6C, BP 106/78, HR 136. Physical exam significant for tachypnea, rhonci in bilateral lung fields, and dry mucosa.  I discussed the plan with the nurse, emphasizing the patient's need for fluids and aggressive care.  The patient had no access and prior unsuccessful attempts, so the nurse quickly identified the need for ultrasound guidance.  Siri reminded me to check on the antibiotics and reassess the vital signs while suturing another patient.  The patient's vital signs, lactate, and clinical appearance normalized in the ED after several liters of fluid and the patient went on to do well.

Are there problems with attempted fixes for systems errors?

• Improvements will fade as time passes secondary to decreased awareness, other foci of improvement, and lessened enthusiasm [1]
• Fixes may produce opportunities for more errors

References:
1. Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what’s the goal? Acad. Med. 2002;77(10):981–92. Available at: http://www.ncbi.nlm.nih.gov/pubmed/12377672.2.
2.  Gaieski DF, Mikkelsen ME, Band RA,et al.  Impact of time to antibiotics on survival in patients with severe sepsis or septic shock in whom early goal-directed therapy was initiated in the emergency department. Crit Care Med. 2010 Apr;38(4):1045-53.
3. Elson, Ordell. Emergency Department Workplace Interruptions: Are Emergency Physicians “‘Interrupt-driven’” and “‘Multitasking’”? Academic Emergency Medicine 2000;7(11):1239–1243.
4.  Faust, JS. "The 'Sultan of Signout'" ACEP News. August 2013. p 12-13.
5. Arora V, Johnson J, Lovinger D et al. Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005 December; 14(6): 401–407.
6.  Dhingra KR, Elms A, Hobgood C. Reducing error in the emergency department: a call for standardization of the sign-out process. Ann Emerg Med. 2010 Dec;56(6):637-42.

The Modern Matthew Effect

The Gist:  In medicine and science, regardless of the medium - traditional or Free Open Access Medical education (FOAM)- the Matthew effect exists, potentially perpetuating knowledge and dogma that doesn't necessarily reflect intrinsic worth.  Question the medical dogma, respectfully and, while it's easy to copy and paste a citation for a quote a popular figure, consider critically evaluating the source of information or primary literature.  In words borrowed from TheSGEM podcast, "Be skeptical of everything you learn.." (to a healthy, not pathologic degree) - it's another arrow in the metacognition quiver.

Conversations on the perils of FOAM at the Social Media and Critical Care Conference (SMACC) spurned the following, something I think is worth reminding ourselves of from time to time:

Do not take what I say as gospel- @emcrit #smacc2013
— smacc (@smacc2013) March 11, 2013

While many of us exercise healthy skepticism we can still fall victim to a common phenomenon because, in the words of Daniel Kahneman in Thinking Fast and Slow, we have "almost unlimited ability to ignore our ignorance."  We may think we are fully aware of our biases, but they are worked into the fabric of life.  

The Matthew Effect (with regard to references):  essentially, the greater number of times a paper is cited, the more citations it will receive.   Coined by Merton in this paper, but initially researched by Harriet Zuckerman, it is borrowed from the Gospel according to St. Matthew: “For to all those who have, more will be given, and they will have an abundance; but from those who have nothing, even what they have will be taken away” (Matthew 25:29) (1).  

• The Matthew effect is partially a byproduct of quality.  A content expert likely becomes trusted and their work becomes highly regarded due to the merit of their prior work(s).  Thus, this is a sort of natural phenomenon in any field that has experts/masters in particular disciplines.  Zuckerman identified this in that Nobel Prize winners tended to generate/produce more awards compared with those who had shared equally in the project but were more junior researchers (1).

How does this manifest in medical literature?

High Impact Journals.  Impact factor (IF) - The IF is essentially the average number of times an article in the journal is cited within the previous two years.  A journal’s prestige is a function of the quality of the articles appearing in it. 

• What happens when the exact same article (title, author, etc) is published in two journals with disparate IFs?  This paper by Lariviere and Gingras (full text) took at look at this question and found that duplicate papers (4532 pairs of papers) in high impact journals obtain, on average, twice as many citations as their identical counterparts published in journals with lower IFs.  
• The intrinsic value of a paper is not the only reason for the citation of a specific paper; there is a Matthew effect attached to journals.  Thus, a paper published in a high impact journal has an added value over its intrinsic quality and will generate more citations.

High Impact Authors.  A high profile author's paper is likely to carry more weight or gain more recognition.

• Example:  In the Feb. 2013 edition of Emergency Medical Abstracts (subscription required), there's a little bit of banter about how this paper on incidence of contrast induced nephropathy was referred to ("the Kline paper").  Pulmonary embolism guru Dr. Jeff Kline is listed on the paper but the first author is actually Dr. AM Mitchell.  

This sounds like splitting hairs..  Perhaps a little, but not necessarily.

• Implicit in the concept of the Matthew effect is the notion is that a piece of research is more valuable or important because of its association with an individual rather than the contents, quality, or implications of the research.
• Without realizing it, we may become susceptible to a cognitive bias secondary to the "Halo Effect," which I first heard about in Thinking Fast and Slow by Daniel Kahneman.  For example, if an individual is widely regarded in the community for a podcast or publication, their institution may be looked upon more favorably.  
• Dr. Weingart's tweet at the beginning of this post demonstrates potential implications of the halo effect - a positive/powerful reputation may have undue influence over whether we see that information as important or valid.  If someone we respect says an article is a "must read" or "garbage" we have formed an impression of the article prior to actually reading it.  They may very well be spot on, but this is something to keep in mind.
• In an era of information overload, especially in medicine, we may deal with this cognitive load by perceiving a reputable person's recommendations as most/more important (known as positional cues).  This may skew our evidence base or perception of prevalence or importance of a medical problem.

Is FOAM impervious to this effect? No.

• FOAM has a form of Impact Factor.  This can be quantified in retweets, blog hits, or a spot  in a Life in the Fast Lane Weekly Review.  Again, this is not necessarily a negative thing and can be harnessed "for good," introducing innovative or important ideas quickly and diffusely across the globe.  
• Example:  

 

• The social connections and the platforms associated with FOAM are intricate (hospital and professional networks, friends/families (social media), affiliations with societies, etc).  As a result, the Matthew effect may be less like the "Nobel Prize" effect noted by Zuckerman as age, rank, and location may not carry as much weight and the sources are vast.
• Recently, Google announced that it would drop its RSS aggregating service, GoogleReader. This move immediately induced a Twitter frenzy regarding replacement services.  One focus of conversation on this topic from some members of the FOAM community was that Twitter has replaced the need for RSS.  This article discusses this notion, a debatable assertion that I don't personally find applicable to my use of RSS.  Should Twitter supplant RSS, individuals who use an RSS aggregator to review journals and/or medical blogs may have increased susceptibility to biases associated with a social media/recommendation system based system. 

So what do we do?

• Question productively and respectfully.  

@akutdok @thomas1973 @mdaware @smacc2013 @precordialthump @sandnsurf If listeners stop questioning every word I say, I will stop the podcast
— Scott Weingart (@emcrit) March 13, 2013

• Check sources.  For example, while putting together this post on elevated blood pressure in the ED, I came across a statistic in Tintinalli:  3.8% of headaches in the ED have serious intracranial pathology (Ch. 159).  Initially, I copied this statistic and reference because Tintinalli is one of the core EM texts.  FOAM has inspired me to check things out further, and upon evaluating the study I found it underwhelming to support the rate quoted.  This study was referenced by others as well, including the famous Perry et al article on subarachnoid hemorrhage and others. 
• Keep the Matthew effect in mind when evaluating articles, watching posts/ideas go "viral", or evaluating the validity of an assertion or claim.

Updated 3/18/13.

References:
1.  Zukerman H.  Scientific Elite:  Nobel Laureates in the United States. 2d ed. (New York:  Transaction Publishers, 1996). 

Metacognition For The Pragmatist

The Gist: Metacognition and cognitive errors in medicine are not merely fluffy, esoteric ideas. There are concrete steps one can take to mitigate these. Check out the Ten Commandments To Reduce Cognitive Error and Ten Commandments To Reduce Diagnostic Error by Dr. Leo Leonidas.

As a student, my interests lie in the "meat" (or tofu, as it were) of medicine. Yet, as the breadth (minute as it is) of my clinical experience grows, I've become increasingly interested and aware of cognitive errors, especially my own. Why? Well, I think it will make me a better future physician. Cognitive bias, previously discussed here, is common in medicine and emergency medicine (EM). Metacognition, discussed in this post, can mitigate cognitive error by evaluating one's thinking. Although this seems esoteric, especially to the trainee, there are some concrete ways to go work through this process.

Commandments to reduce cognitive and diagnostic errors

Adapted from Dr. Leo Leonidas (1,2)

Thou shalt think of serious and treatable conditions and act on them without delay. 

• This is the crux of EM.  We must be facile with some of the most intense minutes of every specialty - retrobulbar hematoma, subarachnoid hemorrhage, asthma, cardiac arrest, or...anything! 
• Know the killers and, as I learned from this EMBasic episode, always place a few in the differential.  

Thou shalt mentally rehearse common and serious conditions that you expect to see in your specialty.

• Simulation plays an expanding role in medical education, especially in EM where physicians must be skilled in life and limb-saving procedures that are rarely used. Preparation is crucial to making logical decisions under pressure and time constraints. Check out this paper by Dr. Cliff Reid (full text).
• There is a cornucopia of FOAM procedure videos and tips, available regardless of one's proximity to a fancy simulation center . Examples include: Life In The Fast Lane repository, Procedurettes, EMCrit's Thoracotomy Episode, and so many more out there (use EMgoogle).

Thou shalt not rely on your memory when making critical decisions.

• Check the actual vitals, labs, etc when making crucial decisions or talking with consultants. EDs are insanely busy and it's difficult to keep things straight, no matter how sharp the mind.  
• I overheard a conversation in which a PCP received a call to admit a syncope patient and, when the PCP asked about the patient's blood pressure, the ED doctor stated it was "about 150's systolic." Upon review of the patient's chart, her systolic BP was 90mmHg throughout her ED stay. This misinformation didn't result in harm, but it certainly pointed to potential cause of syncope and a lack of attention to detail.

Thou shalt consider other possibilities even though you are sure of your first diagnosis.  

• Always think of a differential, even if it's a "slam dunk" case. This can help avoid anchoring bias as well as diagnostic momentum bias.
• Continue the debate and questioning even though the data is “in,” this will help mitigate confirmation bias and premature closure (see ALiEM post). 

Thou shalt maintain a high index of suspicion or uncommon presentations of the common.

• Know the probability and epidemiology of the diseases in the your differential but, in order to lessen availability bias, also recognize that zebras exist.  Include one in each differential, it's a great thought exercise and the diagnosis doesn't have to be worked up fully solely because it's on the differential.   
• Patient's often don't present in the "classic" or "textbook" way.  According to Dr. Amal Mattu, "Classic, in medicine, means 15% of the time."  Sounds about right.

Thou shalt be wary of your hunches and intuitions; use Evidence Based Medicine (EBM).  

• Gestalt plays a huge role in EM.  As discussed in this post, however, this is something that evolves over the course of an entire career and is something that trainees like myself are fiercely trying to cultivate. 
• Clinical experience and anecdotes play a formative role in the development of a physician yet come at a price.  We are more likely to remember the 1 in 1,000,000 exception than the mundane 999,999.  These make great stories and provide learning points, but we probably shouldn't become reactionary to the outlier.    
• EBM is not infallible and there are instances in which there is insufficient, applicable evidence to guide decision making.  However, in many cases there is a body of global experience with a particular disease or intervention.  It would be foolish to not capitalize on the best available information to provide the best patient care.

Thou shalt take time to decide.

• At the end of a night shift with a packed waiting room, it may be worth pausing for a moment to think about the 55 year old with abdominal pain. Reconstruct the story without segments spent dealing with the combative drunk, the code, and the literal pain in the neck. There are situations in which action is paramount; however, in most cases there's time to think. 

 Thou shalt engage in accountability procedures and follow up for decisions. 

• Follow patients - it's a myth that emergency physicians can not or should not engage in follow up.
• Keep a protected list of patients for each shift - using logs, stickers, or lists in the medical record software. 

Thou shalt request a test only if it will change your plan or help in predicting the outcome.

• Danger exists in too much information as it can lead one astray.  Think about how a test is relevant and will change management (ALiEM post). The pieces must fit, otherwise an extraneous bit of information could lead to search satisfying bias. 
• Testing and treatment has associated risks, for a review, see the St. Emyln's posts on this topic.
• Diagnosis is largely in the history and physical exam, make sure that the data fits with the story and the patient's clinical exam and vice versa.  As trainees, it's crucial to hone the history and physical skills, particularly for short ED encounters. 

Thou shalt use available databases.

• Use an iPad, tablet, or iPhone to readily access information. Consult applications, calculators, and pharmacopeias to double check medication dosages and adverse effects.
• Colleagues often have niches - in ECGs, stroke, toxicology, etc. Pause to run cases by others for a fresh or alternative perspective.  
• Lucky enough to have an ED pharmacist? Consult with them to reduce error (Patanwala et al, Cohen et al).
• Electronic medical records systems allow one to view recent hospital visits and lab work, helpful in piecing together a coherent story.
• Prescription monitoring databases can be a valuable resource. ACEP recommends using them (Level C).

Thou shalt reflect on how you think and decide.

• Recognize your own beliefs, biases, prejudices, and thinking style.  This is one of the toughest commandments, as it requires introspection and reflection.  For example, I know that I often have the tendency to mentally apply EBM and literature before fully assessing a patient's nuances.  Knowing this, however, I benefit from a mental pause checkpoint.  
• Check out talks from FreeEmergencyTalks.net on this by Dr. Croskerry.
• Heuristics, often cultivated by gestalt, are frequently used in medicine.  Check out these articles by Wegwarth et al and Marewski to see some ways in some ways in which heuristics can be created and tweaked to as a form of decision support.  

References:

1.Leonardo Leonidas.  "Ten Commandments to Reduce Diagnostic Errors."  Given to the Class of 2001 Tufts University School of Medicine From Dr. Leonardo Leonidas, Bangor, Maine, 20 May 2001  

2.Leonardo Leonidas. "Ten Commandments to Reduce Cognitive Errors."  2003.