A System for the System - System Errors in the ED

The Gist: In addition to cognitive biases, systems errors are ubiquitous in the emergency department (ED).  Free Open Access Medical education (FOAM) has inspired me to realize that one may mitigate some of these errors to improve patient care/outcomes by vigilance.

As a fan of metacognition, I attempt to mitigate my cognitive errors through simple practices. A few months into residency, I occasionally find myself frustrated by systems-based errors.  Finally, something to blame other than myself!  As that attitude is not terribly productive, I've adopted steps to attempt to overcome these errors, something I foresee as an evolving and expanding process.

Systems-based errors: reflect flaws or problems with process that are part and parcel of the health care delivery system.  These errors are often rooted in inefficiencies, issues with coordination of care, and communication [1].

Case #1:  A 54 y/o male presented to Janus General with fever to 38.8C, malaise, and weakness over the past 2 days. BP 126/82, HR 90. Patient had a PICC line to the right arm that appeared clean, status-post left hip wash out for septic arthritis 3 weeks prior.  I ordered labs and fluid, with plans for antibiotics.  Despite repeatedly checking for the lab results, they didn't appear.  Eventually, I called the lab - who reported receiving the specimens just minutes earlier as the hospital's tube delivery system had malfunctioned.  This resulted in another blood draw from the patient for a repeat lactate (which turned out to be 6), as the specimen was too old, a delay in more aggressive care, and a silly/guilty feeling doctor.  Studies demonstrate that delays in antibiotic administration impact mortality, so while the patient didn't appear to be in the sickest group of patients this still could have resulted in a bad outcome [2].  But, it's the system's fault, right?

Things I try to do to mitigate systems errors:

Communicate with nursing and support staff.  Oftentimes, they can help get things done more expeditiously or identify barriers to the proposed treatment plan.

Call the lab, radiology, pharmacy, etc.  Many steps exist between placing an order for a diagnostic evaluation or intervention and the completion of the order and an error can occur at any point. Furthermore, it's anecdotal, but I've been impressed with how face-to-face or verbal discussion of the "why" or need for urgency can expedite care.

Establish a consistent method to reassess patients/labs.  Time can fly in the ED and often our attention is divided by unexpected sick patients.  For example, we can handle a mostly stable GI bleeder, a septic patient, and a chest pain patient.  However, the minute one begins to crash or a code rolls in, our attention becomes divided and non-critical patients may be placed on the back-burner.  As a trainee, it's easy to think that a stable patient will remain stable but this isn't always the case.  Furthermore, interruptions are rampant in the ED and this forced shift in attention may lead to delays in reassessment [3].

• Some keep running list of things in their pocket/workstation that need follow up.  A unique solution offered by Dr. Jeremy Faust - use Siri [4].  "Siri, remind me to re-examine room 4 in 30 minutes." 

Beware of alarm fatigue.  Alarms constantly ring in the ED and pop up in the EMR - but on occasion, they actually mean something.  It's important to catch it when it does.

Approach sign out with caution.  This area of emergency medicine has garnered much attention as it may lead to a hotbed of cognitive errors and the nature of sign-out can vary within the institution. Most of the literature revolves around the inpatient experience, but I think that the vulnerability of this process translates into the ED [5]. Physicians and hospitals approach sign out differently, but there's a call for increased standardization [6].  Consider standardizing your own approach.

• Re-examine the patient, their vital signs, and crucial diagnostic/interventional endeavors.  
• Tip from Dr. Jeremy Faust - start sign-out saying something along the lines of: I'm intentionally going to be a bit annoying, don't take it personally.  Then, aggressively go through the case. Two heads are better than one. 

Reexamine information received from outside physicians/transferring facilities.  Information often gets left out or lost in the series of communications surrounding transfers in care or partial work ups - minimize this by utilizing the patient's data.

• Take a gander at a patient's EKG or diagnostics for yourself.  The "sinus bradycardia" for suspected accidental beta-blocker overdose may actually be a high degree AV block or a radiograph may be revealing.  

Establish a consistent method of follow up.  A myth exists that emergency physicians do not or should not follow up their patients.  As a result of the discontinuity of care, we will not typically see our mistakes unless we look for them. Check out the EM Res podcast on this topic.

• In the EMR, I keep a list, by month, of patients I see in the ED so I can easily check up on patients.  This takes 1 extra click per patient but saves time attempting to recollect the name and has created the expectation within myself that I will follow up on some of those patients.
• Consider a "bounceback" program.  This provides larger buy-in, but provides an invaluable educational opportunity.  At my institution, if a patient returns within 7 days, we receive a notification.  Oftentimes these are unpreventable (the daily drunk patient) or an indication that we provided good discharge instructions; however, more often than not there's a pearl for the future.

Use the EMR or family members to get to know the patient.  In the ED we don't know (most of) our patients, which can create fragmented care and an incomplete picture of our patient.  Valuable information can be found in the EMR (ex: the patient does, in fact, have baseline confusion and left sided weakness) or from family members who may be able to explicitly detail how the patient is different from baseline.

Case #2 (months later): A 51 y/o female presented to Janus General with cough, tachypnea, decreased oral intake.  Temperature 37.6C, BP 106/78, HR 136. Physical exam significant for tachypnea, rhonci in bilateral lung fields, and dry mucosa.  I discussed the plan with the nurse, emphasizing the patient's need for fluids and aggressive care.  The patient had no access and prior unsuccessful attempts, so the nurse quickly identified the need for ultrasound guidance.  Siri reminded me to check on the antibiotics and reassess the vital signs while suturing another patient.  The patient's vital signs, lactate, and clinical appearance normalized in the ED after several liters of fluid and the patient went on to do well.

Are there problems with attempted fixes for systems errors?

• Improvements will fade as time passes secondary to decreased awareness, other foci of improvement, and lessened enthusiasm [1]
• Fixes may produce opportunities for more errors

References:
1. Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what’s the goal? Acad. Med. 2002;77(10):981–92. Available at: http://www.ncbi.nlm.nih.gov/pubmed/12377672.2.
2.  Gaieski DF, Mikkelsen ME, Band RA,et al.  Impact of time to antibiotics on survival in patients with severe sepsis or septic shock in whom early goal-directed therapy was initiated in the emergency department. Crit Care Med. 2010 Apr;38(4):1045-53.
3. Elson, Ordell. Emergency Department Workplace Interruptions: Are Emergency Physicians “‘Interrupt-driven’” and “‘Multitasking’”? Academic Emergency Medicine 2000;7(11):1239–1243.
4.  Faust, JS. "The 'Sultan of Signout'" ACEP News. August 2013. p 12-13.
5. Arora V, Johnson J, Lovinger D et al. Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005 December; 14(6): 401–407.
6.  Dhingra KR, Elms A, Hobgood C. Reducing error in the emergency department: a call for standardization of the sign-out process. Ann Emerg Med. 2010 Dec;56(6):637-42.

The Modern Matthew Effect

The Gist:  In medicine and science, regardless of the medium - traditional or Free Open Access Medical education (FOAM)- the Matthew effect exists, potentially perpetuating knowledge and dogma that doesn't necessarily reflect intrinsic worth.  Question the medical dogma, respectfully and, while it's easy to copy and paste a citation for a quote a popular figure, consider critically evaluating the source of information or primary literature.  In words borrowed from TheSGEM podcast, "Be skeptical of everything you learn.." (to a healthy, not pathologic degree) - it's another arrow in the metacognition quiver.

Conversations on the perils of FOAM at the Social Media and Critical Care Conference (SMACC) spurned the following, something I think is worth reminding ourselves of from time to time:

Do not take what I say as gospel- @emcrit #smacc2013
— smacc (@smacc2013) March 11, 2013

While many of us exercise healthy skepticism we can still fall victim to a common phenomenon because, in the words of Daniel Kahneman in Thinking Fast and Slow, we have "almost unlimited ability to ignore our ignorance."  We may think we are fully aware of our biases, but they are worked into the fabric of life.  

The Matthew Effect (with regard to references):  essentially, the greater number of times a paper is cited, the more citations it will receive.   Coined by Merton in this paper, but initially researched by Harriet Zuckerman, it is borrowed from the Gospel according to St. Matthew: “For to all those who have, more will be given, and they will have an abundance; but from those who have nothing, even what they have will be taken away” (Matthew 25:29) (1).  

• The Matthew effect is partially a byproduct of quality.  A content expert likely becomes trusted and their work becomes highly regarded due to the merit of their prior work(s).  Thus, this is a sort of natural phenomenon in any field that has experts/masters in particular disciplines.  Zuckerman identified this in that Nobel Prize winners tended to generate/produce more awards compared with those who had shared equally in the project but were more junior researchers (1).

How does this manifest in medical literature?

High Impact Journals.  Impact factor (IF) - The IF is essentially the average number of times an article in the journal is cited within the previous two years.  A journal’s prestige is a function of the quality of the articles appearing in it. 

• What happens when the exact same article (title, author, etc) is published in two journals with disparate IFs?  This paper by Lariviere and Gingras (full text) took at look at this question and found that duplicate papers (4532 pairs of papers) in high impact journals obtain, on average, twice as many citations as their identical counterparts published in journals with lower IFs.  
• The intrinsic value of a paper is not the only reason for the citation of a specific paper; there is a Matthew effect attached to journals.  Thus, a paper published in a high impact journal has an added value over its intrinsic quality and will generate more citations.

High Impact Authors.  A high profile author's paper is likely to carry more weight or gain more recognition.

• Example:  In the Feb. 2013 edition of Emergency Medical Abstracts (subscription required), there's a little bit of banter about how this paper on incidence of contrast induced nephropathy was referred to ("the Kline paper").  Pulmonary embolism guru Dr. Jeff Kline is listed on the paper but the first author is actually Dr. AM Mitchell.  

This sounds like splitting hairs..  Perhaps a little, but not necessarily.

• Implicit in the concept of the Matthew effect is the notion is that a piece of research is more valuable or important because of its association with an individual rather than the contents, quality, or implications of the research.
• Without realizing it, we may become susceptible to a cognitive bias secondary to the "Halo Effect," which I first heard about in Thinking Fast and Slow by Daniel Kahneman.  For example, if an individual is widely regarded in the community for a podcast or publication, their institution may be looked upon more favorably.  
• Dr. Weingart's tweet at the beginning of this post demonstrates potential implications of the halo effect - a positive/powerful reputation may have undue influence over whether we see that information as important or valid.  If someone we respect says an article is a "must read" or "garbage" we have formed an impression of the article prior to actually reading it.  They may very well be spot on, but this is something to keep in mind.
• In an era of information overload, especially in medicine, we may deal with this cognitive load by perceiving a reputable person's recommendations as most/more important (known as positional cues).  This may skew our evidence base or perception of prevalence or importance of a medical problem.

Is FOAM impervious to this effect? No.

• FOAM has a form of Impact Factor.  This can be quantified in retweets, blog hits, or a spot  in a Life in the Fast Lane Weekly Review.  Again, this is not necessarily a negative thing and can be harnessed "for good," introducing innovative or important ideas quickly and diffusely across the globe.  
• Example:  

 

• The social connections and the platforms associated with FOAM are intricate (hospital and professional networks, friends/families (social media), affiliations with societies, etc).  As a result, the Matthew effect may be less like the "Nobel Prize" effect noted by Zuckerman as age, rank, and location may not carry as much weight and the sources are vast.
• Recently, Google announced that it would drop its RSS aggregating service, GoogleReader. This move immediately induced a Twitter frenzy regarding replacement services.  One focus of conversation on this topic from some members of the FOAM community was that Twitter has replaced the need for RSS.  This article discusses this notion, a debatable assertion that I don't personally find applicable to my use of RSS.  Should Twitter supplant RSS, individuals who use an RSS aggregator to review journals and/or medical blogs may have increased susceptibility to biases associated with a social media/recommendation system based system. 

So what do we do?

• Question productively and respectfully.  

@akutdok @thomas1973 @mdaware @smacc2013 @precordialthump @sandnsurf If listeners stop questioning every word I say, I will stop the podcast
— Scott Weingart (@emcrit) March 13, 2013

• Check sources.  For example, while putting together this post on elevated blood pressure in the ED, I came across a statistic in Tintinalli:  3.8% of headaches in the ED have serious intracranial pathology (Ch. 159).  Initially, I copied this statistic and reference because Tintinalli is one of the core EM texts.  FOAM has inspired me to check things out further, and upon evaluating the study I found it underwhelming to support the rate quoted.  This study was referenced by others as well, including the famous Perry et al article on subarachnoid hemorrhage and others. 
• Keep the Matthew effect in mind when evaluating articles, watching posts/ideas go "viral", or evaluating the validity of an assertion or claim.

Updated 3/18/13.

References:
1.  Zukerman H.  Scientific Elite:  Nobel Laureates in the United States. 2d ed. (New York:  Transaction Publishers, 1996). 

Metacognition For The Pragmatist

The Gist: Metacognition and cognitive errors in medicine are not merely fluffy, esoteric ideas. There are concrete steps one can take to mitigate these. Check out the Ten Commandments To Reduce Cognitive Error and Ten Commandments To Reduce Diagnostic Error by Dr. Leo Leonidas.

As a student, my interests lie in the "meat" (or tofu, as it were) of medicine. Yet, as the breadth (minute as it is) of my clinical experience grows, I've become increasingly interested and aware of cognitive errors, especially my own. Why? Well, I think it will make me a better future physician. Cognitive bias, previously discussed here, is common in medicine and emergency medicine (EM). Metacognition, discussed in this post, can mitigate cognitive error by evaluating one's thinking. Although this seems esoteric, especially to the trainee, there are some concrete ways to go work through this process.

Commandments to reduce cognitive and diagnostic errors

Adapted from Dr. Leo Leonidas (1,2)

Thou shalt think of serious and treatable conditions and act on them without delay. 

• This is the crux of EM.  We must be facile with some of the most intense minutes of every specialty - retrobulbar hematoma, subarachnoid hemorrhage, asthma, cardiac arrest, or...anything! 
• Know the killers and, as I learned from this EMBasic episode, always place a few in the differential.  

Thou shalt mentally rehearse common and serious conditions that you expect to see in your specialty.

• Simulation plays an expanding role in medical education, especially in EM where physicians must be skilled in life and limb-saving procedures that are rarely used. Preparation is crucial to making logical decisions under pressure and time constraints. Check out this paper by Dr. Cliff Reid (full text).
• There is a cornucopia of FOAM procedure videos and tips, available regardless of one's proximity to a fancy simulation center . Examples include: Life In The Fast Lane repository, Procedurettes, EMCrit's Thoracotomy Episode, and so many more out there (use EMgoogle).

Thou shalt not rely on your memory when making critical decisions.

• Check the actual vitals, labs, etc when making crucial decisions or talking with consultants. EDs are insanely busy and it's difficult to keep things straight, no matter how sharp the mind.  
• I overheard a conversation in which a PCP received a call to admit a syncope patient and, when the PCP asked about the patient's blood pressure, the ED doctor stated it was "about 150's systolic." Upon review of the patient's chart, her systolic BP was 90mmHg throughout her ED stay. This misinformation didn't result in harm, but it certainly pointed to potential cause of syncope and a lack of attention to detail.

Thou shalt consider other possibilities even though you are sure of your first diagnosis.  

• Always think of a differential, even if it's a "slam dunk" case. This can help avoid anchoring bias as well as diagnostic momentum bias.
• Continue the debate and questioning even though the data is “in,” this will help mitigate confirmation bias and premature closure (see ALiEM post). 

Thou shalt maintain a high index of suspicion or uncommon presentations of the common.

• Know the probability and epidemiology of the diseases in the your differential but, in order to lessen availability bias, also recognize that zebras exist.  Include one in each differential, it's a great thought exercise and the diagnosis doesn't have to be worked up fully solely because it's on the differential.   
• Patient's often don't present in the "classic" or "textbook" way.  According to Dr. Amal Mattu, "Classic, in medicine, means 15% of the time."  Sounds about right.

Thou shalt be wary of your hunches and intuitions; use Evidence Based Medicine (EBM).  

• Gestalt plays a huge role in EM.  As discussed in this post, however, this is something that evolves over the course of an entire career and is something that trainees like myself are fiercely trying to cultivate. 
• Clinical experience and anecdotes play a formative role in the development of a physician yet come at a price.  We are more likely to remember the 1 in 1,000,000 exception than the mundane 999,999.  These make great stories and provide learning points, but we probably shouldn't become reactionary to the outlier.    
• EBM is not infallible and there are instances in which there is insufficient, applicable evidence to guide decision making.  However, in many cases there is a body of global experience with a particular disease or intervention.  It would be foolish to not capitalize on the best available information to provide the best patient care.

Thou shalt take time to decide.

• At the end of a night shift with a packed waiting room, it may be worth pausing for a moment to think about the 55 year old with abdominal pain. Reconstruct the story without segments spent dealing with the combative drunk, the code, and the literal pain in the neck. There are situations in which action is paramount; however, in most cases there's time to think. 

 Thou shalt engage in accountability procedures and follow up for decisions. 

• Follow patients - it's a myth that emergency physicians can not or should not engage in follow up.
• Keep a protected list of patients for each shift - using logs, stickers, or lists in the medical record software. 

Thou shalt request a test only if it will change your plan or help in predicting the outcome.

• Danger exists in too much information as it can lead one astray.  Think about how a test is relevant and will change management (ALiEM post). The pieces must fit, otherwise an extraneous bit of information could lead to search satisfying bias. 
• Testing and treatment has associated risks, for a review, see the St. Emyln's posts on this topic.
• Diagnosis is largely in the history and physical exam, make sure that the data fits with the story and the patient's clinical exam and vice versa.  As trainees, it's crucial to hone the history and physical skills, particularly for short ED encounters. 

Thou shalt use available databases.

• Use an iPad, tablet, or iPhone to readily access information. Consult applications, calculators, and pharmacopeias to double check medication dosages and adverse effects.
• Colleagues often have niches - in ECGs, stroke, toxicology, etc. Pause to run cases by others for a fresh or alternative perspective.  
• Lucky enough to have an ED pharmacist? Consult with them to reduce error (Patanwala et al, Cohen et al).
• Electronic medical records systems allow one to view recent hospital visits and lab work, helpful in piecing together a coherent story.
• Prescription monitoring databases can be a valuable resource. ACEP recommends using them (Level C).

Thou shalt reflect on how you think and decide.

• Recognize your own beliefs, biases, prejudices, and thinking style.  This is one of the toughest commandments, as it requires introspection and reflection.  For example, I know that I often have the tendency to mentally apply EBM and literature before fully assessing a patient's nuances.  Knowing this, however, I benefit from a mental pause checkpoint.  
• Check out talks from FreeEmergencyTalks.net on this by Dr. Croskerry.
• Heuristics, often cultivated by gestalt, are frequently used in medicine.  Check out these articles by Wegwarth et al and Marewski to see some ways in some ways in which heuristics can be created and tweaked to as a form of decision support.  

References:

1.Leonardo Leonidas.  "Ten Commandments to Reduce Diagnostic Errors."  Given to the Class of 2001 Tufts University School of Medicine From Dr. Leonardo Leonidas, Bangor, Maine, 20 May 2001  

2.Leonardo Leonidas. "Ten Commandments to Reduce Cognitive Errors."  2003.

Three Dirty Words: Do You Remember...

The Gist:  There's more uncertainty in medicine than most of us would like, especially in emergency medicine.  We are often required to act with limited information and data and sometimes we make mistakes.  Good medical students and physicians reflect on these mistakes in order to improve.  EMCrit's Scott Weingart and Cliff Reid of Resus.Me demonstrate an excellent example of the benefit of reflecting upon cases with unfavorable outcomes in this podcast.

The case:  "I have a sick one for you, do you remember that woman y'all sent home on Tuesday..."  A sinking feeling crept over me as I glanced at the name on the "board," rushing to the room.

Two days prior: A 63 year old female with a history of diabetes came into the ED in the midst of influenza season with a two days of myalgias, subjective fever, dry cough, congestion, nausea, and vomiting.

• VS: T 99.3F, P 90, R 18, BP 124/78
• PE: The patient looked somewhat sick.  Nothing focal on physical exam. 
• Significant Diagnostics: Chemistry and blood counts within normal limits, Glucose 155. CXR - normal
• I sat down with the patient to discuss options for disposition - observation versus discharge.  After our talk, the patient stated she felt significantly better with the ondansetron and fluids and wanted to go home with an anti-emetic.  She and her significant other promised to return for worsening of symptoms. 

This visit: The same patient presented with the chief complaint, "I feel like I'm dying," after two syncopal episodes at home.

• VS T 102F, P 160's, R 22 (actually closer to 30 upon visualization), BP 90/64
• PE: Gen-Patient pale and diaphoretic, unable to stand under own power.  CV-Tachycardic. Resp-rhonchi in left lower lung. 
• Significant Diagnostics: ECG-atrial fibrillation with rapid ventricular response, BMP - Cr 1.88, Glucose 160, WBC 13.1

This case filled me with a sense of unease.  I had been rather confident on the patient's initial presentation that she would improve at home.  Nasty bugs were going around the community and this patient visibly improved in the ED.  Instead, she developed sepsis and I felt apologetic.  I thought back to my thinking two days prior and searched for cognitive errors.  Premature closure and anchoring likely played a role in the case - all diagnostics were negative and the patient didn't even meet SIRS criteria.  I stopped searching based on lab reassurance, despite my gestalt (limited as it may be).  Furthermore, she promised me she would return if she felt worse - I felt betrayed.   

I was reminded of a grand rounds lecture delivered by University of Florida chief resident, Dr. Brandon Allen, entitled "Do You Remember?" in which he shared the following video (very much worth watching!).

Take Away:

• Engage in shared decision making.  Empower patients to understand their medical situation, risks and benefits of interventions, and allow them make some decisions for themselves.  Understand that patients don't always do what we want them to do. Wyer et al state it succinctly in this article: 
• "We should be careful about complacency when it comes to reversion to the paternalistic medical model of decisionmaking. An abundance of evidence attests to the fact that physicians’ decisions on behalf of their patients may be entirely contrary to the decisions the same physicians would make on their own behalf"
• Inform the patient, in simple language, of their condition, any evidence that may aid them in making a decision (ex using PECARN: fewer than 1 in 2000 kids that have an injury like this would have something that needs be treated picked up on a scan).  Decision aids or instruments, many of which are found on MDcalc may be helpful (just phrase them in a way the patient can understand).
• Document this process. 
• Check for cognitive errors when working patients up, especially if the clinical picture doesn't correlate with the diagnostics.  
• Learn something from each case and carry it forward in a productive fashion.  We're not perfect and predicting a patient's future is dangerous; however, discussions with peers and mentors about cases may add another perspective and enable one to hone clinical performance and thinking.  Alternatively, it may affirm your decision making and action.  
• Again, Drs. Reid and Weingart have an excellent example in this podcast.
• Beware of a reactionary response. 
• An attending once told me that I should never send a patient with a DVT home without a CT scan of the chest to look for a PE.  The physician had a bad experience once and had adopted this practice which is really not the standard and potentially detrimental to many patients.  Clinical experiences can be surprising and terrifying.  I certainly have very limited experience, but I think that the cases that weigh on our minds have the potential to alter our clinical practice meaningfully which can be good or deleterious.  Beware of overcompensation.

Case Conclusion:

• BP improved in the ED with 3L of IVF.  Patient spontaneously converted into normal sinus rhythm.  Patient treated for sepsis secondary to pneumonia, had a small troponin bump during the hospital course, but had improved and was discharged after a few days in the hospital.
• I was very thankful that I had documented the shared decision making discussion and return precautions well.

I've Been Framed! My #twitterstatslesson

The Gist:  The way in which one frames a drug or intervention may severely alter the way in which one perceives the most objective information.  Patients and physicians are similarly affected by framing bias (a cognitive bias, perhaps amenable to metacognition).  The use of absolute risks and terms rather than relative terms helps mitigate this bias.  Number needed to treat (NNT) and number needed to harm (NNH) are probably better ways of evaluating interventions and testing, even though these have limitations as well.  Bookmark theNNT.com.

I have no statistics background except a biostatistics class that's part of my MPH curriculum, and attempts to keep up with SMART EM and Dr. David Newman on EM:RAP's mini JC section.  I realized we're often not formally taught to interpret and integrate bits of statistics, although the USMLE Step 2 is now beginning to attempt testing this aspect.  Thus, this stuff is important both to trainees and clinicians.  As I've begun to learn to sort through evidence and literature, I've found that FOAM delivers.  One evening, I found following conversation regarding absolute versus relative risk among some world-renowned physicians dominated my Twitter feed (only parts are listed below):

@precordialthump @karimbrohi @broomedocstrials judged by ARR, trials applied by RRR
— Scott Weingart (@emcrit) September 11, 2012

Discrepancy worse for lower baseline risk: 1% -> 0.9%. NNT = 1000.Only RR is unchanged across populations. #twitterstatslesson :)
— Karim Brohi (@karimbrohi) September 11, 2012

X in your population lowers to 9%. ARR=1%, NNT=100. But baseline risk in my population 20%. X lowers to 18%. ARR=2%, NNT=50. @broomeddocs
— Karim Brohi (@karimbrohi) September 11, 2012

NNT a flawed measure of utility:@broomedocs Say intervention X reduces risk of death by 10%. Your population baseline risk of death = 10%.
— Karim Brohi (@karimbrohi) September 11, 2012

RT @precordialthump: @karimbrohi If baseline event rate different, RCT lacks external validity & doesn't apply to your population.[No...
— Karim Brohi (@karimbrohi) September 11, 2012

@precordialthump I take your point though. The problem is that RRR is amenable to spin by those on the dark side...
— Chris Nickson (@precordialthump) September 11, 2012

So, I began to think about my own shortcomings in understanding these statistical dilemmas and figures.  I found an older podcast by the St. Emlyn's group on iTunes under "Statistics 2" to be helpful in sorting out absolute and relative risks, as mentioned in the Twitter conversation.  Then, amazingly, I stumbled upon a paper on framing the following day via Emergency Medicine Abstracts.  Apparently, I had good reason to worry...

We frame things constantly, providing a context for information and a means of understanding the information.  Like most things, this has both positive and negative implications.  For example, when one is trying to admit a patient to a particular service, one might phrase things rather differently than when one tells a patient why you plan on discharging them.  Oftentimes we use paternalism or the legitimate best interest of the patient to justify these 'frames.'

However, pharmaceutical companies do this as well through well-crafted consumer (and provider) advertising.  For example, this advertisement states that Pradaxa (dabigatran) reduced stroke risk 35% more than warfarin.  Thus, individuals may believe that the drug will reduce their stroke risk by a similar amount.  However, the 35% reduction in stroke risk only existed for a subset of patients.  Patients and populations are heterogeneous so these numbers don't necessarily apply any given patient.  As a pretty well educated bunch, health care providers should be really good at detecting framing differences and less susceptible to this bias, right?  Wrong!

The Paper:  Perneger T,  Agoritsas T.  Doctors and Patients’ Susceptibility to Framing Bias: A Randomized Trial.  J Gen Intern Med 26(12):1411–7

• Sent questionnaires to physicians and patients recently discharged from the hospital asking the respondent to judge the efficacy of a new drug based on the information provided
• Information on the new and old drug was identical but presented in only one of the following ways:  
• Absolute mortality:  with the new drug, 4% died by the end of the study versus 6% for those who received the old drug
• Absolute survival: with the new drug, 96% lived versus 94% who lived with the old drug.   Least favorable reviews of the drug by both groups (Physicians 51.8%, patients 51.7%, p=0.98 between groups)
• Relative mortality reduction:  of those who received the old drugthose that got the new drug had their mortality reduced by 1/3.  Both patients and physicians presented with relative mortality reduction perceived the drug most favorably (Physicians 93.8%, patients 89.3% p=0.11 between groups).  The odds ratio (OR) of a respondent perceiving the drug more favorably with the information presented in this format compared with other formats was 4.40 (95% CI 3.05 – 6.34, p<0.001)
• All three presentations of risk.  with the new drug, 96% lived, 4% died versus 94% lived, 6% died  with the old drug, so those that got the new drug had their mortality reduced by 1/3
• Physicians were also tested on the NNT (50) and the relative survival extension.
• Doctors responded to the presentations with no statistical difference from the patients. OR of Doctor (versus patient) = 1.06 (95% CI 0.87 – 1.29, p = 0.55)
• The notion that information presented in relative terms rather than absolute terms is more favorably received is not new (1).

There is so much to learn from the ever-relevant FOAM world.

References:
1.  Malenka DJ, Baron JA, Johansen S, Wahrenberger JW, Ross JM. The framing effect of relative and absolute risk. Journal of general internal medicine. 1993;8(10):543-8.

Anchors Aweigh! Cognitive Bias - Where IS This Ship Headed?

The Gist:  We all succumb to cognitive errors from time to time.  Identifying these errors in our medical decision making though exercising metacognition may improve patient safety but it may also allow us to be better clinicians.  NB:  This is not a comprehensive review of all types of cognitive biases, which one can find here in "List 1."  Rather, this is a synopsis of some of the commonest cognitive biases, which I've learned from first-hand.

Anchoring Bias - when a first impression or one piece of evidence exerts undue influence in the diagnostic process.

• Case:  Listen to a great new blogger/podcaster, Dr. Bob Stuntz, present a case on Anchoring Bias in which he gives an excellent example of the patient who comes in complaining of "I have a kidney stone."
• Solution: wait until information about the case is complete before forming an impression or selling yourself on a diagnosis.  Note:  clearly in some critical situations one must act before information is complete.   

Triage Cueing - bias initiated by the patient's initial triage level, assuming that a patient can't be sicker than their triage level.  

• Case:  A patient was placed in the "minor care" area for a "sore throat."  The patient's PMH included hypertension and the history elucidated that the patient's complaint was more of a dry throat (drinking copious amounts of water) coupled with a yeast infection that wouldn't go away.  She also generally felt terrible and weak.  The attending was initially wary of the idea of a fingerstick glucose level but acquiesced after discussing that polydipsia and intractable yeast infections are harbingers of uncontrolled diabetes.  The result = 587 mg/dL.  Chemistry demonstrated that the patient was in mild DKA, with newly diagnosed diabetes.
• Solution:  Recognize that patients have the potential to be sick regardless of initial triage level.  Triage cueing may also set up another cognitive bias, Diagnostic Momentum, where a patient's workup is based solely on one diagnosis or label (hand-offs at sign out serve as notorious examples).  

Premature closure - when one accepts a diagnosis before verification of the diagnosis. 

• Case: A 48 year old male is transferred from an outside hospital for CHF.  He presented with acute dyspnea accompanied by some pinkish phlegm.  He denies chest pain, pressure, leg swelling, travel, or cough. He was slightly hypertensive, sating 93% on 3L (non-smoker). Troponin was negative, ecg showed potentially new LBBB. Patient was given furosemide and a diagnosis of CHF. Upon arrival to our ED he had a BNP of 52 and was found to have a PE upon further work up since his story of CHF didn't seem to fit with our independent evaluation.
• Solution:  Look at the evidence that both supports and refutes the diagnosis and, if lacking, obtain appropriate evidence.

Confirmation bias - look for evidence to confirm the hypothesis rather than searching for evidence to refute.

• Case:  A 40-something year old patient presents to the hospital with nausea, vomiting, and epigastric pain. The patient has a history of pancreatitis and helicobacter pylori in addition to the all-American trio:  diabetes, hypertension, and hypercholesterolemia.  When the lipase came back over 300 and the bilirubin and transaminases were also fairly elevated, the patient was observed until his pain and nausea were controlled and he passed a PO challenge.  The patient bounced back within 24 hours in heart failure from a sizable MI.
• Solution:  Look at incoming data objectively before selecting out certain pieces.

Search Satisfying - the tendency to cease looking for other findings/disease processes once something is found.

• Case:  A patient presents s/p motorcycle crash with right arm pain.  Exam demonstrates an avulsion injury over the patient's right elbow.  X-rays were negative, the wound was repaired, and the patient was readied for discharge.  Upon an additional exam, the patient had tenderness in the anatomic snuff box and we found the following:

Scaphoid Fracture!

• Solution:  Ask yourself - Is there anything else going on here? ATLS has helped decrease the tendency for search satisfying bias in trauma situations through algorithms.

Availability and Non-availability - the greater prevalence (in the ED,literature,community,news,etc), the more likely we will think of and pursue the diagnosis (and the converse also holds true).

• Case:  A 4 year old male presents with several days of fever.  He also had cracking of his lips, a maculopapular rash, and cervical lymphadenopathy.  The patient was diagnosed with Kawasaki Disease after a day.  The next several patients that came in with more than a few days of fever got complete workups/evaluation for Kawasaki. 
• Solution:  Ask yourself - Is the diagnosis based on the case and data or based on something you're comfortable with?  Uncommon things happen too - keep these in mind as well (I'm biased as I love a good Zebra!).

Ascertainment Bias - one sees what one expects to see (self-fulfilling prophecy)

• Case: A 34 year old male, well known to the ED and EMS for frequent overdoses, presents with AMS and respiratory depression after his friends watched him shoot up heroin.  The patient was brought in with the diagnosis of overdose and his initial workup and treatment revolved around that one diagnosis.  Eventually the patient required intubation and upon further exam was noted to have unequal pupils.  Although this "frequent flyer" did have some level of overdose going on, the label as a "frequent flyer" and "overdose" initially obscured the fact that he was actively herniating due to a large subdural hematoma.
• Solution:  Realize that patients who abuse drugs or have "red flag" diagnoses or allergies get sick, too. Look at each patient with fresh eyes.

As a student, I force myself to generate 5 items on a differential before I present.  Sometimes this is ridiculous and a clear stretch but I use that tiny bit of time to think about why I'm thinking the way I am and potentially identify some of my cognitive bias.  As information comes in from further evaluation, diagnostics, etc I look at the data and integrate it into my leading diagnosis, as well as my differential.  It really doesn't take additional time if I force myself to do it every time.

We will never be able to eliminate all errors, especially since systems errors play an enormous role in medical errors (and contribute to cognitive error), but perhaps we can train ourselves to reduce those that are in our control.  Life in the Fast Lane provides some succinct case-based insight into cognitive errors with these case scenarios.  There's some argument that recognition of these biases may not translate into meaningful patient outcomes, but I still think it's good form to think with intention and act when necessary.  Also, if you haven't yet, check out Dr. Patrick Croskerry's  free lectures,  on the subject (most span all of clinical decision making and errors).

References: 
Croskerry, P. The importance of Cognitive Errors in Diagnosis and Strategies to Minimize Them.  Academic Medicine.  August 2003, Vol 78, Issue 8. p775-780.
Croskerry, P.  Achieving Quality in Clinical Decision Making:  Cognitive Strategies and Detection of Bias.  Academic Emergency Medicine.  Nov 2002, Vol 9. No 11.
Jepson, Zak.  University of Massachusetts.  Medical Student Lecture. August 15, 2012.

Thinking About Thinking

The Gist:  Clinical reasoning in the EM setting is different than other arenas as one must often make life and death decisions (and actions) with limited information and even less time.  Our patients are often undifferentiated and nearly any ailment exists as a possibility.  Furthermore, EPs are constantly juggling multiple patients and responsibilities.  As a result, cognitive errors are common and play a role in clinical decision making (although this also applies to all fields of medicine).  Thinking about thinking or, "metacognition," may help reduce these cognitive errors to allow Emergency Physicians (EPs) to improve diagnostic and treatment decision making.   

The case that made me care about cognitive errors:  A 40-something year old patient presented to the hospital with nausea, vomiting, and epigastric pain stating "this feels a lot like my pancreatitis."  The patient had a history of pancreatitis and helicobacter pylori in addition to the all-American trio:  diabetes, hypertension, and hypercholesterolemia.  The labs, work up, and patient all seemed to proclaim pancreatitis.  Unfortunately, the patient bounced back within a day in heart failure, status-post massive MI.  Ever since, I've been exceptionally wary of diabetics with nausea and vomiting and garnered a fascination with the thinking of an EP.

In medical school, we're overtly taught clinical reasoning through data mining in the form of history, physical exam, and diagnostic testing.  I suppose we're subtly and indirectly taught to think about the way we integrate and assimilate this data into a coherent picture of the patient and an accurate diagnosis.  We subconsciously use heuristics, cognitive short cuts, to inform clinical gestalt.  My formal medical education, however, did not include any discussion of cognitive bias or meta-cognition until a resident at a program I rotated at gave a brief presentation on metacognition.

• Apparently, we can reduce errors if we step back for a moment and think:  What doesn't fit? What have I failed to consider (perhaps a zebra? or a different horse?)?  What biases may be present?  What is leading me to think that this patient has X?

This is so important that I wanted to disseminate this information to help others who strive to be excellent thinkers and swell clinicians.

Basic systems of thinking.  Between which we have the capability to toggle, which we should probably exercise more often. Check out this lecture.

• "System 1" or "Fast":  This is the instinct, intuitive, adaptive, associative, quick thinking.  We use this system when we say, "this patient just looks sick" or "I have this gut feeling the patient has ______."  Caution: medical students and trainees should really not rely on this type of thinking although it is often key in EM (think clinical gestalt)
• "System 2" or "Analytic":  This is a slower process of thinking which is more deliberate and analytic.

Common EM thinking patterns:

• Hypothetico-Deductive Model:  One of the most common cognitive pathways used in medicine - useful in non-critical situations, as algorithms such as ATLS and ACLS tend to dominate in the more critical circumstances.  
• Main steps:  Generation, Evaluation, Refinement, Verification.  Errors can be present in any step.  
• Error in Generation - failure to consider a potential diagnosis (influenced by disease prevalence, atypical presentations, etc).  
• Error in Evaluation - problems in gathering data, interpreting and assimilating the data, and putting the data in the proper context.
• Error in Verification - failure to ensure that the final diagnosis fits with the clinical picture and established data/workup.
• Pattern Recognition Model:  This dominates when an experienced clinician uses clinical gestalt to inform the diagnosis rather than generating a complete differential diagnosis (and thereby prone to error for failing to consider alternative diagnoses).
• Rule Out Worse Case Scenario Model:  Some clinicians employ this most of the time and others tend to apply it when particularly high-risk diagnoses are on the table.  This method of reasoning is expensive and exposes the patient to excess harms through extensive investigations.

Why may the ED be a breeding ground for cognitive error?

• High levels of diagnostic uncertainty
• High decision density and cognitive load
• High levels of activity
• Inexperience of some providers (and students)
• Interruptions and distractions
• Shift changes
• Many of these, integrated, produce fatigue
• These errors occur in every facet of medicine but in EM, there is a certain expectation that EPs not miss the badness (or anything).

The papers to read (note:  Dr. Patrick Croskerry is a world-reknowned expert in this arena.  If you're looking for even more to read, check out his plethora of articles.  Also, he has several free talks on freeemergencytalks.net, which can be converted to podcasts within iTunes)  

Life in the Fast Lane provides some succinct case-based insight into cognitive errors with To Err is Human 1 and To Err is Human 2

Achieving quality in clinical decision making: cognitive strategies and detection of bias - Croskerry

This article, also by Croskerry has a rather complete list of cognitive errors in the tables embedded in the free text article.

Clinical Decision Making in Emergency Medicine (free full text) 

Clinical Decision Making:  An Emergency Medicine Perspective (free full text)

Other References:
Jepson, Zak. University of Massachusetts.  Medical Student Lecture. August 15, 2012.