Misrepresented: EBM

The Gist: Evidence based medicine (EBM) is misunderstood; it's not a randomized control trial (RCT) or "the literature." Rather, EBM is the intersection of the best available evidence, clinical expertise, and patient values [1-2]. Avoid BARF (Brainless Application of Research Findings), with tips from Emergency Medicine Cases. 

We have a cultural problem.  Clinicians are increasingly called upon to practice EBM.  Yet, the term EBM does not sit well on the palate of many physicians.  Conversations involving a mention of EBM have resulted in some of the following refrains...

"See, my patients are different..." 

"We'll never get an RCT on that..." 

"The culture is different here, I don't want to get sued." 

"Patients don't understand, but they do hold the power with satisfaction scores." 

"It's cookbook medicine."

With these words and reactionary body language, the dialogue quickly shuts down - by both parties.  First, this is a shame.  We should learn from one another but there seems to be a "hard stop" between many who champion EBM and those who find EBM off-putting. Second, this is a misunderstanding.  EBM is not an RCT.  In fact, EBM is not the best statistical methods or the rationing of care. EBM is not nihilism.  

EBM is the intersection of the best available evidence, clinical expertise, and patient values:

"the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research" [1].

Why, then, the misunderstanding? 

Here are some thoughts...

Misrepresentation. EBM is often used to refer to literature or studies, rather than to the application of research and evidence to particular patients and situations, using one's clinical experience (example and discussion: "EBM is Crap").  As a result, EBM may be misunderstood as a cost-cutting venture or a cookbook for medicine [3]. I have been complicit in perpetuating this misrepresentation of EBM.   As a novice physician-in-training with limited clinical experience, I draw predominantly upon the literature base.  I have unknowingly quoted the literature, thereby proudly proclaiming my practice of EBM, while unconsciously dismissing the other components of EBM.  

• A remedy:  Remind ourselves and others that the evidence is part of the trifecta of EBM, along with the patient's values and clinical expertise.  We can be clear in what we mean by EBM and refrain from referring to a body of literature as EBM. 

Zeal. A religiosity exists amongst many champions of EBM, or people who believe they are championing EBM.  We tout our pyramids of evidence and may scoff at a lack of evidence or rigorous trials.  This may be off-putting as not all fields are amenable to RCTs and patient populations vary.  Moreover, there's a human tendency to form a reactionary attitude when someone exerts a strong identity [4].  Hence, EBM zeal may engender an anti-EBM attitude and cause people to be wary of solid practice changing evidence.

• A remedy:  While championing good research and employing the best available evidence, we can balance our enthusiasm with important caveats and understand the importance for tailored approaches for patients.  Gentle education about EBM rather than diatribes may aid individuals in understanding the values of EBM beyond evidence.

Fear.  People like to be right.  We may reflexively become defensive when we are (possibly) wrong. EBM or "literature" can be used in an antagonizing way and, subconsciously, a way to exert a feeling of superiority.  "You haven't read that study?"

• A remedy: Understand that unlearning in medicine is difficult.  Aggressive assertions may push people further away.  Think of it as a Kubler-Ross like grief cycle, as explained in this post.  This may help us become more cognitively flexible, understand the reticence of others, and perhaps make our points more effectively.  

@LWestafer @sandnsurf Change as a grief reaction - Kubler Ross curve model of change/grief pic.twitter.com/4w2I0BD74o
— Simon Carley (@EMManchester) March 6, 2014

Confusion.  Historically, researchers, clinicians, physicians in training, and allied health professionals have limited understanding of fundamental statistics [5,6].  As such, we may not understand what we're reading or how it applies to our patient population.  We may have difficulty understanding why something we believed was proper at one time is no longer the case.  Often, this is because the research was, in fact, initially wrong or misleading [7]. 

• A remedy: Read.  This podcast proffers tips on getting started; however, even the most seemingly rigorous papers published in high impact journals are subject to bias (publication bias and otherwise), which can be difficult to parse through.  For example, the oseltamivir (tamiflu) recommendations from Cochrane changed after they were allotted access to data, demonstrating the profound impact of publication bias [Jefferson et al].  More on this here.

Time. The number of journal articles needed to read (NNR) to obtain valid and relevant information is typically cited as 20-200, an insurmountable task [8].  The process of trolling through the literature is time consuming and may be overwhelming.  Frustration can turn into apathy, confusion, and mistrust.

• A remedy: Use Free Open Access Medical education (FOAM) resources to complement regular reading.  Sites such as The Skeptics Guide to Emergency Medicine, the Life in the Fastlane Research & Reviews, and HEFT EMcast exist as ways to keep up to date on important literature. 

There are legitimate issues with EBM.  Evidence is often subject to the biases of industry and legislative bodies.  Guidelines or recommendations billed as "EBM" may be hijacked by individuals with conflicts of interest or other agendas. Further, the grading of evidence isn't always objective or consistent, as seen by the grading of evidence for thromboylitics in acute ischemic stroke listed in the ACEP clinical policy.   In addition, guidelines harness EBM and disseminate the body of evidence to practitioners.  For example, the 2008 AHA/ACC guidelines are based largely on low levels of evidence and expert opinion,  many of whom have financial conflicts of interest.  Only 11% of the recommendations were based on high quality evidence [9].  

So, while EBM has imperfections in concept, representation, and implementation, the model incorporates the primary things we, as providers, care about - the evidence, the patient, and clinical experience.  Let's understand what EBM means and apply the term and principles appropriately.

References:
1. Sackett DL, Rosenberg WM, Gray JAM, et al. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71–72. 

2. Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ 2014;348:g3725

3. Straus SE, McAlister FA. Evidence-based medicine: a commentary on common criticisms. CMAJ. 2000;163(7):837–41. 
4.  Maalouf A.  In the Name of Identity: Violence and the Need to Belong. New York: Penguin Books, 2000.
5.  Windish D, Huot S, Green M. Medicine residents’ understanding of the biostatistics and results in the medical literature. Jama. 2007;298(9). 
6.  Mavros MN, Alexiou VG, Vardakas KZ, Falagas ME. Understanding of statistical terms routinely used in meta-analyses: an international survey among researchers. PLoS One. 2013;8(1):e47229. 
7.Ioannidis JP a. How many contemporary medical practices are worse than doing nothing or doing less? Mayo Clin Proc. 2013;88(8):779–81.
8. McKibbon KA, Wilczynski NL, Haynes RB. What do evidence-based secondary journals tell us about the publication of clinically important articles in primary care journals? BMC Med. 2004;2:33. 
9.  Tricoci P1, Allen JM, Kramer JM, et al.  Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA. 2009 Feb 25;301(8):831-41.

Tools in the ED - Clinical Decision Instrument Basics

The Gist:  Clinical decision instruments (CDIs) are all the rage in Emergency Medicine, especially for trainees still developing gestalt; however, these tools often require proper understanding and finesse for correct utilization.  FOAM (Free Open Access Medical education) sources such as Dr. Radecki's posts on NEXUS, PECARN abdominal trauma, and the Ottawa SAH Rule as well as Dr. Spiegel's post on the Ottawa SAH Rule have helped hone the way I think about and utilize decision aids. This editorial in Annals of Emergency Medicine (podcast here) is a concise, excellent synopsis of questions to ask when evaluating CDIs.

CDIs are tools, not rules.  These are typically derived through statistical methods in discrete populations. While the tools then undergo validation, these aids are artificial creations to assist providers in decision making and are not infallible. In the words of Mel Herbert regarding CDIs in Oct 2013's EMRAP : "you don't need to slavishly follow them."

What does the decision tool add to the clinical context?

• Is the CDI better than clinician gestalt in pursuing work-up or treatment of a disease process?
• In the editorial, Green explains this well using the PECARN blunt abdominal tool.  Physician gestalt in ordering CTs for clinically significant abdominal injury: Sensitivity 99%, Specificity 56%.  The PECARN tool offered a sensitivity of 97% and specificity of 42% [1].  Thus, no real added benefit from the tool.
• Numerous studies investigating pulmonary embolism (PE) have determined that strict application of tools perform no better than physician gestalt within the study populations [5]. 
• Is the tool usable? Washington University's EM Journal club covered an example of issues with usability ACS CDIs.

Clinical decision aids shouldn't replace gestalt.  

• CDIs often appear to distill and codify components that comprise gestalt, which may be an enticing way to substitute clinical judgment.  As a medical student, I used these tools to aid in developing gestalt.  However, this could potentially be a bad habit in the making (see next point).
• Many CDIs utilize gestalt as an entry criteria or as part of the actual aid.  
• For example, in Tintinalli, Dr. Jeff Kline recommends applying PERC when the gestalt is there's a <15% chance that the patient has a PE, as this was the way in which the CDI was validated [3,4]. Thus, applying PERC to the wrong population may be deleterious.
• Dr. Seth Trueger posted his PE diagnostic algorithm following an international Twitter debate on pathways and pre-test probability.  The gist of both of these is that a provider should consider the patient's clinical situation and downstream consequences or work up that may result. 

All gestalts are not created equal.
— Chris Cole (@DocOnSkis) September 12, 2013

What clinical question was the decision tool designed to answer?

• Tools such as the Wells and Geneva scores were designed and validated as risk stratification tools, not rule-out or rule-in criteria.  
• In this podcast, Dr. Scott Weingart offered some points of clarification on using CDIs to determine which patients to work up for PE.  He also harps on the point above - these scores are not designed to make the decision to work up/not work up a PE.
• Measured outcome. Does the outcome reflect the clinical parameter you care about?
• The Canadian Head CT aid seeks to identify head injuries that required neurosurgical intervention, not those that would resolve with no alteration in management.  One must decide whether this is the outcome both provider and patient care about.

Is the patient part of the applicable population?

• For example, it's important to note that the Canadian Head CT aid only applies to patients with: GCS 13-15, witnessed LOC, amnesia to the head injury event, or confusion and the authors excluded patients with "minor head injuries" that didn't have one of the aforementioned factors (see this post for more specific discussion of this example) [6].  Broadly applying the tool to patients who don't meet inclusion criteria or were excluded in the studied populations may lead to inappropriate stratification or intervention.
• The performance of decision aids may depend on the prevalence of disease in the population. For example, PERC and Wells perform less well in high prevalence populations [5].
• Various other factors such as developing vs developed setting, resources, etc may also alter the applicability of the decision aid in one's population. The more similar a paper's population is to your own, the more usable the decision aid.  For example, some decision aids may rely on a neurological exam performed by a neurologist versus an emergency physician.

Has the decision aid been validated? If so, how?

• Once a group derives a CDI, the tool must be validated to test it's rigor.  Dr. Newman gives a great explanation on this podcast (20 min mark). There are a few ways in which this typically happens:
• Internal or external - validated in the same institution(s) or in other populations
• Prospective or retrospective - data collected prospectively or retrospectively
• Statistical or clinical - tool validated through statistical means or in "real life." The latter demonstrates usability and utility. 
• Example: one can continue a data-collection study of parameters of the derivation portion of the study or one can use the tool in a population going forward to determine clinical utility. This is an example of the latter using the Canadian Head CT aid.

Know whether a decision tool is a one-way or two-way instrument.  Misapplication of these tools may lead to excessive resource utilization and undermine the specificity of the aids.  [1]

• One Way Decision Tools - Useful if all criteria are met.
• Example: If someone is negative by PERC when utilized appropriately, it can indicate that the patient's risk of PE is below the test threshold. Conversely, one cannot say that if a patient is not PERC negative, then they necessitate work up for PE. 
• Two Way Decision Tools - Can help a clinician decide both when to pursue an action and when not to pursue the action.  The "Ottawa ankle rule" is an example. [1]

Note: I'm a mere novice with minimal statistics or EBM training, so these thoughts are more to be a reminder for myself than an in-depth analysis.

References:

1.  Green SM.  When do clinical decision rules improve patient care?  Ann Emerg Med. 2013 Aug;62(2):132-5. doi: 10.1016/j.annemergmed.2013.02.006. Epub 2013 Mar 30.

3.  Kline, J.  Thromboembolism.  Tintinalli's Emergency Medicine.  7th ed.  p 434.

4.  Kline, J. Prospective multicenter evaluation of the pulmonary embolism rule-out criteria. Thromb Haemost. 2008 May;6(5):772-80. doi: 10.1111/j.1538-7836.2008.02944.x. Epub 2008 Mar 3.
5. Lucassen W, Geersing GJ, Erkens PM, et al. Clinical decision rules for excluding pulmonary embolism: a meta-analysis. Ann Intern Med. 2011 Oct 4;155(7):448-60. doi: 10.7326/0003-4819-155-7-201110040-00007.
6.  Stiell IG, Lesiuk H, Wells GA, et al.  The Canadian CT Head Rule Study for patients with minor head injury: rationale, objectives, and methodology for phase I (derivation). Ann Emerg Med. 2001 Aug;38(2):160-9.

(Don't) Mind The Gap

The Gist:  Despite our best intentions and with regard to the combined literature, research, and clinical experience, we practice dated medicine.  Information disseminates and is adopted by individuals instantaneously in many other aspects of life.  Public discourse resulting from this information sharing, applied in medicine (Knowledge Translation) has the potential to improve health care...and FOAM (Free Open Access Meducation) is a promising means to tackle this problem. As a trainee, I think it's important to build solid habits and integrate this way of thinking/tackling medical learning early on.

What's the problem?  In an epidemiology class for my Master of Public Health, I was shocked when my professor declared that it often took a decade, if not more, before evidence was practiced by clinicians.  But we're so educated!

• Gaps between knowledge/information/experience and clinical practice (1).  Medical and health care research is booming.  Things change quickly and it's difficult to stay up to date, especially if your specialty involves every organ system and environment imaginable.  What is this research worth if we can't integrate it in clinical practice to benefit our present patients?
• Physicians practice despite guidelines or evidence favoring a different outcome (2).  We have collections and evaluations of the best evidence from the Cochrane Library and BestBets

Knowledge translation (KT): “Knowledge translation is defined as the exchange, synthesis and ethically sound application of knowledge—within a complex system of interactions among researchers and users—to accelerate the capture of the benefits of research… through improved health, more effective services and products, and a strengthened health care system.”  (1)

KT tutorials:

• Freeemergencytalks.net has a couple of great talks that summarize knowledge translation in EM by Eddy Lang and Gary Gaddis
• Indiana University's program has a pretty extensive website full of FOAM goodness, including worksheets to evaluate and interpret studies for those of us who are amateurs.  
• KT Clearinghouse
• The Canadian Institutes of Health has an amazing pool of KT modules and resources

Goals of KT:

• Changing behavior
• Changing health outcomes
• Achieving both of the above outcomes in an ethical, non-coercive way.

How can FOAM improve KT?

• Can precede national guidelines.

Told you that two years ago! thennt.com/antivirals-for… MT @medscapeem: New guidelines on Bell's palsy from the AAN bit.ly/TwHY3z
— Graham Walker (@grahamwalker) November 8, 2012

• Easily accessible from nearly anywhere. 
• TheNNT has a host of evidence-based reviews on frequently encountered topics.  These are frequently revised.
• MDCalc allows one to easily calculate a score like PESI or CHADS2-VASC score in seconds.
• Many apps for smart phones and tablets have these built in calculators as well (Medscape under "calculators" and other ones that are paid apps).
• Asynchronous updates in literature and research at no charge to the consumer.  Continuing Medical Education (CME) can be expensive and time consuming but blogs, podcasts, and various RSS feeds allow one to access information when, where, and in the quantity one desires (I prefer mine at the gym, in the car, or during anything that involves waiting).
• SMARTEM is a podcast that takes deep dives through the literature to assess and interpret the evidence behind various clinical practices.
• The Skeptics Guide to Emergency Medicine (The SGEM) has a free podcast in which they address specific articles or guidelines.
• Bridging academic and community settings, "flattening the world" (to borrow Thomas Friedman's analogy for technology and knowledge/goods dissemination).  Knowledge and experience varies across the regions (ex: see the Prehospital and Retrieval Medicine multinational podcast on procedural sedation).
• A group of FOAM masters recently began a "Rural Masterclass," to extend and involve rural physicians in a relevant and current continuing learning endeavor.
• Dialogue.   Individuals in the medicine field frequently debate and discuss guidelines, criteria, and "standards of care" on Twitter, blogs, and podcasts.  This frequently engenders further examination of preconceptions, understandings, and barriers to implementation of interventions/therapies.
• At times it seems there's peer pressure to conform to how others are doing things, even if it's not necessarily the most appropriate intervention (example: prescribing antibiotics for acute sinusitis in otherwise healthy patients or failing to prescribe steroids in acute asthma exacerbations).  Discourse between professionals can function as a support system and allow individuals to troubleshoot and benefit from each others experiences in implementation.

Is there a downside to KT?

• Medicine is not a unilateral encounter but a dialogue and decision making process with a patient. Some individuals worry that emphasis on evidence and guidelines have the potential to overshadow the individual nature of clinical encounters (4).  Properly understood, however, KT is not a trendy guise for CMS guidelines or core measures (which are designed to be coercive).  The essence of KT is to produce better health outcomes based on all available evidence.
• Implementation is not homogenous for each system or practice.  KT involves the attitudes, knowledge base, and infrastructure of complex systems.  In fact, there's an entire journal dedicated to implementing evidence (Implementation Science).  It's pretty daunting work, but again, the FOAM community may allow for a shared learning experience in success and hardships of implementation of current knowledge.
• Creating and disseminating guidelines do not necessarily result influence practice at the bedside.  Effective KT is the product of integration into a clinician's cognitive approach of each situation, which is not an insignificant endeavor (5). 

References

1.  Davis D et atl. The case for knowledge translation: shortening the journey from evidence to effect BMJ 2003; 327 
2.  Lang E, Wyer P, Haynes R. Knowledge translation: closing the evidence-to-practice gap Ann Emerg Med. 2007 Mar;49(3):355-63. Epub 2006 Nov 3 (full text)
3.   Straus S, Tetroe J, Graham I.  Defining Knowledge Translation. CMAJ August 4, 2009 vol. 181 no. 3-4 Full Text

4.  Greenhalgh T,  Wieringa S.   Is it time to drop the ‘knowledge translation’ metaphor? A critical literature review  J R Soc Med December 2011 vol. 104 no. 12501-509

5.  Green L and Siefert C.  Translation of Research Into Practice: Why We Can’t “Just Do It” J Am Board Fam Med November-December 2005 vol. 18 no. 6541-545

Not Always Evident - A Self-Guided Approach to Evidence

The Gist:  Evidenced based medicine (EBM) plays a critical role in emergency medicine and it's crucial to be able to interpret and apply data, especially if you don't want to practice medicine 10 years in the past.  Rather than glancing barely beyond the abstract, check out some quick tools that demonstrate key skills to help one piece together the influx of new data in a meaningful, interesting, and quick way.  Use Emergency Medicine Literature of Note, The Skeptics Guide to Emergency Medicine (The SGEM), Twitter, and EM Nerd as the springboard - it takes minimal time and proffers great returns.  It's really not as dry or time consuming as you may think.

In medical school, we briefly covered bare bones epidemiology in preparation for board exams, but had nothing to equip us to critique articles and data.  That was fine with me until we spent two classes in one of my Master of Public Health courses on article/study analysis.  I was amazed at what the data, buried in complex inclusion criteria and analyses, actually concluded.  You can find a study to prove absolutely anything.  Perhaps many medical students glean these skills in college; however, as a Middle Eastern History major, my research was on Orientalism and gender.  I acquired an affinity for Turkish coffee, but little understanding of things like study design, confounders, absolute risk reduction, and subgroup analysis.  I'm still climbing the learning curve in this arena but seriously, if I can learn and be intrigued by this stuff, then absolutely anyone can.

Not convinced that analyzing and interpreting medical literature is important?

• It's now tested on the USMLE Step 2 in practical application format.
• Medicine changes constantly.  As a result, I've encountered many attendings who use students as a means of staying current.  Apply the things you learn from the literature (ex: on Family Medicine, I championed antibiotic stewardship and the new cervical cancer screening guidelines.  Saving the world, one less pap smear and one less antibiotic prescription at a time).
• Most medical students keep the touted evidence based medicine database, UpToDate, at our fingertips.  The evidence supporting these articles is not always as robust as it seems. For example, UpToDate's overview of hyperkalemia management cites the reduction of potassium through the use of sodium polysterene resins like Kayexelate.  The actual evidence is disguised in the parenthetical reference to the journal article from 1961, sans abstract and featuring seven subjects.  Underpowered? Methodologically flawed?

Fundamentals. Here's a basic tutorial.

• What's the primary outcome measurement?  Was it met?
• Was there a good control group?
• Is the study sample reflective of the population? Who was left out of the study?  How was sampling conducted?
• Are the methods clear?  Lots of loss to follow up?
• Are there confounders?
• Do the results apply to other people/populations?
• Did what was measured actually mean anything to the patient?

"But this takes forever."  It can be overwhelming to think about critiquing articles, taking care of patients, cheering on our sports team, studying for boards, and pursuing a personal life.  Fortunately, there are individuals skilled in this endeavor that one can simultaneously learn from and emulate while staying current with medical literature. Note: It's neat to read the article in question before checking these opinions/interpretations and see how the analysis matches up.

• The Skeptics Guide to Emergency Medicine (The SGEM) - A podcast aimed at encouraging evidence based practice among physicians, reducing the time from which knowledge is translated into practice.  In under 20 minutes, they review a major article/subject using the PICO format.  
• EM Lit of Note - concise, insightful, readable synopsis of popular literature several times each week.  These critiques are easy to read and, although colored by his own opinion, provide insight into important pitfalls and clinical implications of these studies. 
• theNNT.com assesses common treatments and diagnostic tests by the Number Needed to Treat (NNT) to prevent a bad outcome and the number of patients harmed in that same process.  Begin by checking the NNT for various standards in medical practice when you're so inclined, take a gander at the section where they describe how they calculated the numbers.  Like most things in medicine, these numbers aren't universally agreed upon, but it's a neat, helpful tool.
• Best Bets - An evidence based medicine project that looks at the evidence behind very specific questions. 
• SMARTEM with Dr. David Newman and Dr. Ashley Shreves.  This duo takes deep dives into the literature, from which one can absorb an incredible amount about how to deconstruct studies.  These are dense and worth more than one listen but they're well done and interesting.
• Check out Twitter.  In this forum there's amazing international dialogue regarding medical literature.  Insightful, fiery, and humorous.  In fact, one of my favorite conversations began with the following tweet after the NEJM published a study on azithromycin and CV disease "just f***in great http://t.co/cdyiYVzf3."
• Dr. Richard Lehman's Journal Review.  Dr. Lehman quickly highlights a few articles from the world's leading medical journals, providing his opinion on these studies and/or the implications that lie therein.
• R&R in the Fast Lane can be used to briefly see what what other physicians think is important, practice changing, weird, or ridiculous in the literature