Tools in the ED - Clinical Decision Instrument Basics

The Gist:  Clinical decision instruments (CDIs) are all the rage in Emergency Medicine, especially for trainees still developing gestalt; however, these tools often require proper understanding and finesse for correct utilization.  FOAM (Free Open Access Medical education) sources such as Dr. Radecki's posts on NEXUS, PECARN abdominal trauma, and the Ottawa SAH Rule as well as Dr. Spiegel's post on the Ottawa SAH Rule have helped hone the way I think about and utilize decision aids. This editorial in Annals of Emergency Medicine (podcast here) is a concise, excellent synopsis of questions to ask when evaluating CDIs.

CDIs are tools, not rules.  These are typically derived through statistical methods in discrete populations. While the tools then undergo validation, these aids are artificial creations to assist providers in decision making and are not infallible. In the words of Mel Herbert regarding CDIs in Oct 2013's EMRAP : "you don't need to slavishly follow them."

What does the decision tool add to the clinical context?

• Is the CDI better than clinician gestalt in pursuing work-up or treatment of a disease process?
• In the editorial, Green explains this well using the PECARN blunt abdominal tool.  Physician gestalt in ordering CTs for clinically significant abdominal injury: Sensitivity 99%, Specificity 56%.  The PECARN tool offered a sensitivity of 97% and specificity of 42% [1].  Thus, no real added benefit from the tool.
• Numerous studies investigating pulmonary embolism (PE) have determined that strict application of tools perform no better than physician gestalt within the study populations [5]. 
• Is the tool usable? Washington University's EM Journal club covered an example of issues with usability ACS CDIs.

Clinical decision aids shouldn't replace gestalt.  

• CDIs often appear to distill and codify components that comprise gestalt, which may be an enticing way to substitute clinical judgment.  As a medical student, I used these tools to aid in developing gestalt.  However, this could potentially be a bad habit in the making (see next point).
• Many CDIs utilize gestalt as an entry criteria or as part of the actual aid.  
• For example, in Tintinalli, Dr. Jeff Kline recommends applying PERC when the gestalt is there's a <15% chance that the patient has a PE, as this was the way in which the CDI was validated [3,4]. Thus, applying PERC to the wrong population may be deleterious.
• Dr. Seth Trueger posted his PE diagnostic algorithm following an international Twitter debate on pathways and pre-test probability.  The gist of both of these is that a provider should consider the patient's clinical situation and downstream consequences or work up that may result. 

All gestalts are not created equal.
— Chris Cole (@DocOnSkis) September 12, 2013

What clinical question was the decision tool designed to answer?

• Tools such as the Wells and Geneva scores were designed and validated as risk stratification tools, not rule-out or rule-in criteria.  
• In this podcast, Dr. Scott Weingart offered some points of clarification on using CDIs to determine which patients to work up for PE.  He also harps on the point above - these scores are not designed to make the decision to work up/not work up a PE.
• Measured outcome. Does the outcome reflect the clinical parameter you care about?
• The Canadian Head CT aid seeks to identify head injuries that required neurosurgical intervention, not those that would resolve with no alteration in management.  One must decide whether this is the outcome both provider and patient care about.

Is the patient part of the applicable population?

• For example, it's important to note that the Canadian Head CT aid only applies to patients with: GCS 13-15, witnessed LOC, amnesia to the head injury event, or confusion and the authors excluded patients with "minor head injuries" that didn't have one of the aforementioned factors (see this post for more specific discussion of this example) [6].  Broadly applying the tool to patients who don't meet inclusion criteria or were excluded in the studied populations may lead to inappropriate stratification or intervention.
• The performance of decision aids may depend on the prevalence of disease in the population. For example, PERC and Wells perform less well in high prevalence populations [5].
• Various other factors such as developing vs developed setting, resources, etc may also alter the applicability of the decision aid in one's population. The more similar a paper's population is to your own, the more usable the decision aid.  For example, some decision aids may rely on a neurological exam performed by a neurologist versus an emergency physician.

Has the decision aid been validated? If so, how?

• Once a group derives a CDI, the tool must be validated to test it's rigor.  Dr. Newman gives a great explanation on this podcast (20 min mark). There are a few ways in which this typically happens:
• Internal or external - validated in the same institution(s) or in other populations
• Prospective or retrospective - data collected prospectively or retrospectively
• Statistical or clinical - tool validated through statistical means or in "real life." The latter demonstrates usability and utility. 
• Example: one can continue a data-collection study of parameters of the derivation portion of the study or one can use the tool in a population going forward to determine clinical utility. This is an example of the latter using the Canadian Head CT aid.

Know whether a decision tool is a one-way or two-way instrument.  Misapplication of these tools may lead to excessive resource utilization and undermine the specificity of the aids.  [1]

• One Way Decision Tools - Useful if all criteria are met.
• Example: If someone is negative by PERC when utilized appropriately, it can indicate that the patient's risk of PE is below the test threshold. Conversely, one cannot say that if a patient is not PERC negative, then they necessitate work up for PE. 
• Two Way Decision Tools - Can help a clinician decide both when to pursue an action and when not to pursue the action.  The "Ottawa ankle rule" is an example. [1]

Note: I'm a mere novice with minimal statistics or EBM training, so these thoughts are more to be a reminder for myself than an in-depth analysis.

References:

1.  Green SM.  When do clinical decision rules improve patient care?  Ann Emerg Med. 2013 Aug;62(2):132-5. doi: 10.1016/j.annemergmed.2013.02.006. Epub 2013 Mar 30.

3.  Kline, J.  Thromboembolism.  Tintinalli's Emergency Medicine.  7th ed.  p 434.

4.  Kline, J. Prospective multicenter evaluation of the pulmonary embolism rule-out criteria. Thromb Haemost. 2008 May;6(5):772-80. doi: 10.1111/j.1538-7836.2008.02944.x. Epub 2008 Mar 3.
5. Lucassen W, Geersing GJ, Erkens PM, et al. Clinical decision rules for excluding pulmonary embolism: a meta-analysis. Ann Intern Med. 2011 Oct 4;155(7):448-60. doi: 10.7326/0003-4819-155-7-201110040-00007.
6.  Stiell IG, Lesiuk H, Wells GA, et al.  The Canadian CT Head Rule Study for patients with minor head injury: rationale, objectives, and methodology for phase I (derivation). Ann Emerg Med. 2001 Aug;38(2):160-9.