I've Been Framed! My #twitterstatslesson

The Gist:  The way in which one frames a drug or intervention may severely alter the way in which one perceives the most objective information.  Patients and physicians are similarly affected by framing bias (a cognitive bias, perhaps amenable to metacognition).  The use of absolute risks and terms rather than relative terms helps mitigate this bias.  Number needed to treat (NNT) and number needed to harm (NNH) are probably better ways of evaluating interventions and testing, even though these have limitations as well.  Bookmark theNNT.com.

I have no statistics background except a biostatistics class that's part of my MPH curriculum, and attempts to keep up with SMART EM and Dr. David Newman on EM:RAP's mini JC section.  I realized we're often not formally taught to interpret and integrate bits of statistics, although the USMLE Step 2 is now beginning to attempt testing this aspect.  Thus, this stuff is important both to trainees and clinicians.  As I've begun to learn to sort through evidence and literature, I've found that FOAM delivers.  One evening, I found following conversation regarding absolute versus relative risk among some world-renowned physicians dominated my Twitter feed (only parts are listed below):

@precordialthump @karimbrohi @broomedocstrials judged by ARR, trials applied by RRR
— Scott Weingart (@emcrit) September 11, 2012

Discrepancy worse for lower baseline risk: 1% -> 0.9%. NNT = 1000.Only RR is unchanged across populations. #twitterstatslesson :)
— Karim Brohi (@karimbrohi) September 11, 2012

X in your population lowers to 9%. ARR=1%, NNT=100. But baseline risk in my population 20%. X lowers to 18%. ARR=2%, NNT=50. @broomeddocs
— Karim Brohi (@karimbrohi) September 11, 2012

NNT a flawed measure of utility:@broomedocs Say intervention X reduces risk of death by 10%. Your population baseline risk of death = 10%.
— Karim Brohi (@karimbrohi) September 11, 2012

RT @precordialthump: @karimbrohi If baseline event rate different, RCT lacks external validity & doesn't apply to your population.[No...
— Karim Brohi (@karimbrohi) September 11, 2012

@precordialthump I take your point though. The problem is that RRR is amenable to spin by those on the dark side...
— Chris Nickson (@precordialthump) September 11, 2012

So, I began to think about my own shortcomings in understanding these statistical dilemmas and figures.  I found an older podcast by the St. Emlyn's group on iTunes under "Statistics 2" to be helpful in sorting out absolute and relative risks, as mentioned in the Twitter conversation.  Then, amazingly, I stumbled upon a paper on framing the following day via Emergency Medicine Abstracts.  Apparently, I had good reason to worry...

We frame things constantly, providing a context for information and a means of understanding the information.  Like most things, this has both positive and negative implications.  For example, when one is trying to admit a patient to a particular service, one might phrase things rather differently than when one tells a patient why you plan on discharging them.  Oftentimes we use paternalism or the legitimate best interest of the patient to justify these 'frames.'

However, pharmaceutical companies do this as well through well-crafted consumer (and provider) advertising.  For example, this advertisement states that Pradaxa (dabigatran) reduced stroke risk 35% more than warfarin.  Thus, individuals may believe that the drug will reduce their stroke risk by a similar amount.  However, the 35% reduction in stroke risk only existed for a subset of patients.  Patients and populations are heterogeneous so these numbers don't necessarily apply any given patient.  As a pretty well educated bunch, health care providers should be really good at detecting framing differences and less susceptible to this bias, right?  Wrong!

The Paper:  Perneger T,  Agoritsas T.  Doctors and Patients’ Susceptibility to Framing Bias: A Randomized Trial.  J Gen Intern Med 26(12):1411–7

• Sent questionnaires to physicians and patients recently discharged from the hospital asking the respondent to judge the efficacy of a new drug based on the information provided
• Information on the new and old drug was identical but presented in only one of the following ways:  
• Absolute mortality:  with the new drug, 4% died by the end of the study versus 6% for those who received the old drug
• Absolute survival: with the new drug, 96% lived versus 94% who lived with the old drug.   Least favorable reviews of the drug by both groups (Physicians 51.8%, patients 51.7%, p=0.98 between groups)
• Relative mortality reduction:  of those who received the old drugthose that got the new drug had their mortality reduced by 1/3.  Both patients and physicians presented with relative mortality reduction perceived the drug most favorably (Physicians 93.8%, patients 89.3% p=0.11 between groups).  The odds ratio (OR) of a respondent perceiving the drug more favorably with the information presented in this format compared with other formats was 4.40 (95% CI 3.05 – 6.34, p<0.001)
• All three presentations of risk.  with the new drug, 96% lived, 4% died versus 94% lived, 6% died  with the old drug, so those that got the new drug had their mortality reduced by 1/3
• Physicians were also tested on the NNT (50) and the relative survival extension.
• Doctors responded to the presentations with no statistical difference from the patients. OR of Doctor (versus patient) = 1.06 (95% CI 0.87 – 1.29, p = 0.55)
• The notion that information presented in relative terms rather than absolute terms is more favorably received is not new (1).

There is so much to learn from the ever-relevant FOAM world.

References:
1.  Malenka DJ, Baron JA, Johansen S, Wahrenberger JW, Ross JM. The framing effect of relative and absolute risk. Journal of general internal medicine. 1993;8(10):543-8.