The Modern Matthew Effect

The Gist:  In medicine and science, regardless of the medium - traditional or Free Open Access Medical education (FOAM)- the Matthew effect exists, potentially perpetuating knowledge and dogma that doesn't necessarily reflect intrinsic worth.  Question the medical dogma, respectfully and, while it's easy to copy and paste a citation for a quote a popular figure, consider critically evaluating the source of information or primary literature.  In words borrowed from TheSGEM podcast, "Be skeptical of everything you learn.." (to a healthy, not pathologic degree) - it's another arrow in the metacognition quiver.

Conversations on the perils of FOAM at the Social Media and Critical Care Conference (SMACC) spurned the following, something I think is worth reminding ourselves of from time to time:

Do not take what I say as gospel- @emcrit #smacc2013
— smacc (@smacc2013) March 11, 2013

While many of us exercise healthy skepticism we can still fall victim to a common phenomenon because, in the words of Daniel Kahneman in Thinking Fast and Slow, we have "almost unlimited ability to ignore our ignorance."  We may think we are fully aware of our biases, but they are worked into the fabric of life.  

The Matthew Effect (with regard to references):  essentially, the greater number of times a paper is cited, the more citations it will receive.   Coined by Merton in this paper, but initially researched by Harriet Zuckerman, it is borrowed from the Gospel according to St. Matthew: “For to all those who have, more will be given, and they will have an abundance; but from those who have nothing, even what they have will be taken away” (Matthew 25:29) (1).  

• The Matthew effect is partially a byproduct of quality.  A content expert likely becomes trusted and their work becomes highly regarded due to the merit of their prior work(s).  Thus, this is a sort of natural phenomenon in any field that has experts/masters in particular disciplines.  Zuckerman identified this in that Nobel Prize winners tended to generate/produce more awards compared with those who had shared equally in the project but were more junior researchers (1).

How does this manifest in medical literature?

High Impact Journals.  Impact factor (IF) - The IF is essentially the average number of times an article in the journal is cited within the previous two years.  A journal’s prestige is a function of the quality of the articles appearing in it. 

• What happens when the exact same article (title, author, etc) is published in two journals with disparate IFs?  This paper by Lariviere and Gingras (full text) took at look at this question and found that duplicate papers (4532 pairs of papers) in high impact journals obtain, on average, twice as many citations as their identical counterparts published in journals with lower IFs.  
• The intrinsic value of a paper is not the only reason for the citation of a specific paper; there is a Matthew effect attached to journals.  Thus, a paper published in a high impact journal has an added value over its intrinsic quality and will generate more citations.

High Impact Authors.  A high profile author's paper is likely to carry more weight or gain more recognition.

• Example:  In the Feb. 2013 edition of Emergency Medical Abstracts (subscription required), there's a little bit of banter about how this paper on incidence of contrast induced nephropathy was referred to ("the Kline paper").  Pulmonary embolism guru Dr. Jeff Kline is listed on the paper but the first author is actually Dr. AM Mitchell.  

This sounds like splitting hairs..  Perhaps a little, but not necessarily.

• Implicit in the concept of the Matthew effect is the notion is that a piece of research is more valuable or important because of its association with an individual rather than the contents, quality, or implications of the research.
• Without realizing it, we may become susceptible to a cognitive bias secondary to the "Halo Effect," which I first heard about in Thinking Fast and Slow by Daniel Kahneman.  For example, if an individual is widely regarded in the community for a podcast or publication, their institution may be looked upon more favorably.  
• Dr. Weingart's tweet at the beginning of this post demonstrates potential implications of the halo effect - a positive/powerful reputation may have undue influence over whether we see that information as important or valid.  If someone we respect says an article is a "must read" or "garbage" we have formed an impression of the article prior to actually reading it.  They may very well be spot on, but this is something to keep in mind.
• In an era of information overload, especially in medicine, we may deal with this cognitive load by perceiving a reputable person's recommendations as most/more important (known as positional cues).  This may skew our evidence base or perception of prevalence or importance of a medical problem.

Is FOAM impervious to this effect? No.

• FOAM has a form of Impact Factor.  This can be quantified in retweets, blog hits, or a spot  in a Life in the Fast Lane Weekly Review.  Again, this is not necessarily a negative thing and can be harnessed "for good," introducing innovative or important ideas quickly and diffusely across the globe.  
• Example:  

 

• The social connections and the platforms associated with FOAM are intricate (hospital and professional networks, friends/families (social media), affiliations with societies, etc).  As a result, the Matthew effect may be less like the "Nobel Prize" effect noted by Zuckerman as age, rank, and location may not carry as much weight and the sources are vast.
• Recently, Google announced that it would drop its RSS aggregating service, GoogleReader. This move immediately induced a Twitter frenzy regarding replacement services.  One focus of conversation on this topic from some members of the FOAM community was that Twitter has replaced the need for RSS.  This article discusses this notion, a debatable assertion that I don't personally find applicable to my use of RSS.  Should Twitter supplant RSS, individuals who use an RSS aggregator to review journals and/or medical blogs may have increased susceptibility to biases associated with a social media/recommendation system based system. 

So what do we do?

• Question productively and respectfully.  

@akutdok @thomas1973 @mdaware @smacc2013 @precordialthump @sandnsurf If listeners stop questioning every word I say, I will stop the podcast
— Scott Weingart (@emcrit) March 13, 2013

• Check sources.  For example, while putting together this post on elevated blood pressure in the ED, I came across a statistic in Tintinalli:  3.8% of headaches in the ED have serious intracranial pathology (Ch. 159).  Initially, I copied this statistic and reference because Tintinalli is one of the core EM texts.  FOAM has inspired me to check things out further, and upon evaluating the study I found it underwhelming to support the rate quoted.  This study was referenced by others as well, including the famous Perry et al article on subarachnoid hemorrhage and others. 
• Keep the Matthew effect in mind when evaluating articles, watching posts/ideas go "viral", or evaluating the validity of an assertion or claim.

Updated 3/18/13.

References:
1.  Zukerman H.  Scientific Elite:  Nobel Laureates in the United States. 2d ed. (New York:  Transaction Publishers, 1996).