Evidence-based medicine is the foundation of how we care for our patients. Every medical decision we make on shift stems from some level of evidence. As physicians, we need to feel confident with critically appraising new research.
Undergraduate and graduate medical education provides minimal preparation for scrupulous literature review. Complex statistical analyses and long-winded articles can quickly become overwhelming without a simplified and consistent approach to article review.
What’s the Question?
Instead of reading an extensive introduction, skip to the punch line. Find out immediately what the question is the authors are trying to answer. This will provide you a frame of reference for the entire study and will allow you do determine if the introduction justifies the importance and originality of the question being asked. Specifically, look at the objective, hypothesis and primary outcome. This is important because the purpose of the study is to evaluate the objective and/or hypothesis through measurement of the primary and secondary outcomes. Some journals have begun to incorporate an editor’s “capsule summary,” which can be helpful to ascertain the study question.
The Devil Is in the Details
The hardest sections to read are often the most important sections to understand. A paper with poorly designed methods and/or inaccurately reported results can invalidate the entire study. Take your time here. Read it more than once to have a good understanding of how the data was collected.
- Begin with answering who, what, where when and how?
- Who: Consider inclusion and exclusion criteria. Is a selection bias present?
- What: What intervention is being performed and what measurements were taken? Is this a patient centered outcome or a laboratory centered outcome?
- Where: What is the study setting? Is it similar to the one I practice in?
- When: Was the study performed a significant amount of time prior to publication?Have patient care practices changed since then? Review the references for any older references and out-of-date citations. For example, has more current research on the topic been published recently and not included in the references?
- How: Analytic strategy and data analyses used.
- Consider internal validity. In other words, are the study design and collection methods appropriate for the question they are trying to answer?
- Consider external validity. Do the results translate to my practice? Are the results generalizable?
Results and Statistical Analyses: Do the Numbers Answer the Questions?
Stay organized in the results sections and keeps things simple. It’s easy to get overwhelmed with statistical tests, sensitivity analyses and subgroup analyses.
- Do the results parallel the methods? Are the specified outcomes measured and reported appropriately?
- Carefully compare the patient cohorts. Are they similar? Are there confounders potentially influencing the results?
- Is the size of the study population appropriate? Was a power calculation reported? Remember that a study is usually only powered based on the primary outcome. Interpret the secondary outcome variables and other a priori or post hoc analyses with this in mind.
- Were the right statistical tests utilized? We went to school to be Emergency Physicians, not statisticians. Most of us do not have an epidemiology background or expertise in statistics. However, we should be able to determine if the statistical tests used were appropriate based on study design and types of variables measured.
- Check the math! Recreating a linear regression curve may be unreasonable, but take the time to make your own 2x2 table and confirm reported sensitivities, specificities, and more. You may be surprised that some published studies have discrepancies here. And you may need to perform these calculations on board examinations.
Does This Change My Practice?
Finally, when reviewing the discussion section, pay careful attention to the authors’ conclusion based on the results presented and the existing literature.
- Consider limitations in the study that could severely affect the results, validity and generalizability.
- Do you agree with the authors’ conclusions? Remember that the conclusion should be based on the objectives and hypotheses. Be skeptical of magnanimous statements. Studies that report extreme results are often poorly designed and prone to inherent bias, or they answer a question we already know to be true or nobody finds relevant.
- Is this relevant to emergency medicine? Can I take the study to the bedside?
Critical appraisal of research does not require a PhD or background in statistics. Stay organized, use a consistent approach, and keep things simple. Lastly, take pride in critical appraisal of the literature; our patients and your career are depending on it!
Reference
Atzema C. Presenting at journal club: a guide. Ann Emerg Med. 2004;44(2):169-174.