Overlooked Sources of Bias in Adult Out-of-Hospital Cardiac Arrest Research

Bias is prevalent throughout the medical literature; however, out-of-hospital cardiac arrest (OHCA) studies have unique sources of bias due to the complex disease state.

These sources of bias are often overlooked without efforts to mitigate their effects on study results. We review commonly overlooked sources of bias specific to OHCA studies to facilitate critical appraisal.

Introduction and Background
Sources of error threaten the validity of medical research. In medical studies, bias is the non-random effect of error on study results.¹ Although there are many types of bias,^1,2 each type can have various causes.^1,3 Bias is prevalent throughout the medical literature,^3,4 and it has variable consequences, from insignificant changes in effect magnitude to false positive and false negative study results.³ However, studies on OHCA have unique causes of bias.

OHCA research has the potential for impactful improvements in patient care; however, there are multiple barriers to conducting unbiased OHCA studies. OHCA occurs when the heart stops due to an abnormal cardiac rhythm.⁵ It is common, occurring in 60-140 adults per 100,000 population; however, only 9% of OHCAs treated by emergency medical services had survival with good neurological outcomes.⁶ Despite this disease burden, OHCA is challenging to study with multiple etiologies, an unpredictable onset, an immediate need for intervention, and a complex recovery phase.^7,8 These challenges create avenues for bias affecting enrollment, treatment, analysis, and implementation of OHCA studies.

Recognizing these unique sources of bias in OHCA research is critical to preventing misinterpretation and misapplication of study results. Therefore, the purpose of this review is to summarize commonly overlooked sources of bias that impact OHCA research.

Review: Enrollment
OHCA studies often exclude patients with characteristics that are expected to have poor outcomes, including those with non-shockable cardiac rhythms.⁹ While this mitigates the risk of false negative study results, it can impair the generalizability of the study population. This selective enrollment of subjects is known as sampling bias, occurring when enrollment yields a study population that differs systematically from the population of interest.² When sampling bias underrepresents one sex over another, a sex bias may occur.¹⁰ For example, excluding OHCAs with non-shockable rhythms can result in a lower proportion of female subjects in the study population since women in OHCA are less likely to present in a shockable rhythm.¹¹ Therefore, inclusion and exclusion criteria in OHCA studies may inadvertently cause a sex bias and make study results less applicable to female patients. Readers should look for a representative sample of female subjects.

Similarly, black race is associated with predictors of poor OHCA outcomes that may preclude enrollment, resulting in a race bias. For example, in a systematic review and meta-analysis of 15 OHCA studies evaluating outcomes between races, black subjects were less likely to have an initial shockable rhythm, bystander resuscitation, or a witnessed arrest compared to white subjects.¹² Therefore, black subjects may be systematically excluded from enrollment of OHCA studies due to predictors of poor outcomes, which may make study results less applicable to black patients. Readers should look for a representative sample of minority subjects.

Review: Treatment
Similar to sampling issues during enrollment, the selection of patients for post-arrest interventions can be a significant source of bias. Like intra-arrest interventions, post-arrest care is critical to OHCA outcomes.¹³ However, unlike intra-arrest interventions such as chest compressions and basic airway management that are indicated for all OHCA patients,¹⁴ aspects of post-arrest care may be provided to study treatment groups unequally due to patient-related factors. This selective treatment indicates a selection bias resulting from study groups differing in ways other than the defined independent variables.²

For example, post-arrest cardiac catheterization and implantable cardioverter defibrillators are associated with reduced all-cause mortality, but not cardiac-specific mortality as would be expected.¹⁵ Therefore, those who receive these interventions are less likely to die from non-cardiac causes than those who do not.¹⁵ While this difference questions the efficacy of these interventions post-arrest, it also indicates a selection bias in who receives these interventions post-arrest.¹⁵ Therefore, the selection of patients for post-arrest treatments may compromise OHCA study results. Readers should look for measurement of and adjustment for post-arrest interventions.

Review: Analysis
A lack of measurement and adjustment of confounding variables is a common source of error in OHCA study analyses. Confounding variables are factors associated with the exposure and outcome of interest that mask the true association or lack of association.² For example, end-of-life decisions may confound mortality outcomes in OHCA studies. However, in a systematic review of 178 randomized control trials of OHCA with in-intensive care unit mortality as an outcome, only 62 (35%) studies addressed end-of-life decisions in their methodology.¹⁶

Similarly, channeling bias occurs when the severity of illness dictates the exposure.³ For example, a patient who is comatose after resuscitation from OHCA has lower odds of survival and would receive additional treatments like mechanical ventilation compared to a patient following commands and protecting the airway.¹⁷ However, in a review of 213 OHCA studies with survival outcomes, only 60 (28%) included illness severity or comorbidity indices and scores, and only 39 of those used the indices and scores to adjust for confounding.¹⁸ Therefore, readers should look for measurement and adjustment for end-of-life decisions, illness severity, and comorbidities, which may confound OHCA study results.

Timing of interventions intra- and post-arrest are also major confounders in OHCA study analyses. Resuscitation time bias refers to the concept that the longer a patient is in cardiac arrest, the more likely they will receive interventions but the less likely they will survive.¹⁹ This association may bias cardiac arrest studies by making intra-arrest interventions like positive pressure ventilation, which usually occurs later in the resuscitation, seem less effective.²⁰Meanwhile, immortality time bias refers to the concept that the longer a critically ill patient (i.e., a post-arrest patient) is alive, the more likely they will receive an intervention and survive.¹⁹ In contrast to resuscitation time bias, immortality time bias may make post-arrest interventions like cardiac catheterization seem more effective.²¹Therefore, readers should look for measurement of and adjustment for the timing of interventions, which may confound OHCA study results.¹⁹

Review: Implementation
OHCA studies often ignore implementation barriers to the studied interventions. Barriers to implementation threaten the generalizability of study results or their applicability to specific populations.³ For example, OHCA patients in rural or difficult to access locations (e.g., high-rise buildings) have worse outcomes due to longer response times and practical barriers to accessing advanced care (e.g., endotracheal intubation).^22-25 Therefore, an intervention may be efficacious when there is a brief response time and more experienced clinicians, but it may be less efficacious when there is a prolonged response time and less experienced clinicians.

Conversely, some interventions facilitate the care of OHCA in rural or difficult to access locations but do not improve outcomes overall. For example, mechanical chest compression devices do not improve outcomes compared to high-quality manual chest compressions.²⁶ However, these devices have practical benefits for clinicians performing prolonged resuscitations or chest compressions in a moving ambulance,²⁶ mitigating existing implementation barriers. Therefore, readers should consider implementation barriers when interpreting study results.

Conclusions
Although bias is pervasive in medical research, OHCA studies have unique sources of bias that are infrequently addressed in study methods (Table). Recognizing these biases is vital to the critical appraisal and application of the OHCA literature. Therefore, OHCA study results should be interpreted within the limitations of these sources of bias.

Table: Summary of Overlooked Sources of Bias in Out-of-Hospital Cardiac Arrest Research

References

1. Straus S, Glasziou P, Richardson W, Haynes R. Evidence-Based Medicine: How To Practice and Teach EBM. 5th ed. Elsevier; 2018.
2. Biases. Centre for Evidence-Based Medicine. Catalog of Bias. Accessed 2021-09-24.
3. Pannucci CJ, Wilkins EG. Identifying and avoiding bias in research. Plast Reconstr Surg. 2010;126(2):619-625.
4. Bradley SH, DeVito NJ, Lloyd KE, et al. Reducing bias and improving transparency in medical research: a critical overview of the problems, progress and suggested next steps. J R Soc Med. Nov 2020;113(11):433-443.
5. Soar J, Becker LB, Berg KM, et al. Cardiopulmonary resuscitation in special circumstances. Lancet. Oct 2 2021;398(10307):1257-1268.
6. Benjamin EJ, Virani SS, Callaway CW, et al. Heart Disease and Stroke Statistics-2018 Update: A Report From the American Heart Association. Circulation. Mar 20 2018;137(12):e67-e492.
7. Institute of Medicine. Strategies to Improve Cardiac Arrest Survival: A Time to Act. The National Academies Press. 2015.
8. Panchal AR, Bartos JA, Cabañas JG, et al. Part 3: Adult Basic and Advanced Life Support: 2020 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2020;142(16_suppl_2):S366-S468.
9. Cummins RO, Chamberlain DA, Abramson NS, et al. Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest: the Utstein Style. A statement for health professionals from a task force of the American Heart Association, the European Resuscitation Council, the Heart and Stroke Foundation of Canada, and the Australian Resuscitation Council. Circulation. Aug 1991;84(2):960-75.
10. Feldman S, Ammar W, Lo K, Trepman E, van Zuylen M, Etzioni O. Quantifying Sex Bias in Clinical Studies at Scale With Automated Data Extraction. JAMA Netw Open. 2019;2(7):e196700-e196700.
11. Kim LK, Looser P, Swaminathan RV, et al. Sex-Based Disparities in Incidence, Treatment, and Outcomes of Cardiac Arrest in the United States, 2003-2012. J Am Heart Assoc. Jun 22 2016;5(6)e003704.
12. Shah KS, Shah AS, Bhopal R. Systematic review and meta-analysis of out-of-hospital cardiac arrest and race or ethnicity: black US populations fare worse. Eur J Prev Cardiol. May 2014;21(5):619-38.
13. Merchant RM, Topjian AA, Panchal AR, et al. Part 1: Executive Summary: 2020 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2020;142(16_suppl_2):S337-S357.
14. Olasveengen TM, Semeraro F, Ristagno G, et al. European Resuscitation Council Guidelines 2021: Basic Life Support. Resuscitation. Apr 2021;161:98-114.
15. Wallace DJ, Coppler P, Callaway C, et al. Selection bias, interventions and outcomes for survivors of cardiac arrest. Heart. Aug 2018;104(16):1356-1361.
16. Kerever S, Jacquens A, Smail-Faugeron V, Gayat E, Resche-Rigon M. Methodological management of end-of-life decision data in intensive care studies: A systematic review of 178 randomized control trials published in seven major journals. PLoS One. 2019;14(5):e0217134.
17. Coppler PJ, Elmer J, Calderon L, et al. Validation of the Pittsburgh Cardiac Arrest Category illness severity score. Resuscitation. 2015;89:86-92.
18. Fouche PF, Carlson JN, Ghosh A, Zverinova KM, Doi SA, Rittenberger JC. Frequency of adjustment with comorbidity and illness severity scores and indices in cardiac arrest research. Resuscitation. Jan 2017;110:56-73.
19. Andersen LW, Grossestreuer AV, Donnino MW. “Resuscitation time bias”-A unique challenge for observational cardiac arrest research. Resuscitation. Apr 2018;125:79-82.
20. Bobrow BJ, Ewy GA, Clark L, et al. Passive oxygen insufflation is superior to bag-valve-mask ventilation for witnessed ventricular fibrillation out-of-hospital cardiac arrest. Ann Emerg Med. Nov 2009;54(5):656-662.e1.
21. Dumas F, Cariou A, Manzo-Silberman S, et al. Immediate percutaneous coronary intervention is associated with better survival after out-of-hospital cardiac arrest: insights from the PROCAT (Parisian Region Out of hospital Cardiac ArresT) registry. Circ Cardiovasc Interv. Jun 1 2010;3(3):200-7.
22. Abbott EE, Buckler DG, Zebrowski AM, Abella BS, Carr BG. Abstract 134: Survival After Out-of-hospital Cardiac Arrest: The Role of Urban-rural Residence and Demographic Factors. Circulation. 2020/11/17 2020;142(Suppl_4):A134-A134.
23. Jennings PA, Cameron P, Walker T, Bernard S, Smith K. Out-of-hospital cardiac arrest in Victoria: rural and urban outcomes. Med J Aust. Aug 7 2006;185(3):135-9.
24. Mathiesen WT, Bjørshol CA, Kvaløy JT, Søreide E. Effects of modifiable prehospital factors on survival after out-of-hospital cardiac arrest in rural versus urban areas. Critical Care. 2018/04/18 2018;22(1):99.
25. Drennan IR, Strum RP, Byers A, et al. Out-of-hospital cardiac arrest in high-rise buildings: delays to patient care and effect on survival. Can Med Assoc J. 2016;188(6):413-419.
26. Wang PL, Brooks SC. Mechanical versus manual chest compressions for cardiac arrest. Cochrane Database Syst Rev. Aug 20 2018;8(8):Cd007260.