Diagnosis-based algorithms attempt ED triage.
Triage ensures the proper use of resources and the delivery of timely care. In 2021, an estimated 140 million visits were made to U.S. emergency departments (EDs). Triaging remains of vital importance in the distribution of resources given the increasing demand on EDs.
Historically, triage has required a nurse to consider one of a variety of different models that commonly include data such as vital signs, chief complaints, and symptoms to determine potential disease severity. Artificial Intelligence (AI) has the potential to aid in triage due to its widespread availability, low cost, quick speed, and ability to free the nurse to perform other tasks. However, few studies have been done to assess the use of AI in the triaging process.
A single-center prospective observational study was performed to address this gap. It used data including chief complaints, vital signs, age, sex, and medical history. Information was put into a GPT-4 (an AI model that incorporates data from text and pictures) to determine appropriate triage of patients. An emergency physician and triage team also independently triaged patients, and the two decisions were compared. A total of 758 patients were included in the study over 3 days.
There was significant agreement between GPT-4's decisions and those of the triage teams; however, lower sensitivity and compliance were found in "moderate acuity" patients, likely due to the varying patient profiles and chief complaints. This study is limited by its short study time, unrepresentative distribution of patients among acuity levels, and GPT-4 requiring frequent input of the local rules, which may hinder implementation.
GPT-4's ease of access, low cost, and widespread availability makes its role in health care beneficial; however, this study did not address its significant privacy concerns. AI may increase the risk of breach of data in cybersecurity attacks. Furthermore, for accurate triaging, AI may require the use of protected health information to train its future models.
If AI is utilized in triaging decisions, policies regarding the type of information entered into models, informed consent discussions, as well as additional security barriers to further protect health care information, must be developed prior to implementation.
ABSTRACT
Paslı S, Samet Şahin A, Fatih Beşer M, Topçuoğlu H, Yadigaroğlu M, İmamoğlu M. Assessing the precision of artificial intelligence in ED triage decisions: Insights from a study with ChatGPT. Am J Emerg Med. 2024;78:170-175.
Background: The rise in emergency department presentations globally poses challenges for efficient patient management. To address this, various strategies aim to expedite patient management. Artificial intelligence's (AI) consistent performance and rapid data interpretation extend its healthcare applications, especially in emergencies. The introduction of a robust AI tool like ChatGPT, based on GPT-4 developed by OpenAI, can benefit patients and healthcare professionals by improving the speed and accuracy of resource allocation. This study examines ChatGPT's capability to predict triage outcomes based on local emergency department rules.
Methods: This study is a single-center prospective observational study. The study population consists of all patients who presented to the emergency department with any symptoms and agreed to participate. The study was conducted on three non-consecutive days for a total of 72 hrs. Patients' chief complaints, vital parameters, medical history and the area to which they were directed by the triage team in the emergency department were recorded. Concurrently, an emergency medicine physician inputted the same data into previously trained GPT-4, according to local rules. According to this data, the triage decisions made by GPT-4 were recorded. In the same process, an emergency medicine specialist determined where the patient should be directed based on the data collected, and this decision was considered the gold standard. Accuracy rates and reliability for directing patients to specific areas by the triage team and GPT-4 were evaluated using Cohen's kappa test. Furthermore, the accuracy of the patient triage process performed by the triage team and GPT-4 was assessed by receiver operating characteristic (ROC) analysis. Statistical analysis considered a value of p < 0.05 as significant.
Results: The study was carried out on 758 patients. Among the participants, 416 (54.9%) were male and 342 (45.1%) were female. Evaluating the primary endpoints of our study - the agreement between the decisions of the triage team, GPT-4 decisions in emergency department triage, and the gold standard - we observed almost perfect agreement both between the triage team and the gold standard and between GPT-4 and the gold standard (Cohen's Kappa 0.893 and 0.899, respectively; p < 0.001 for each).
Conclusion: Our findings suggest GPT-4 possesses outstanding predictive skills in triaging patients in an emergency setting. GPT-4 can serve as an effective tool to support the triage process.
EMRA + PolicyRx Health Policy Journal Club: A collaboration between Policy Prescriptions and EMRA
As emergency physicians, we care for all members of society, and as such have a unique vantage point on the state of health care. What we find frustrating in our EDs - such as inadequate social services, the dearth of primary care physicians, and the lack of mental health services - are universal problems. As EM residents and fellows, we learn the management of myocardial infarctions and traumas, and how to intubate, but we are not taught how health policy affects all aspects of our experience in the ED. Furthermore, given our unique position in the health care system, we have an incredible opportunity to advocate for our patients, for society, and for physicians. Yet, with so many competing interests vying for our conference education time, advocacy is often not included in the curricula. This is the gap this initiative aims to fill. Each month, you will see a review of a new health policy article and how it is applicable to emergency physicians.