Presentation
PS8 - Leveraging LLM to Identify Missed Information in Patient-Physician Interactions: Improving Healthcare Service Quality
SessionPoster Session 2
DescriptionNew technologies have significantly changed the dynamics of physician-patient interactions. For example, Alkureishi et al. (2016) found that physicians spend more time on electronic medical records (EMR) than on direct communication with patients. While this shift has improved efficiency and documentation accuracy, it has led to a decline in patient satisfaction with the interpersonal aspects of care. Also, the essential information during the visiting can be missed by communication breakdowns between physicians and patients which leads to the result of missed diagnosis. Diagnostic errors in primary care are a significant concern, with studies indicating that they occur in approximately 5% to 15% of encounters, depending on the conditions examined and the study parameters. Recently, various artificial intelligence (AI) tools such as Large Language Models (LLMs) have been adopted in healthcare (Harmon et al., 2022), it is significant for us to find a method to identify the patterns of the visiting with different tools to ensure the quality of visiting. In particular, we will evaluate quality of communication and decision-making during patient-physician interactions in a visiting with LLM.
The primary objective of this study is to identify patterns of interaction during clinical visits and check if physicians have missed any information or procedures which are suggested by the BMJ Best Practices. Specifically, we analyze the video data from a previous study with survey on patient satisfaction collected after each clinical visit. There are 10 physicians and 100 eligible patients participate in this study, including 56 males and 44 females in the patient group. And the eligible patients are between 18 and 65 years old, English speakers. By using the transcripts of the videos, we applied one of the variant of LLM: Phi3 to evaluate whether physicians adhere to standard medical protocols and adequately address patient concerns. By doing so, we aim to minimize the risk of misdiagnosis, enhance communication, and improve overall patient satisfaction and safety. We then examine the relationship between different types of missed information or procedures and the level of patient satisfaction. Our findings indicate that failing to check patients’ past medical history and address their needs results in a decrease in patient satisfaction.
Our methodology involves processing transcripts to identify instances where critical information was missed or patient concerns were not addressed. These insights are used to evaluate the quality of communication and decision-making. The analysis also investigates how the adoption of Phi3 impacts the efficiency and effectiveness of healthcare consultations by bridging communication gaps caused by time constraints, medical jargon, or misunderstandings of patient needs. By leveraging Phi3's capabilities, this study provides data-driven insights to improve the alignment between patients' expectations and physicians' practices. The ultimate goal is to create a framework that enhances healthcare quality by reducing miscommunication and improving adherence to medical protocols. Through this approach, we aim to ensure safer, more effective, and more satisfying healthcare experiences for both patients and providers.
The primary objective of this study is to identify patterns of interaction during clinical visits and check if physicians have missed any information or procedures which are suggested by the BMJ Best Practices. Specifically, we analyze the video data from a previous study with survey on patient satisfaction collected after each clinical visit. There are 10 physicians and 100 eligible patients participate in this study, including 56 males and 44 females in the patient group. And the eligible patients are between 18 and 65 years old, English speakers. By using the transcripts of the videos, we applied one of the variant of LLM: Phi3 to evaluate whether physicians adhere to standard medical protocols and adequately address patient concerns. By doing so, we aim to minimize the risk of misdiagnosis, enhance communication, and improve overall patient satisfaction and safety. We then examine the relationship between different types of missed information or procedures and the level of patient satisfaction. Our findings indicate that failing to check patients’ past medical history and address their needs results in a decrease in patient satisfaction.
Our methodology involves processing transcripts to identify instances where critical information was missed or patient concerns were not addressed. These insights are used to evaluate the quality of communication and decision-making. The analysis also investigates how the adoption of Phi3 impacts the efficiency and effectiveness of healthcare consultations by bridging communication gaps caused by time constraints, medical jargon, or misunderstandings of patient needs. By leveraging Phi3's capabilities, this study provides data-driven insights to improve the alignment between patients' expectations and physicians' practices. The ultimate goal is to create a framework that enhances healthcare quality by reducing miscommunication and improving adherence to medical protocols. Through this approach, we aim to ensure safer, more effective, and more satisfying healthcare experiences for both patients and providers.
Event Type
Poster Presentation
TimeTuesday, April 14:45pm - 6:15pm EDT
LocationFrontenac Foyer