Presentation
The Disposition Mapping for Physician Sign-out (DMAPS): Validating a Measure of Emergency Physician Workload During End of Shift Handoffs
DescriptionBackground
Recent increases in emergency physician (EP) workload have been identified as contributors to increased EP burnout, increased staff attrition, decreased patient safety, and increased patient admission rates. Patient handoffs have been extensively researched as critical points in patient care, with existing research primarily focusing on communication errors and interventions designed to standardize handoff communication protocols.
There is a scarcity of research identifying and utilizing validated measures of EP workload to evaluate patient handoffs and handoff interventions. While several studies have been conducted which primarily focus on the quality of handoff communication, understanding EP workload during handoffs requires researchers to view handoffs through the lens of EPs’ handoff workflows. Through prior qualitative research, the state of patients’ disposition at the time of handoff was identified as the primary source of workload (both task-load and mental workload) for practicing EPs (Foster et al., 2024). A patient’s disposition is their intended destination within the patient care pathway which is often either admitted to the hospital as an inpatient for further treatment, transferred to a separate medical facility for further treatment, or discharged from the ED, commonly with plans for follow-up outpatient treatment. Disposition has been well established as a keystone element of emergency physician workflows and a primary goal of emergency medical care (Rutkowski et al., 2023; Foster et al., 2024). The state of a patient’s disposition at the time of handoff is representative of the state of the progress of that patient’s visit in the ED, and thus definitionally representative of the task-load of an EP at a given point in time. Many studies have evaluated the importance of patient disposition in hospital EDs, yet research pertaining to patient handoffs in the ED scarcely mentions the importance of patient disposition in physician communication and decision-making during handoff.
Purpose Statement
Increased physician workload has previously been associated with increased EP burnout, increased staff attrition, decreased patient safety, and increased patient admission rates. Regularly occurring end of shift patient handoffs represent immediate transfers of patient caseload between EPs that may contribute to increased EP task load and mental workload. The disposition mapping for physician sign-out (DMAPS) is a patient categorization tool which indexes patients based on the state of their disposition at the time of handoff. This research examines the validity of DMAPS as an objective measure of EP workload by defining the statistical relationship between patient caseload transferred during handoffs (as categorized by DMAPS) and EPs’ perceived mental workload as measured by the NASA-TLX questionnaire. By quantifying the efficacy of DMAPS as a measure of EP workload, this research presents a novel and validated measure of EP workload designed for application during end of shift patient handoffs.
Methods
DMAPS was designed with consultation from attending EPs and was primarily based on a covariate measure of EP caseload developed by Foster et al., (2023). DMAPS is a patient categorization system designed to index the patients that are transferred during an end of shift handoff by the state of their disposition at the time of said handoff. DMAPS is comprised of 5 disposition states: Level 1 – Disposition Unknown, Level 2 – Disposition Dependent, Level 3 – Disposition Expected, Level 4 – Disposition Assigned, Level 5 – Disposition Assigned and Patient Boarding.
Patient caseload and EP workload data were collected from a large hospital system in the Southeastern United States. Participants for this study consisted of 64 attending EPs from the same large hospital system observed across 32 patient handoffs. Each handoff was observed by a researcher experienced with mapping patients using DMAPS who then categorized each patient transferred during the handoff by the state of their disposition at handoff. Both the outgoing and incoming EP of each observed end of shift handoff then completed a NASA-TLX questionnaire where they rated their anticipated remaining workload for the patients transferred during the handoffs across 5 of the 6 NASA-TLX subscales (mental demand, physical demand, temporal demand, effort, and frustration). Retrospective data were also collected based on standard measures of EP workload used in existing literature. These measures include patient volume, the proportion of high acuity patients in the handoff, and the national emergency department overcrowding score (NEDOCS).
Data were analyzed to 1) identify significant correlations between the NASA-TLX questionnaire subscales and composite dispositions based on DMAPS, and 2) the efficacy of DMAPS as a predictor of the NASA-TLX subscales in comparison to other commonly used measures of EP workload in existing literature.
Results
Results of this study identified significant negative correlations between DMAPS-based patient composites and EPs’ anticipated mental demand (r = -0.419, p = 0.001), physical demand (r = -0.371, p = 0.003), effort (r = -0.392, p = 0.001), and frustration (r = -0.318, p = 0.010). These correlations imply that as the state of a patient’s disposition decision progresses, that patient represents relatively less mental workload for the incoming EP who is responsible for the remainder of that patient’s emergency care. Furthermore, regression of the NASA-TLX questionnaire subscales by the DMAPS categorization (treating the number of patients in each category as independent variables) and select DMAPS-based composites identified the full 5-dimension DMAPS categorization as the most effective predictor of EP mental demand (r2 = 0.294), physical demand (r2 = 0.263), effort (r2 = 0.602), and frustration (r2 = 0.453) when compared against other commonly used measures of EP workload in existing literature.
Discussion
The results of this study demonstrate that DMAPS provides a more accurate and representative measure of EPs’ anticipated perceived mental workload than other commonly used measures of EP workload. DMAPS was designed specifically for evaluating the work-based workload of EPs in the ED, but also represents aspects of EPs’ mental workload as rated on the NASA TLX subscales of effort and frustration, and therefore is also an alternative to metrics such as SWAT and the NASA TLX questionnaire which may be inappropriately utilized in observational studies concerning complex tasks which lack clear timelines.
Future work based on this study should focus on further validating DMAPS as a measure of EP workload using other common measures of workload in existing literature. Further validation could be completed using time-motion studies and reliable physiological measures such as the number of steps taken by EPs during their shift. Future work should also explore the usage of DMAPS outside of the context of patient handoffs. Because DMAPS relies on researchers listening to descriptions of patients’ disposition states, DMAPS is in theory applicable as a point measure of workload at any point during an EPs shift. While this study was opportunistically conducted during end of shift patient handoffs in part because EPs discuss patient dispositions during these handoffs, future work could explore the usage of DMAPS to capture point measures of workload at any point during EPs’ shifts. Lastly, there exists little validation of the reliable usage of DMAPS by non-clinical researchers. Future research should test the accuracy and reliability of raters with diverse backgrounds and skillsets applying DMAPS to index patients in end of shift patient handoffs.
Conclusion
This research provides a novel measure of EP workload during end of shift patient handoffs that can be used to evaluate handoff and policy interventions and provide a basis for future measures of ED staff workload that can be used to address EP burnout, staff attrition, and decreased patient safety.
Recent increases in emergency physician (EP) workload have been identified as contributors to increased EP burnout, increased staff attrition, decreased patient safety, and increased patient admission rates. Patient handoffs have been extensively researched as critical points in patient care, with existing research primarily focusing on communication errors and interventions designed to standardize handoff communication protocols.
There is a scarcity of research identifying and utilizing validated measures of EP workload to evaluate patient handoffs and handoff interventions. While several studies have been conducted which primarily focus on the quality of handoff communication, understanding EP workload during handoffs requires researchers to view handoffs through the lens of EPs’ handoff workflows. Through prior qualitative research, the state of patients’ disposition at the time of handoff was identified as the primary source of workload (both task-load and mental workload) for practicing EPs (Foster et al., 2024). A patient’s disposition is their intended destination within the patient care pathway which is often either admitted to the hospital as an inpatient for further treatment, transferred to a separate medical facility for further treatment, or discharged from the ED, commonly with plans for follow-up outpatient treatment. Disposition has been well established as a keystone element of emergency physician workflows and a primary goal of emergency medical care (Rutkowski et al., 2023; Foster et al., 2024). The state of a patient’s disposition at the time of handoff is representative of the state of the progress of that patient’s visit in the ED, and thus definitionally representative of the task-load of an EP at a given point in time. Many studies have evaluated the importance of patient disposition in hospital EDs, yet research pertaining to patient handoffs in the ED scarcely mentions the importance of patient disposition in physician communication and decision-making during handoff.
Purpose Statement
Increased physician workload has previously been associated with increased EP burnout, increased staff attrition, decreased patient safety, and increased patient admission rates. Regularly occurring end of shift patient handoffs represent immediate transfers of patient caseload between EPs that may contribute to increased EP task load and mental workload. The disposition mapping for physician sign-out (DMAPS) is a patient categorization tool which indexes patients based on the state of their disposition at the time of handoff. This research examines the validity of DMAPS as an objective measure of EP workload by defining the statistical relationship between patient caseload transferred during handoffs (as categorized by DMAPS) and EPs’ perceived mental workload as measured by the NASA-TLX questionnaire. By quantifying the efficacy of DMAPS as a measure of EP workload, this research presents a novel and validated measure of EP workload designed for application during end of shift patient handoffs.
Methods
DMAPS was designed with consultation from attending EPs and was primarily based on a covariate measure of EP caseload developed by Foster et al., (2023). DMAPS is a patient categorization system designed to index the patients that are transferred during an end of shift handoff by the state of their disposition at the time of said handoff. DMAPS is comprised of 5 disposition states: Level 1 – Disposition Unknown, Level 2 – Disposition Dependent, Level 3 – Disposition Expected, Level 4 – Disposition Assigned, Level 5 – Disposition Assigned and Patient Boarding.
Patient caseload and EP workload data were collected from a large hospital system in the Southeastern United States. Participants for this study consisted of 64 attending EPs from the same large hospital system observed across 32 patient handoffs. Each handoff was observed by a researcher experienced with mapping patients using DMAPS who then categorized each patient transferred during the handoff by the state of their disposition at handoff. Both the outgoing and incoming EP of each observed end of shift handoff then completed a NASA-TLX questionnaire where they rated their anticipated remaining workload for the patients transferred during the handoffs across 5 of the 6 NASA-TLX subscales (mental demand, physical demand, temporal demand, effort, and frustration). Retrospective data were also collected based on standard measures of EP workload used in existing literature. These measures include patient volume, the proportion of high acuity patients in the handoff, and the national emergency department overcrowding score (NEDOCS).
Data were analyzed to 1) identify significant correlations between the NASA-TLX questionnaire subscales and composite dispositions based on DMAPS, and 2) the efficacy of DMAPS as a predictor of the NASA-TLX subscales in comparison to other commonly used measures of EP workload in existing literature.
Results
Results of this study identified significant negative correlations between DMAPS-based patient composites and EPs’ anticipated mental demand (r = -0.419, p = 0.001), physical demand (r = -0.371, p = 0.003), effort (r = -0.392, p = 0.001), and frustration (r = -0.318, p = 0.010). These correlations imply that as the state of a patient’s disposition decision progresses, that patient represents relatively less mental workload for the incoming EP who is responsible for the remainder of that patient’s emergency care. Furthermore, regression of the NASA-TLX questionnaire subscales by the DMAPS categorization (treating the number of patients in each category as independent variables) and select DMAPS-based composites identified the full 5-dimension DMAPS categorization as the most effective predictor of EP mental demand (r2 = 0.294), physical demand (r2 = 0.263), effort (r2 = 0.602), and frustration (r2 = 0.453) when compared against other commonly used measures of EP workload in existing literature.
Discussion
The results of this study demonstrate that DMAPS provides a more accurate and representative measure of EPs’ anticipated perceived mental workload than other commonly used measures of EP workload. DMAPS was designed specifically for evaluating the work-based workload of EPs in the ED, but also represents aspects of EPs’ mental workload as rated on the NASA TLX subscales of effort and frustration, and therefore is also an alternative to metrics such as SWAT and the NASA TLX questionnaire which may be inappropriately utilized in observational studies concerning complex tasks which lack clear timelines.
Future work based on this study should focus on further validating DMAPS as a measure of EP workload using other common measures of workload in existing literature. Further validation could be completed using time-motion studies and reliable physiological measures such as the number of steps taken by EPs during their shift. Future work should also explore the usage of DMAPS outside of the context of patient handoffs. Because DMAPS relies on researchers listening to descriptions of patients’ disposition states, DMAPS is in theory applicable as a point measure of workload at any point during an EPs shift. While this study was opportunistically conducted during end of shift patient handoffs in part because EPs discuss patient dispositions during these handoffs, future work could explore the usage of DMAPS to capture point measures of workload at any point during EPs’ shifts. Lastly, there exists little validation of the reliable usage of DMAPS by non-clinical researchers. Future research should test the accuracy and reliability of raters with diverse backgrounds and skillsets applying DMAPS to index patients in end of shift patient handoffs.
Conclusion
This research provides a novel measure of EP workload during end of shift patient handoffs that can be used to evaluate handoff and policy interventions and provide a basis for future measures of ED staff workload that can be used to address EP burnout, staff attrition, and decreased patient safety.
Event Type
Oral Presentations
TimeTuesday, April 18:52am - 9:15am EDT
LocationQueens Quay
Patient Safety and Research Initiatives (PS)