Presentation
SE1 - Adaptable Synergistic Measurement and Debriefing Tool for Medical Device Training Using Event-Based Approach to Training (EBAT)
SessionPoster Session 1
DescriptionMedical devices are advancing rapidly, outpacing the educational provisions available to practitioners, leading to insufficient competence in safe and effective use. As a result, many professionals rely on trial-and-error learning or brief vendor-led sessions, which are often inadequate (Caruana, 2024). This situation is exacerbated by the lack of standardization in training content, limited hands-on practice, and the inability to learn at an individual pace (Grundgeiger, 2023).
To address these issues, e-learning platforms and tools are being increasingly utilized, offering flexibility and potential for more consistent training. Studies show that combining e-learning with structured hands-on activities enhances procedural skill acquisition compared to e-learning alone (Grundgeiger, 2023). This blended learning approach bridges the gap between theory and practice, enabling healthcare professionals (HCPs) to better handle medical devices in clinical settings.
Enhancing training effectiveness, particularly for mastering medical equipment and procedures, is the use of simulation-based methodologies, integrated with e-learning solutions. One approach is Event-Based Approach to Training (EBAT), which offers a structured framework for developing critical skills in clinical environments. EBAT is a simulation-based training methodology that introduces specific events with training exercises to observe and assess targeted behaviors (Fowlkes, 1998). It links training objectives, scenario design, and performance evaluation to ensure learners demonstrate required skills. EBAT in healthcare offers significant benefits by focusing on critical events, allowing for targeted skill development in high-pressure, variable environments (Fernandez, 2022). EBAT ensures HCPs are trained to respond effectively to specific situations, improving team coordination, communication, and patient-care. By practicing in realistic simulations, HCPs enhance their ability to handle diverse clinical scenarios, ultimately leading to safer, more efficient patient outcomes (Rosen, 2008).
Despite its utility, there is limited literature focusing on the EBAT application specifically for medical devices, highlighting the importance of discussing its potential benefits. Extending this approach to medical device training has not been explored in detail, but its foundational principles suggest it could offer advantages in this domain. Creating training exercises where events are purposefully designed to trigger specific responses, EBAT ensures that learners demonstrate skills in a highly relevant and context-specific manner.
The variability in patient conditions and device functionality makes it difficult to evaluate whether HCPs are prepared for potential challenges. HCPs often operate in unpredictable environments, especially when using technologies such as continuous monitoring devices for anesthesia or portable sensors for drug delivery (Aiassa, 2021). EBAT could provide a framework to introduce a wide range of clinical scenarios, allowing learners to encounter both routine and high-risk situations they might face in real clinical settings. EBAT’s focus on explicit links between training objectives and performance assessments allows for systematic measurement of competencies. Ensuring learners are evaluated not just on their knowledge but also on their ability to apply that knowledge during real-world, device-related tasks. Theoretically, EBAT should be an effective approach for medical device training due to its ability to simulate real-world complexity while maintaining control over the training environment. Allowing for the replication of rare but critical event scenarios within a controlled, measurable context. Effective training is followed by a debriefing session between the trainer and learner (Keiser, 2021). Accelerating the debriefing process allows feedback and reflection to be delivered promptly which is essential for reinforcing learning and improving performance. A streamlined debrief ensures that more time is allocated to active training and skill development, rather than extended reflection, increasing the overall efficiency of training programs (Keiser, 2021). As medical devices increase complexity, integrating EBAT into training programs could enhance the preparedness of HCPs, ultimately improving patient safety and care outcomes.
Upon these insights, a fully customizable, synergistic event-based measurement tool was developed to evaluate the training of routine-use and complex medical devices. This tool addresses gaps identified in current training approaches by allowing tailored assessments that align with the specific competencies required for different devices.
According to Fowlkes (1998), there are seven components EBAT must follow to ensure effective training:
1. Training Objectives: Clearly defined goals or competency training aims to develop, specific skills or behaviors.
2. Scenario Design: Structured training exercises that systematically incorporate events, designed to create opportunities to demonstrate mastery of the training objectives.
3. Events: Specific, embedded "trigger" events within scenarios that require learners to act, enabling the observation of behaviors related to the training objectives.
4. Observation and Feedback: A system for observing learner’s responses to events, allowing trainers to provide feedback with pre-defined, acceptable responses to each event, which trainers use to assess performance in real-time.
5. Independence of Events: Events are structured to minimize dependencies, ensuring that one event's outcome does not influence another.
6. Scenario Control: Scripted control of task conditions to maintain consistent standardization across training sessions, ensuring all intended events are presented as planned.
7. Performance Measurement Systems: Tools that help systematically record and evaluate learner performance based on their responses to the events.
The tool aims to combine the first six components into the 7th (Performance Measurement Systems). Due to formatting constraints, the actual tool will only be presented at the conference and only textually described in this submission.
There are two parts of the tool, the measurement tool itself, and the debriefing tool. Both were created using Microsoft Excel and are adjacent sheets to each other and have interconnected elements. On the measurement sheet there are nine columns:
- Task ID, a specific set of numbers/letters to identify the specific tasks
- Task, a brief task description
- Trainer’s “Script”, which helps the trainer follow the training procedures
- The response of a paid actor (or mannequin, whatever is necessary to complete training), which is added to this tool to keep the trainer on track
- What Triggers happen to warrant a response, the action that occur to move the training forward
- Targeted Response from Learner, what should the learner accomplish as a result of the trigger
- Reached Response, which is left blank but with the options to respond with “Yes”, “No”, “Somewhat”, and “Not Applicable”
- Comments, which are also left blank
- Response ID, that directly corresponds with the Targeted Responses and Reached Responses
The debriefing portion of the tool has five columns, with three boxes on the side. The first column combines a brief description of the task and the task ID, the second displays the Response ID. The next two columns are mimicked directly from the measurement sheet and display the response status and comments made during the training session. As the trainer fills out the training sheet according to the learner’s performance the debriefing sheet auto-populates the responses and comments. When the training is over, the debriefing session can start right away and not require extra time to fill out. The last column is left blank for any comments the trainer wants to add during debriefing. Two of the boxes added on the side are for additional comments as well, one for anything else the learner’s would like to mention, and one for any further comments the trainer would like to record about the training session. The last box is the Response Status, this counts the “Yes”, “No”, and “Somewhat” responses to automatically score the learner as soon as the training is over.
The proposed tool auto-populates debriefing sheets and automatically scores learner performance by reducing the manual effort involved in recording data and offering real-time feedback, this tool facilitates faster and more consistent evaluations. Automation allows trainers to focus on addressing key learning points and improving outcomes. The sooner the debrief occurs post-training, the more information will be retained by all participating parties. This tool streamlines the training process, enhances each stage of training, and optimizes debriefs ensuring opportunity for knowledge gain.
To address these issues, e-learning platforms and tools are being increasingly utilized, offering flexibility and potential for more consistent training. Studies show that combining e-learning with structured hands-on activities enhances procedural skill acquisition compared to e-learning alone (Grundgeiger, 2023). This blended learning approach bridges the gap between theory and practice, enabling healthcare professionals (HCPs) to better handle medical devices in clinical settings.
Enhancing training effectiveness, particularly for mastering medical equipment and procedures, is the use of simulation-based methodologies, integrated with e-learning solutions. One approach is Event-Based Approach to Training (EBAT), which offers a structured framework for developing critical skills in clinical environments. EBAT is a simulation-based training methodology that introduces specific events with training exercises to observe and assess targeted behaviors (Fowlkes, 1998). It links training objectives, scenario design, and performance evaluation to ensure learners demonstrate required skills. EBAT in healthcare offers significant benefits by focusing on critical events, allowing for targeted skill development in high-pressure, variable environments (Fernandez, 2022). EBAT ensures HCPs are trained to respond effectively to specific situations, improving team coordination, communication, and patient-care. By practicing in realistic simulations, HCPs enhance their ability to handle diverse clinical scenarios, ultimately leading to safer, more efficient patient outcomes (Rosen, 2008).
Despite its utility, there is limited literature focusing on the EBAT application specifically for medical devices, highlighting the importance of discussing its potential benefits. Extending this approach to medical device training has not been explored in detail, but its foundational principles suggest it could offer advantages in this domain. Creating training exercises where events are purposefully designed to trigger specific responses, EBAT ensures that learners demonstrate skills in a highly relevant and context-specific manner.
The variability in patient conditions and device functionality makes it difficult to evaluate whether HCPs are prepared for potential challenges. HCPs often operate in unpredictable environments, especially when using technologies such as continuous monitoring devices for anesthesia or portable sensors for drug delivery (Aiassa, 2021). EBAT could provide a framework to introduce a wide range of clinical scenarios, allowing learners to encounter both routine and high-risk situations they might face in real clinical settings. EBAT’s focus on explicit links between training objectives and performance assessments allows for systematic measurement of competencies. Ensuring learners are evaluated not just on their knowledge but also on their ability to apply that knowledge during real-world, device-related tasks. Theoretically, EBAT should be an effective approach for medical device training due to its ability to simulate real-world complexity while maintaining control over the training environment. Allowing for the replication of rare but critical event scenarios within a controlled, measurable context. Effective training is followed by a debriefing session between the trainer and learner (Keiser, 2021). Accelerating the debriefing process allows feedback and reflection to be delivered promptly which is essential for reinforcing learning and improving performance. A streamlined debrief ensures that more time is allocated to active training and skill development, rather than extended reflection, increasing the overall efficiency of training programs (Keiser, 2021). As medical devices increase complexity, integrating EBAT into training programs could enhance the preparedness of HCPs, ultimately improving patient safety and care outcomes.
Upon these insights, a fully customizable, synergistic event-based measurement tool was developed to evaluate the training of routine-use and complex medical devices. This tool addresses gaps identified in current training approaches by allowing tailored assessments that align with the specific competencies required for different devices.
According to Fowlkes (1998), there are seven components EBAT must follow to ensure effective training:
1. Training Objectives: Clearly defined goals or competency training aims to develop, specific skills or behaviors.
2. Scenario Design: Structured training exercises that systematically incorporate events, designed to create opportunities to demonstrate mastery of the training objectives.
3. Events: Specific, embedded "trigger" events within scenarios that require learners to act, enabling the observation of behaviors related to the training objectives.
4. Observation and Feedback: A system for observing learner’s responses to events, allowing trainers to provide feedback with pre-defined, acceptable responses to each event, which trainers use to assess performance in real-time.
5. Independence of Events: Events are structured to minimize dependencies, ensuring that one event's outcome does not influence another.
6. Scenario Control: Scripted control of task conditions to maintain consistent standardization across training sessions, ensuring all intended events are presented as planned.
7. Performance Measurement Systems: Tools that help systematically record and evaluate learner performance based on their responses to the events.
The tool aims to combine the first six components into the 7th (Performance Measurement Systems). Due to formatting constraints, the actual tool will only be presented at the conference and only textually described in this submission.
There are two parts of the tool, the measurement tool itself, and the debriefing tool. Both were created using Microsoft Excel and are adjacent sheets to each other and have interconnected elements. On the measurement sheet there are nine columns:
- Task ID, a specific set of numbers/letters to identify the specific tasks
- Task, a brief task description
- Trainer’s “Script”, which helps the trainer follow the training procedures
- The response of a paid actor (or mannequin, whatever is necessary to complete training), which is added to this tool to keep the trainer on track
- What Triggers happen to warrant a response, the action that occur to move the training forward
- Targeted Response from Learner, what should the learner accomplish as a result of the trigger
- Reached Response, which is left blank but with the options to respond with “Yes”, “No”, “Somewhat”, and “Not Applicable”
- Comments, which are also left blank
- Response ID, that directly corresponds with the Targeted Responses and Reached Responses
The debriefing portion of the tool has five columns, with three boxes on the side. The first column combines a brief description of the task and the task ID, the second displays the Response ID. The next two columns are mimicked directly from the measurement sheet and display the response status and comments made during the training session. As the trainer fills out the training sheet according to the learner’s performance the debriefing sheet auto-populates the responses and comments. When the training is over, the debriefing session can start right away and not require extra time to fill out. The last column is left blank for any comments the trainer wants to add during debriefing. Two of the boxes added on the side are for additional comments as well, one for anything else the learner’s would like to mention, and one for any further comments the trainer would like to record about the training session. The last box is the Response Status, this counts the “Yes”, “No”, and “Somewhat” responses to automatically score the learner as soon as the training is over.
The proposed tool auto-populates debriefing sheets and automatically scores learner performance by reducing the manual effort involved in recording data and offering real-time feedback, this tool facilitates faster and more consistent evaluations. Automation allows trainers to focus on addressing key learning points and improving outcomes. The sooner the debrief occurs post-training, the more information will be retained by all participating parties. This tool streamlines the training process, enhances each stage of training, and optimizes debriefs ensuring opportunity for knowledge gain.
Event Type
Poster Presentation
TimeMonday, March 314:45pm - 6:15pm EDT
LocationFrontenac Foyer
Digital Health (DH)
Simulation and Education (SE)
Hospital Environments (HE)
Medical and Drug Delivery Devices (MDD)
Patient Safety and Research Initiatives (PS)