Presentation
Streamlined Creation of EHR-Based Simulation to Evaluate Cognitive Processes: Application of a Novel Sequence of Methods to Elicit Clinician Decision-Making and Information Processing
DescriptionModern E-health services cannot be designed without considering interaction with the EHR. Decision support, remote patient monitoring, resource planning, care coordination, and clinical communication are only a few examples of the many subdomains where informatics tools are seeing rapid development in large numbers. All of these interface with clinicians and the EHR at some point, and all are developed with the targets of good adoption and usability, qualities which in turn reduce errors and help these tools contribute to improved care. Adoption, usability and other positive qualities flow from building processes aligned with the principles of human centered design (HCD). HCD science holds that the socio-technical context of the design problem, in this case, clinician interaction with the HR, must be understood before auguring into solutions. This proof-of-concept study suggests a powerful new method for human centered design in clinical contexts.
Electronic health record (EHR) based simulation, in which participants engage with a simulated copy of their native EHR, is a method with which to evaluate clinical decision-making. This may seem like a mere extension of EHR-based simulation for training and education, with which most in the clinical space are familiar, but make no mistake, EHR-based simulation for research is profoundly distinct. While it lacks volume, the existing literature on the EHR-based simulation for research points its success as a method. EHR-based simulation can augment other observational, interview, and chart review methods in order to create a holistic understanding of clinical decision-making processes.
Research and patient safety teams eager to use it, however, face a challenge. For guidance, practitioners must resort to adapting lessons from the literature describing applications of EHR-based simulation for training and education, which is robust. Its volume and the ubiquity of its practice make blueprints for the design of education/training focused EHR-based simulations readily available. There are no such blueprints to use the method for research of usability and the study of patient safety.
We created an approach for design of EHR simulation for elicitation of usability and patient safety-related data. Our aim was to develop a standard sequence of methods for their development. We provide this standard and a report of that method’s use. Our EHR simulation acted as the basis for a task analysis to evaluate clinician information processing in the outpatient setting. Our approach was sufficiently structured to constitute a repeatable system, scalable for broad use within our institution’s research and quality and safety ecosystem or outside of it by any human factors professional focused on clinical contexts. We also developed strategies for evaluation of EHR-simulation performance, including analysis of audit-log data and structured observation.
This approach will be the core of our presentation, illustrated by our example of its use. Our case, characterizing information processing and decision-making in the outpatient diagnostic journey, will walk our audience through the process of building a task analysis with a practical example. They will see theory mapped to practice and be able to apply our system for task analysis or other data elicitation in e-health design.
We will present study phases completed at time of this submission. A description follows. Initial observational and interview data informed simulation development. We then moved from quotes and coded excerpts from observational data to a set of themes and issues that would benefit from further elucidation in EHR-based simulation. Specifically, observational data were coded using domains of SEIPS framework. Subject matter experts (SMEs) in primary care pediatrics, human factors engineering, and diagnostic processes evaluated codes pertinent to tools/technology and barriers to the diagnostic journey. SMEs performed simulation-appropriate evaluation of the topics and issues to proceed into scenario development. Topics/issues were then prioritized using a matrix evaluation designed from the domain of complexity factor analysis. This ensured appropriate complexity in the simulated scenarios.
We will also present study phases on pace for completion well in advance of conference proceedings. Essential topics/issues from the complexity analysis will have been integrated into scenarios, upon which we will have iterated. We will have established a script and protocol, identified evaluation metrics, and set up data capture for qualitative data and quantitative data, the latter, notably, at log level in the EHR. With respect to log level data capture setup, we will present this from an organizational and technical perspective. The audience will learn stakeholders and departments within the organization, highlighting roles at their own institutions that compose the required team to execute this method. Following the completion of all simulation materials, we will have finalized one more instrument and put it to SMEs, validating the complete simulation for the purpose of eliciting the appropriate data. Lastly, we will have run our simulation and analyzed outcomes from our task analysis. This will conclude our case.
Our EHR based simulation development system offers a well-marked path for research. Referring to our system, researchers or quality improvement officers can perform effective task analyses quickly, with internal validity, and where conclusions are supported with both qualitative and quantitative data. Outside of task analysis, our system can be used for any kind of knowledge elicitation. In short, these methods will streamline development of EHR-based simulations for research and evaluating collected data.
Electronic health record (EHR) based simulation, in which participants engage with a simulated copy of their native EHR, is a method with which to evaluate clinical decision-making. This may seem like a mere extension of EHR-based simulation for training and education, with which most in the clinical space are familiar, but make no mistake, EHR-based simulation for research is profoundly distinct. While it lacks volume, the existing literature on the EHR-based simulation for research points its success as a method. EHR-based simulation can augment other observational, interview, and chart review methods in order to create a holistic understanding of clinical decision-making processes.
Research and patient safety teams eager to use it, however, face a challenge. For guidance, practitioners must resort to adapting lessons from the literature describing applications of EHR-based simulation for training and education, which is robust. Its volume and the ubiquity of its practice make blueprints for the design of education/training focused EHR-based simulations readily available. There are no such blueprints to use the method for research of usability and the study of patient safety.
We created an approach for design of EHR simulation for elicitation of usability and patient safety-related data. Our aim was to develop a standard sequence of methods for their development. We provide this standard and a report of that method’s use. Our EHR simulation acted as the basis for a task analysis to evaluate clinician information processing in the outpatient setting. Our approach was sufficiently structured to constitute a repeatable system, scalable for broad use within our institution’s research and quality and safety ecosystem or outside of it by any human factors professional focused on clinical contexts. We also developed strategies for evaluation of EHR-simulation performance, including analysis of audit-log data and structured observation.
This approach will be the core of our presentation, illustrated by our example of its use. Our case, characterizing information processing and decision-making in the outpatient diagnostic journey, will walk our audience through the process of building a task analysis with a practical example. They will see theory mapped to practice and be able to apply our system for task analysis or other data elicitation in e-health design.
We will present study phases completed at time of this submission. A description follows. Initial observational and interview data informed simulation development. We then moved from quotes and coded excerpts from observational data to a set of themes and issues that would benefit from further elucidation in EHR-based simulation. Specifically, observational data were coded using domains of SEIPS framework. Subject matter experts (SMEs) in primary care pediatrics, human factors engineering, and diagnostic processes evaluated codes pertinent to tools/technology and barriers to the diagnostic journey. SMEs performed simulation-appropriate evaluation of the topics and issues to proceed into scenario development. Topics/issues were then prioritized using a matrix evaluation designed from the domain of complexity factor analysis. This ensured appropriate complexity in the simulated scenarios.
We will also present study phases on pace for completion well in advance of conference proceedings. Essential topics/issues from the complexity analysis will have been integrated into scenarios, upon which we will have iterated. We will have established a script and protocol, identified evaluation metrics, and set up data capture for qualitative data and quantitative data, the latter, notably, at log level in the EHR. With respect to log level data capture setup, we will present this from an organizational and technical perspective. The audience will learn stakeholders and departments within the organization, highlighting roles at their own institutions that compose the required team to execute this method. Following the completion of all simulation materials, we will have finalized one more instrument and put it to SMEs, validating the complete simulation for the purpose of eliciting the appropriate data. Lastly, we will have run our simulation and analyzed outcomes from our task analysis. This will conclude our case.
Our EHR based simulation development system offers a well-marked path for research. Referring to our system, researchers or quality improvement officers can perform effective task analyses quickly, with internal validity, and where conclusions are supported with both qualitative and quantitative data. Outside of task analysis, our system can be used for any kind of knowledge elicitation. In short, these methods will streamline development of EHR-based simulations for research and evaluating collected data.
Authors
Event Type
Oral Presentations
TimeTuesday, April 13:30pm - 3:50pm EDT
LocationPier 9
Simulation and Education (SE)