Presentation
Unlocking the Tesseract: Exploring Immersive Auditory Environments in Clinical Simulation
DescriptionKeywords: Healthcare simulation, auditory simulation, extended reality, immersive simulation, multi-sensory platform.
Introduction: Simulation-based research has proven highly effective across diverse medical applications, including procedural skills development, patient-centric care, and mitigation of medical errors. Empirical evidence indicates that simulation significantly enhances medical education by fostering clinical decision-making and technical proficiency among healthcare professionals (Okuda et al., 2009). Additionally, simulation-based training has been linked to improved patient outcomes, particularly in high-stakes scenarios such as emergency response and complex surgical procedures (Issenberg et al., 2005). By providing a controlled environment where practitioners can safely practice, evaluate, and refine their skills, simulations play a critical role in minimizing medical errors without compromising patient safety (Kohn et al., 2000). These simulations vary in fidelity and utilize distinct modalities, including physical trainer boxes (Rosenblatt et al., 2008), lower-fidelity setups without physical models (Ulrich et al., 2020), virtual reality (VR) simulators (Heinrich et al., 2021), and augmented reality (AR) platforms, to replicate realistic medical scenarios (Heinrich et al., 2021).
Significant consideration in simulation-based research is driven by the concept of fidelity, which refers to the degree of accuracy with which sensory stimuli replicate real-world experiences (Rehmann et al., 1995). In human factors research, fidelity is categorized by physical resemblance, encompassing the visual, tactile, and auditory components of a simulated environment (Alexander et al., 2005). High levels of physical fidelity have been correlated with enhanced training outcomes, as they promote better skill transfer to real-world clinical settings (Gaba, 2004). The visual and tactile elements have traditionally dominated simulation design and research. The auditory fidelity in simulation is comparatively underexplored, despite its significance in healthcare contexts. Studies indicate that auditory stimuli—such as alarms, equipment noises, and ambient sounds—can substantially affect cognitive load, situational awareness, and team communication in high-pressure environments (Momtahan et al., 1993). For instance, research on anesthesia training has shown that the presence of auditory distractions can impair task performance and elevate the risk of errors (Hedman et al., 2016). Moreover, the inclusion of realistic auditory cues has been demonstrated to improve procedural accuracy in complex scenarios, underscoring the importance of auditory fidelity as a critical but underutilized component in achieving holistic simulation realism (Weinger et al., 2000). Current research calls for a deeper investigation into integrating immersive auditory environments into simulation-based research for improving performance and operational outcomes in healthcare.
Purpose of the Study: This paper explores the potential of immersive auditory environments in healthcare simulation to study the impacts of soundscape in healthcare settings on clinicians. The central hypothesis is that realistic soundscapes are essential components in simulation-based training and research for improving clinical decision-making and performance. To facilitate this exploration, the research team will examine and test the Tesseract, a novel simulation platform capable of generating high-fidelity auditory experiences for human factors healthcare research.
The Tesseract - A Multi-Sensory Simulation Platform: The Institute for Creativity, Arts, and Technology (ICAT) at Virginia Tech and the Carilion Center for Simulation, Research, and Patient Safety developed the Tesseract, a portable, modular high density loudspeaker array for spatial sonification and auralization (Upthegrove et al., 2024). Coupled with visual displays such as three-sided large projection screens, the Tesseract can recreate complex auditory and visual environments to provide an immersive multimodal platform. Unlike traditional simulation platforms confined to fixed locations, the Tesseract’s portable nature and flexible design enable deployment in a variety of settings, making it suitable for diverse clinical applications and under-resourced facilities.
The Tesseract’s research capabilities include a suite of integrated data-collection tools. High-resolution cameras are strategically installed to capture participant movements and facial expressions, enabling detailed behavioral analyses. Multi-channel microphones embedded within the system provide comprehensive auditory monitoring, capturing not only environmental sounds but also verbal communication patterns and ambient noise levels. These tools, combined with the ability to integrate physiological sensors such as heart rate and eye-tracking devices, allow for a multifaceted assessment of cognitive and physical behaviors in simulated clinical settings.
Applications: Given these unique features, the Tesseract possesses high potential for conducting meaningful, repeatable experiments to investigate how sounds could impact clinicians and thus their performance. For example, first responders in air ambulances are always situated in intense sound/noise environments, which could have tremendous impacts on attention, workload, situation awareness and thus performance. The Tesseract system can simulate these auditory conditions for studying the impacts of the soundscape, investigating technological solutions, acclimatizing trainees to the challenges of auditory environments, training to maintain focus and execute procedures effectively.
Another potential research and training application area is coordinated team communication in chaotic settings due to natural disaster like earthquakes and wildfires, which can escalate environmental noise levels or alter the surrounding soundscape. In these settings, medical teams must perform under stress while managing auditory overload from various sound sources such as team communication, environmental noises, and distressed individuals. Immersive auditory simulations can replicate these conditions, enabling teams to devise and practice adaptive strategies in communication, task coordination, and decision-making in a controlled yet realistic environment.
Exploratory Study: To examine the potential of the Tesseract to support research and training in high fidelity auditory environment, we are designing and will be conducting an exploratory study to assess the impact of different soundscapes on clinicians to provide care in a simulated Emergency Department (ED) setting. Waiting at the ED represents a critical issue in emergency care where up to 1-2% of patients leave without being seen due to prolonged wait times and staff shortages—a situation exacerbated since the COVID-19 pandemic (Janke et al., 2022). To investigate how varying noise levels affect clinicians, the Tesseract will simulate three distinct sound conditions: a quiet baseline, a moderately busy setting, and a chaotic setting with auditory and visual distractions such as ambulance sirens, equipment alarms, and patient chatter. The recruitment will target 50 nurses, who will perform routine tasks such as checking vitals and drawing blood at simulated workstations. The study will collect heart rate variability, eye-tracking metrics, and task completion times to assess cognitive load and efficiency, while post-experiment debriefing interviews and video analysis will be analyzed to provide insights into behavioral responses and subjective experiences. The recruitment process will begin November 2024, and the data collection will be complete by January 2025.
Expected study outcomes include a significant increase in stress levels and loss of situation awareness in the chaotic condition compared to quieter conditions, as indicated by higher heart rate variability and disrupted eye movement patterns. Task completion times and error rates are expected to rise in the chaotic condition due to cognitive processing to suppress the noise and distractions. The qualitative data is expected to provide verification and description of the challenges induced by different soundscapes on clinicians, such as maintaining focus or effective communication amidst overwhelming sensory inputs. Together, these findings will advance our knowledge on how high-fidelity auditory settings can help investigate the impacts of soundscape on clinicians, their performance, and inform mitigating strategies.
Conclusion: The Tesseract is a versatile, multi-sensory simulation platform with unique high-fidelity audio reproduction capabilities that can address the lack of consideration of complex auditory conditions in simulation research and training. This study will explore the potential of the Tesseract for studying the impact of soundscapes on clinicians in high fidelity audio environments. Specifically, the study will collect a wide range of physiological, behavioral (video) and self-reported data on 50 nurses performing typical tasks in emergency waiting room area under quiet, busy and chaotic audio conditions. The findings will provide new evidence on the importance of audio fidelity in simulation research and training.
Introduction: Simulation-based research has proven highly effective across diverse medical applications, including procedural skills development, patient-centric care, and mitigation of medical errors. Empirical evidence indicates that simulation significantly enhances medical education by fostering clinical decision-making and technical proficiency among healthcare professionals (Okuda et al., 2009). Additionally, simulation-based training has been linked to improved patient outcomes, particularly in high-stakes scenarios such as emergency response and complex surgical procedures (Issenberg et al., 2005). By providing a controlled environment where practitioners can safely practice, evaluate, and refine their skills, simulations play a critical role in minimizing medical errors without compromising patient safety (Kohn et al., 2000). These simulations vary in fidelity and utilize distinct modalities, including physical trainer boxes (Rosenblatt et al., 2008), lower-fidelity setups without physical models (Ulrich et al., 2020), virtual reality (VR) simulators (Heinrich et al., 2021), and augmented reality (AR) platforms, to replicate realistic medical scenarios (Heinrich et al., 2021).
Significant consideration in simulation-based research is driven by the concept of fidelity, which refers to the degree of accuracy with which sensory stimuli replicate real-world experiences (Rehmann et al., 1995). In human factors research, fidelity is categorized by physical resemblance, encompassing the visual, tactile, and auditory components of a simulated environment (Alexander et al., 2005). High levels of physical fidelity have been correlated with enhanced training outcomes, as they promote better skill transfer to real-world clinical settings (Gaba, 2004). The visual and tactile elements have traditionally dominated simulation design and research. The auditory fidelity in simulation is comparatively underexplored, despite its significance in healthcare contexts. Studies indicate that auditory stimuli—such as alarms, equipment noises, and ambient sounds—can substantially affect cognitive load, situational awareness, and team communication in high-pressure environments (Momtahan et al., 1993). For instance, research on anesthesia training has shown that the presence of auditory distractions can impair task performance and elevate the risk of errors (Hedman et al., 2016). Moreover, the inclusion of realistic auditory cues has been demonstrated to improve procedural accuracy in complex scenarios, underscoring the importance of auditory fidelity as a critical but underutilized component in achieving holistic simulation realism (Weinger et al., 2000). Current research calls for a deeper investigation into integrating immersive auditory environments into simulation-based research for improving performance and operational outcomes in healthcare.
Purpose of the Study: This paper explores the potential of immersive auditory environments in healthcare simulation to study the impacts of soundscape in healthcare settings on clinicians. The central hypothesis is that realistic soundscapes are essential components in simulation-based training and research for improving clinical decision-making and performance. To facilitate this exploration, the research team will examine and test the Tesseract, a novel simulation platform capable of generating high-fidelity auditory experiences for human factors healthcare research.
The Tesseract - A Multi-Sensory Simulation Platform: The Institute for Creativity, Arts, and Technology (ICAT) at Virginia Tech and the Carilion Center for Simulation, Research, and Patient Safety developed the Tesseract, a portable, modular high density loudspeaker array for spatial sonification and auralization (Upthegrove et al., 2024). Coupled with visual displays such as three-sided large projection screens, the Tesseract can recreate complex auditory and visual environments to provide an immersive multimodal platform. Unlike traditional simulation platforms confined to fixed locations, the Tesseract’s portable nature and flexible design enable deployment in a variety of settings, making it suitable for diverse clinical applications and under-resourced facilities.
The Tesseract’s research capabilities include a suite of integrated data-collection tools. High-resolution cameras are strategically installed to capture participant movements and facial expressions, enabling detailed behavioral analyses. Multi-channel microphones embedded within the system provide comprehensive auditory monitoring, capturing not only environmental sounds but also verbal communication patterns and ambient noise levels. These tools, combined with the ability to integrate physiological sensors such as heart rate and eye-tracking devices, allow for a multifaceted assessment of cognitive and physical behaviors in simulated clinical settings.
Applications: Given these unique features, the Tesseract possesses high potential for conducting meaningful, repeatable experiments to investigate how sounds could impact clinicians and thus their performance. For example, first responders in air ambulances are always situated in intense sound/noise environments, which could have tremendous impacts on attention, workload, situation awareness and thus performance. The Tesseract system can simulate these auditory conditions for studying the impacts of the soundscape, investigating technological solutions, acclimatizing trainees to the challenges of auditory environments, training to maintain focus and execute procedures effectively.
Another potential research and training application area is coordinated team communication in chaotic settings due to natural disaster like earthquakes and wildfires, which can escalate environmental noise levels or alter the surrounding soundscape. In these settings, medical teams must perform under stress while managing auditory overload from various sound sources such as team communication, environmental noises, and distressed individuals. Immersive auditory simulations can replicate these conditions, enabling teams to devise and practice adaptive strategies in communication, task coordination, and decision-making in a controlled yet realistic environment.
Exploratory Study: To examine the potential of the Tesseract to support research and training in high fidelity auditory environment, we are designing and will be conducting an exploratory study to assess the impact of different soundscapes on clinicians to provide care in a simulated Emergency Department (ED) setting. Waiting at the ED represents a critical issue in emergency care where up to 1-2% of patients leave without being seen due to prolonged wait times and staff shortages—a situation exacerbated since the COVID-19 pandemic (Janke et al., 2022). To investigate how varying noise levels affect clinicians, the Tesseract will simulate three distinct sound conditions: a quiet baseline, a moderately busy setting, and a chaotic setting with auditory and visual distractions such as ambulance sirens, equipment alarms, and patient chatter. The recruitment will target 50 nurses, who will perform routine tasks such as checking vitals and drawing blood at simulated workstations. The study will collect heart rate variability, eye-tracking metrics, and task completion times to assess cognitive load and efficiency, while post-experiment debriefing interviews and video analysis will be analyzed to provide insights into behavioral responses and subjective experiences. The recruitment process will begin November 2024, and the data collection will be complete by January 2025.
Expected study outcomes include a significant increase in stress levels and loss of situation awareness in the chaotic condition compared to quieter conditions, as indicated by higher heart rate variability and disrupted eye movement patterns. Task completion times and error rates are expected to rise in the chaotic condition due to cognitive processing to suppress the noise and distractions. The qualitative data is expected to provide verification and description of the challenges induced by different soundscapes on clinicians, such as maintaining focus or effective communication amidst overwhelming sensory inputs. Together, these findings will advance our knowledge on how high-fidelity auditory settings can help investigate the impacts of soundscape on clinicians, their performance, and inform mitigating strategies.
Conclusion: The Tesseract is a versatile, multi-sensory simulation platform with unique high-fidelity audio reproduction capabilities that can address the lack of consideration of complex auditory conditions in simulation research and training. This study will explore the potential of the Tesseract for studying the impact of soundscapes on clinicians in high fidelity audio environments. Specifically, the study will collect a wide range of physiological, behavioral (video) and self-reported data on 50 nurses performing typical tasks in emergency waiting room area under quiet, busy and chaotic audio conditions. The findings will provide new evidence on the importance of audio fidelity in simulation research and training.
Event Type
Oral Presentations
TimeTuesday, April 19:37am - 10:00am EDT
LocationPier 9
Simulation and Education (SE)