Presentation
MDD5 - A Human Factors Evaluation of Intramuscular Naloxone Kits for Use in Opioid Overdose Prevention
SessionPoster Session 1
DescriptionIntroduction. The global opioid overdose crisis has had a severe impact on public health. In Canada, approximately 5,975 people died of opioid overdoses from January to September of 2023, and the number of deaths has increased each year since the Public Health Agency of Canada started national surveillance in 2016 (Government of Canada, 2024). Interventions that effectively reduce opioid overdose deaths are important to society.
Naloxone is a medication that can reverse the effects of opioids quickly and temporarily. In Alberta, pharmacies provide Naloxone kits to the public but do not offer any training on how to use the kits. Inexperienced users may have difficulty administering Naloxone intramuscularly to overdose victims—especially in high-stress situations. Correctly drawing Naloxone out of a vial and needle-stick injuries are several concerns about the usability of intramuscular Naloxone kits that were identified after an initial heuristic evaluation by human factors and nursing experts. User testing of intramuscular Naloxone kits by inexperienced bystanders was expected to identify additional issues and potential solutions.
Objectives. The focus of this study was to evaluate the usability of intramuscular Naloxone kits in Alberta. The primary research question was ‘Can untrained individuals successfully use an intramuscular Naloxone kit?’. A secondary question was ‘What are the barriers to timely, safe and effective use of intramuscular Naloxone kits?’.
Methods. This study received ethics approval (REB23-1604). Fifteen participants were recruited using the Department of Psychology research participation pool. After informed consent, participants were instructed to administer Naloxone to an orange using the Alberta Health Services (AHS) intramuscular Naloxone kit. Video (iPhone 14 camera) and audio (Zoom H5 Audio Recorder) recordings were collected while participants attempted to administer Naloxone while using a talk-aloud protocol (Ericsson & Simon, 1993). After the scenario was over, participants completed a modified System Usability Scale (SUS) (Kortum & Bangor, 2013), answered several interview questions and filled out a background demographic questionnaire.
Results. A total of 15 participants took part in the study. Participants were undergraduate students from the University of Calgary, and all were included in the data analysis. Participants ranged from 18 to 22 years old (M = 19.6, SD = 1.4), and 86.7% identified as female (n = 13), 6.7% as male (n = 1), and 6.7% as genderqueer (n = 1). Sixty percent of participants (n = 9) had little knowledge or background about the opioid overdose crisis or intervention methods to prevent opioid overdoses, with some participants having no (n = 1) or some (n = 5) knowledge.
The average time taken to administer naloxone was four minutes and 34 seconds (range: two minutes 37 seconds to six minutes 52 seconds).
Participants rated the kit with an average SUS score of 69.3 out of 100. The SUS score corresponded to an overall letter grade of B to B+, which indicates reasonably good ease of use. Participants said that they felt more comfortable administering Naloxone after completing the study than before.
Qualitative data (i.e., transcribed recordings) was coded and analyzed using thematic analysis (Braun & Clarke, 2008). Four major themes were identified: 1) the usability of kit contents, 2) the performance of tasks, 3) user sentiments, and 4) the importance of training. For example, most participants (N = 13/15) said that the Naloxone kit was difficult to use on the first try. Training or practice with the kit prior to use with a person who is overdosing would make it easier to use. All participants said that they were more comfortable administering naloxone after they completed the study.
Suggestions for improving the ease of use of these kits included: placing the instruction booklet on top of all other kit contents to make it harder to miss when opening the kit and making the cap of the vial easier to take off with a pull tab, which would make it easier to access the Naloxone.
Discussion. Several barriers to timely, safe and effective use were identified based on this descriptive study. It is crucial to administer Naloxone as quickly as possible after the onset of overdose symptoms (Kim et al., 2009). The length of time from opening the kit to administering Naloxone with a syringe (M = 4 minutes, 30 seconds) has implications for the person who is overdosing and the bystander trying to help.
If the person who is overdosing is not breathing, performing rescue breaths and preparing a Naloxone syringe may be mutually exclusive if there is only one bystander present. The scenario tested in this study, however, did not include calling 911 for an ambulance or performing rescue breathing if needed. If rescue breathing is not attempted, the length of time before Naloxone is administered may affect whether a brain injury occurs in the person who is overdosing (Casillas et al., 2024 ). Past research has found that brain cells deprived of oxygen can start to die within approximately four minutes (Raffa, et al., 2017).
In a laboratory setting, inexperienced participants were able to use the AHS intramuscular Naloxone kit. All participants successfully administered Naloxone to an orange. On a scale of 0 to 100, participants rated the usability of the intramuscular Naloxone kit a 69.3. A score above 70 typically indicates that the product has reasonably good usability (Soegaard, 2023).
Aspects of the kit that helped facilitate usability included organization and packaging of the kit and the instruction booklet. Moreover, participants liked using a safety syringe, which protects users from needlestick injuries because the needle is covered up by a sheath once it is used.
Limitations and Future Research. The generalizability of the results of this study is to educated university students. A larger sample of participants who may encounter individuals who are overdosing from the general population is needed, such as librarians, cashiers and bus drivers. Future research could focus on direct comparisons between the AHS Naloxone kit and other Naloxone administration methods such as pre-filled syringes and nasal sprays, which are likely much faster to administer. Willingness of a bystander to check vitals and administer Naloxone to a person who is semiconscious and behaving in an unusual manner is likely a barrier to some and requires additional research.
Naloxone is a medication that can reverse the effects of opioids quickly and temporarily. In Alberta, pharmacies provide Naloxone kits to the public but do not offer any training on how to use the kits. Inexperienced users may have difficulty administering Naloxone intramuscularly to overdose victims—especially in high-stress situations. Correctly drawing Naloxone out of a vial and needle-stick injuries are several concerns about the usability of intramuscular Naloxone kits that were identified after an initial heuristic evaluation by human factors and nursing experts. User testing of intramuscular Naloxone kits by inexperienced bystanders was expected to identify additional issues and potential solutions.
Objectives. The focus of this study was to evaluate the usability of intramuscular Naloxone kits in Alberta. The primary research question was ‘Can untrained individuals successfully use an intramuscular Naloxone kit?’. A secondary question was ‘What are the barriers to timely, safe and effective use of intramuscular Naloxone kits?’.
Methods. This study received ethics approval (REB23-1604). Fifteen participants were recruited using the Department of Psychology research participation pool. After informed consent, participants were instructed to administer Naloxone to an orange using the Alberta Health Services (AHS) intramuscular Naloxone kit. Video (iPhone 14 camera) and audio (Zoom H5 Audio Recorder) recordings were collected while participants attempted to administer Naloxone while using a talk-aloud protocol (Ericsson & Simon, 1993). After the scenario was over, participants completed a modified System Usability Scale (SUS) (Kortum & Bangor, 2013), answered several interview questions and filled out a background demographic questionnaire.
Results. A total of 15 participants took part in the study. Participants were undergraduate students from the University of Calgary, and all were included in the data analysis. Participants ranged from 18 to 22 years old (M = 19.6, SD = 1.4), and 86.7% identified as female (n = 13), 6.7% as male (n = 1), and 6.7% as genderqueer (n = 1). Sixty percent of participants (n = 9) had little knowledge or background about the opioid overdose crisis or intervention methods to prevent opioid overdoses, with some participants having no (n = 1) or some (n = 5) knowledge.
The average time taken to administer naloxone was four minutes and 34 seconds (range: two minutes 37 seconds to six minutes 52 seconds).
Participants rated the kit with an average SUS score of 69.3 out of 100. The SUS score corresponded to an overall letter grade of B to B+, which indicates reasonably good ease of use. Participants said that they felt more comfortable administering Naloxone after completing the study than before.
Qualitative data (i.e., transcribed recordings) was coded and analyzed using thematic analysis (Braun & Clarke, 2008). Four major themes were identified: 1) the usability of kit contents, 2) the performance of tasks, 3) user sentiments, and 4) the importance of training. For example, most participants (N = 13/15) said that the Naloxone kit was difficult to use on the first try. Training or practice with the kit prior to use with a person who is overdosing would make it easier to use. All participants said that they were more comfortable administering naloxone after they completed the study.
Suggestions for improving the ease of use of these kits included: placing the instruction booklet on top of all other kit contents to make it harder to miss when opening the kit and making the cap of the vial easier to take off with a pull tab, which would make it easier to access the Naloxone.
Discussion. Several barriers to timely, safe and effective use were identified based on this descriptive study. It is crucial to administer Naloxone as quickly as possible after the onset of overdose symptoms (Kim et al., 2009). The length of time from opening the kit to administering Naloxone with a syringe (M = 4 minutes, 30 seconds) has implications for the person who is overdosing and the bystander trying to help.
If the person who is overdosing is not breathing, performing rescue breaths and preparing a Naloxone syringe may be mutually exclusive if there is only one bystander present. The scenario tested in this study, however, did not include calling 911 for an ambulance or performing rescue breathing if needed. If rescue breathing is not attempted, the length of time before Naloxone is administered may affect whether a brain injury occurs in the person who is overdosing (Casillas et al., 2024 ). Past research has found that brain cells deprived of oxygen can start to die within approximately four minutes (Raffa, et al., 2017).
In a laboratory setting, inexperienced participants were able to use the AHS intramuscular Naloxone kit. All participants successfully administered Naloxone to an orange. On a scale of 0 to 100, participants rated the usability of the intramuscular Naloxone kit a 69.3. A score above 70 typically indicates that the product has reasonably good usability (Soegaard, 2023).
Aspects of the kit that helped facilitate usability included organization and packaging of the kit and the instruction booklet. Moreover, participants liked using a safety syringe, which protects users from needlestick injuries because the needle is covered up by a sheath once it is used.
Limitations and Future Research. The generalizability of the results of this study is to educated university students. A larger sample of participants who may encounter individuals who are overdosing from the general population is needed, such as librarians, cashiers and bus drivers. Future research could focus on direct comparisons between the AHS Naloxone kit and other Naloxone administration methods such as pre-filled syringes and nasal sprays, which are likely much faster to administer. Willingness of a bystander to check vitals and administer Naloxone to a person who is semiconscious and behaving in an unusual manner is likely a barrier to some and requires additional research.
Event Type
Poster Presentation
TimeMonday, March 314:45pm - 6:15pm EDT
LocationFrontenac Foyer
Digital Health (DH)
Simulation and Education (SE)
Hospital Environments (HE)
Medical and Drug Delivery Devices (MDD)
Patient Safety and Research Initiatives (PS)