Presentation
MDD6 - Automating Autoinjector Hold Time Measurements for Improved Precision and Accuracy
SessionPoster Session 2
DescriptionAccurately measuring injection hold times of autoinjectors in human factors (HF) usability studies is critical for determining the severity of an observed use problem where intended users failing to deliver a full dose of medication during administration. With our partners, we have received feedback requesting timing data from a number of pre-submission advice letters for autoinjector products. Specifically, the requests have pertained to data on the time required for the device to complete drug delivery and on the time that participants held the device at the injection site.
Manual timing using a stopwatch is an inaccurate measure of autoinjector injections. Based on the results of study using manual timing of swimming competitions, human reaction time for an anticipated visual cue was shown to average 0.11 seconds with a standard deviation of 0.07 seconds. For an event involving the manual start and manual stop, the timing error was shown to have a standard deviation of 0.10 seconds (Faux, 2019). In another study timing participants reading a piece of text, showed a mean timing difference of 0.10 seconds between the manual measurements made by an observer and the automatic measurements made by a computer software. Additionally, this study showed that the human timer with the least experience had the highest mean recording time, standard deviation of their recordings, and maximum deviations, indicating the importance of experience and the variability of measurements between observers (Radner, 2016).
In addition to the limitation of human reaction time when starting and stopping the stopwatch, the visual stimuli for when an injection begins and ends is unclear. The needle in an autoinjector pierces the injection site before the needle guard is fully retracted and as such, there is no visual method that a moderator can use to know with absolute certainty when the needle has entered and exited the injection site. Further, for a given device, a moderator is aware of the typical injection time which may cause delays in reaction when unexpected actions, such as an early liftoff, are taken by a participant. Not only is there is timing error introduced due to an individual moderator’s reaction time, but there is also variance in timing due to studies typically involving multiple moderators, each of whom will have unique means and standard deviations of the measurements that they record due to individual differences.
Design Science has developed a system to measure when a device’s needle pierces the surface of an injection pad and report on audible feedback (e.g., “clicks”) and needle entry to removal times. This system eliminates timing inaccuracies due to human error, the limitations of human reaction time, and uncertainty over when an injection begins and ends. The aim of this research is to provide a comparison between the capabilities of this system and the current industry standard of hand-timing injections using a stopwatch either live or during video replay.
Autoinjectors of various brands, medications, and/or dosage were used in this study. For each, an injection was administered into an injection pad captured via video and injection time data was automatically determined using Design Science’s sensor system. Scenarios were used to simulate the commonly observed methods of delivery commonly seen in usability studies (e.g., wet injection, immediate liftoff after second click, slowly/quickly lifting autoinjector.) Experienced Design Science engineers were asked to watch the recording of the injection scenarios and manually time the hold duration of the injections using a stopwatch.
This data was analyzed to compare manual and automatically timed values for each injection. The difference between manually and automatically timed values for both the click times and the needle entry to removal times. The standard deviation of manually timed values from their respective automatically timed values for both time windows is recorded for all trials overall, for trials specific to each autoinjector, and for trials specific to each moderator.
Analysis is ongoing, but early indications support that manual timing is significantly less accurate than mechanical timing. The application of automatic timing to autoinjector injections will provide greater accuracy in measuring injections times. This will allow for analysis of the proportion of a dose that a user successfully administers and will allow deeper assessment of user adherence to the IFU. By synthesizing quantitative injection time data with usability data, HF engineers will be better able to assess a participant’s intentions in relation to their actions.
References
Faux, D. A., & Godolphin, J. (2019). Manual timing in physics experiments: Error and uncertainty. American Journal of Physics, 87(2), 110–115. https://doi.org/10.1119/1.5085437
Radner, W., Diendorfer, G., Kainrath, B., & Kollmitzer, C. (2016). The accuracy of reading speed measurement by stopwatch versus measurement with an automated computer program (rad‐rd©). Acta Ophthalmologica, 95(2), 211–216. https://doi.org/10.1111/aos.13201
Manual timing using a stopwatch is an inaccurate measure of autoinjector injections. Based on the results of study using manual timing of swimming competitions, human reaction time for an anticipated visual cue was shown to average 0.11 seconds with a standard deviation of 0.07 seconds. For an event involving the manual start and manual stop, the timing error was shown to have a standard deviation of 0.10 seconds (Faux, 2019). In another study timing participants reading a piece of text, showed a mean timing difference of 0.10 seconds between the manual measurements made by an observer and the automatic measurements made by a computer software. Additionally, this study showed that the human timer with the least experience had the highest mean recording time, standard deviation of their recordings, and maximum deviations, indicating the importance of experience and the variability of measurements between observers (Radner, 2016).
In addition to the limitation of human reaction time when starting and stopping the stopwatch, the visual stimuli for when an injection begins and ends is unclear. The needle in an autoinjector pierces the injection site before the needle guard is fully retracted and as such, there is no visual method that a moderator can use to know with absolute certainty when the needle has entered and exited the injection site. Further, for a given device, a moderator is aware of the typical injection time which may cause delays in reaction when unexpected actions, such as an early liftoff, are taken by a participant. Not only is there is timing error introduced due to an individual moderator’s reaction time, but there is also variance in timing due to studies typically involving multiple moderators, each of whom will have unique means and standard deviations of the measurements that they record due to individual differences.
Design Science has developed a system to measure when a device’s needle pierces the surface of an injection pad and report on audible feedback (e.g., “clicks”) and needle entry to removal times. This system eliminates timing inaccuracies due to human error, the limitations of human reaction time, and uncertainty over when an injection begins and ends. The aim of this research is to provide a comparison between the capabilities of this system and the current industry standard of hand-timing injections using a stopwatch either live or during video replay.
Autoinjectors of various brands, medications, and/or dosage were used in this study. For each, an injection was administered into an injection pad captured via video and injection time data was automatically determined using Design Science’s sensor system. Scenarios were used to simulate the commonly observed methods of delivery commonly seen in usability studies (e.g., wet injection, immediate liftoff after second click, slowly/quickly lifting autoinjector.) Experienced Design Science engineers were asked to watch the recording of the injection scenarios and manually time the hold duration of the injections using a stopwatch.
This data was analyzed to compare manual and automatically timed values for each injection. The difference between manually and automatically timed values for both the click times and the needle entry to removal times. The standard deviation of manually timed values from their respective automatically timed values for both time windows is recorded for all trials overall, for trials specific to each autoinjector, and for trials specific to each moderator.
Analysis is ongoing, but early indications support that manual timing is significantly less accurate than mechanical timing. The application of automatic timing to autoinjector injections will provide greater accuracy in measuring injections times. This will allow for analysis of the proportion of a dose that a user successfully administers and will allow deeper assessment of user adherence to the IFU. By synthesizing quantitative injection time data with usability data, HF engineers will be better able to assess a participant’s intentions in relation to their actions.
References
Faux, D. A., & Godolphin, J. (2019). Manual timing in physics experiments: Error and uncertainty. American Journal of Physics, 87(2), 110–115. https://doi.org/10.1119/1.5085437
Radner, W., Diendorfer, G., Kainrath, B., & Kollmitzer, C. (2016). The accuracy of reading speed measurement by stopwatch versus measurement with an automated computer program (rad‐rd©). Acta Ophthalmologica, 95(2), 211–216. https://doi.org/10.1111/aos.13201
Event Type
Poster Presentation
TimeTuesday, April 14:45pm - 6:15pm EDT
LocationFrontenac Foyer