Presentation
MDD19 - Identifying Usability Challenges in COVID-19 Home Test Kits: A Comprehensive User Experience Survey of 1,045 South Korean Participants
SessionPoster Session 2
DescriptionThis study provides a detailed evaluation of the user experience of COVID-19 home test kits in South Korea from a user experience survey involving 1,045 participants. The survey assessed three core usability metrics: effectiveness, efficiency, and satisfaction through questionnaire categorized as ease of use, compliance to IFU, use error experience, and SUS (Systemic Usability Scale). The primary aim of this research is to acquire the idea of how ordinary users interact with COVID-19 home test kits and identify usability challenges. From there we attempted to inform improvements in user-centered design, and ultimately enhance public health outcomes by reducing use errors and better understanding user experiences across diverse demographics.
The user experience survey, conducted using online platform, collected data from varied sample of participants, encompassing different age groups (20s to 60s), education levels, and prior experiences with home test kits. This diversity facilitated a understanding of interaction patterns among different user demographics and the challenges they faced. Key findings indicated that while general user satisfaction on the test kit was high, significant issues persisted regarding both the effectiveness and efficiency of the kits.
For the results, all the participants had prior experience using COVID-19 home test kits, and the frequency and recency of use varied. For instance, 57.8% had used the kits 2 to 5 times, and 26.8% had used them within the past 1 to 3 months. The survey results highlighted numerous instances of non-compliance with instructions for use (IFU). A significant proportion of participants admitted to not checking the expiration date on the kit (49%), not using the designated stand hole for holding the solution tube (62.6%), and incorrectly collecting samples from their throat instead of their nostrils (24.3%), which constitutes a serious use error. Additionally, 22.0% of participants did not adhere to recommended hygiene practices, and 21.9% improperly disposed of test components without proper wrapping.
The data also revealed significant associations between demographic factors such as age and gender and responses to questions under instruction compliance. Younger participants demonstrated higher compliance with instructions regarding swab insertion and proper waste disposal compared to older participants. Educational attainment was also significantly linked to instruction compliance, with participants holding higher educational degree more likely to correctly interpret test results (p < .0001). Participants who had used the test kit more frequently better understood how to hold the solution tube after removing its seal (p = .0008). Additionally, those who had used the test kit more recently were more likely to incorrectly collect throat samples (p < .0001). The high rates of non-compliance with critical steps and the significant associations between demographic and experience factors and instruction compliance suggest that there are significant barriers to proper use of the test kit.
The survey results also identified several common use errors encountered during the use of COVID-19 home test kits. The most frequently reported errors included collecting the sample from only one nostril (19.3%), spilling the solution when removing its seal (17.0%), and dropping or touching the swab head (14.5%). Notably, 53.0% of participants reported making no mistakes during the testing process. Analysis of the associations between demographic factors and use error experience indicated that individuals with lower educational attainment were more prone to damaging kit components (p = 0.001), and those with fewer prior uses of the kit were more likely to encounter various errors, including incorrect swab handling and improper sample collection.
The number of experiences with the kit was significantly associated with use errors, such as spilling the solution when removing the seal (p < 0.001), dropping or touching the swab (p = 0.0065), and improper sample collection from one nostril (p = 0.0045). Participants who had used the kit more frequently reported fewer errors, highlighting the role of familiarity in reducing errors.
Participants rated the ease of use for each task and the overall experience using a 5-point Likert scale, and the System Usability Scale (SUS) score was calculated as a percentage. The overall ease of use score was 3.78 (SD = 0.58), and the SUS score was 74.37% (SD = 13.33), reflecting moderate usability. The survey revealed that most tasks had mean scores above 3.66, except for reading the IFU (due to font size) and inserting the swab into the nostrils (3.00 and 3.08, respectively). Differences in ease of use were significant across age groups (p = 0.0026), education levels (p = 0.034), and occupation groups (p = 0.047), with older participants and those with lower education levels generally finding the test kits more challenging to use.
Male participants reported significantly higher SUS scores compared to female participants (75.41 vs. 73.11, p = 0.0049), and the number of prior experiences with the test kits significantly influenced both ease of use and SUS scores. Participants with more experience reported higher ease of use and SUS scores, suggesting that familiarity with the kits was a critical factor in determining overall usability.
The user experience survey involving 1,045 participants in South Korea provides insights into the usability challenges associated with COVID-19 home test kits. Addressing usability issues—such as improving instruction clarity, reducing component complexity, and ensuring ergonomic design—can significantly improve user compliance and reduce use errors. By tailoring the kits to accommodate users with varying levels of education and health literacy, manufacturers can enhance accessibility and effectiveness during health crises.
The associations between demographic characteristics and user performance highlight the need for targeted educational campaigns. Public health authorities can design training and support for those prone to errors, such as older adults or individuals with limited experience using these devices. Given that a majority of the population uses such devices during international disasters like pandemics, it is important to emphasize the need for better usability. The moderate System Usability Scale (SUS) scores also underscore the necessity for iterative testing and refinement, incorporating user feedback to ensure the final product meets user needs. These improvements are essential for individual safety, the reliability of results, and promoting the adoption of home testing as part of public health strategies.
To enhance the usability of self-test kits, manufacturers should focus on developing clear and easily understandable instructions, designing components with prominent visual cues, and conducting iterative usability testing. Clear instructions should feature quick guides with simple language appropriate for all education levels, with important notes and cautions presented before tasks to ensure user awareness. Illustrations should be listed in use order, easy to interpret regardless of age or experience. Small components optimized with visual cues can make their functions immediately apparent, minimizing the need to constantly refer to instructions, risk of losing, and enhancing overall satisfaction. By prioritizing these improvements, self-test kits can become more accessible and user-friendly, thereby enhancing individual safety, ensuring the reliability of results, and supporting public health strategies during pandemics and other international disasters.
The user experience survey, conducted using online platform, collected data from varied sample of participants, encompassing different age groups (20s to 60s), education levels, and prior experiences with home test kits. This diversity facilitated a understanding of interaction patterns among different user demographics and the challenges they faced. Key findings indicated that while general user satisfaction on the test kit was high, significant issues persisted regarding both the effectiveness and efficiency of the kits.
For the results, all the participants had prior experience using COVID-19 home test kits, and the frequency and recency of use varied. For instance, 57.8% had used the kits 2 to 5 times, and 26.8% had used them within the past 1 to 3 months. The survey results highlighted numerous instances of non-compliance with instructions for use (IFU). A significant proportion of participants admitted to not checking the expiration date on the kit (49%), not using the designated stand hole for holding the solution tube (62.6%), and incorrectly collecting samples from their throat instead of their nostrils (24.3%), which constitutes a serious use error. Additionally, 22.0% of participants did not adhere to recommended hygiene practices, and 21.9% improperly disposed of test components without proper wrapping.
The data also revealed significant associations between demographic factors such as age and gender and responses to questions under instruction compliance. Younger participants demonstrated higher compliance with instructions regarding swab insertion and proper waste disposal compared to older participants. Educational attainment was also significantly linked to instruction compliance, with participants holding higher educational degree more likely to correctly interpret test results (p < .0001). Participants who had used the test kit more frequently better understood how to hold the solution tube after removing its seal (p = .0008). Additionally, those who had used the test kit more recently were more likely to incorrectly collect throat samples (p < .0001). The high rates of non-compliance with critical steps and the significant associations between demographic and experience factors and instruction compliance suggest that there are significant barriers to proper use of the test kit.
The survey results also identified several common use errors encountered during the use of COVID-19 home test kits. The most frequently reported errors included collecting the sample from only one nostril (19.3%), spilling the solution when removing its seal (17.0%), and dropping or touching the swab head (14.5%). Notably, 53.0% of participants reported making no mistakes during the testing process. Analysis of the associations between demographic factors and use error experience indicated that individuals with lower educational attainment were more prone to damaging kit components (p = 0.001), and those with fewer prior uses of the kit were more likely to encounter various errors, including incorrect swab handling and improper sample collection.
The number of experiences with the kit was significantly associated with use errors, such as spilling the solution when removing the seal (p < 0.001), dropping or touching the swab (p = 0.0065), and improper sample collection from one nostril (p = 0.0045). Participants who had used the kit more frequently reported fewer errors, highlighting the role of familiarity in reducing errors.
Participants rated the ease of use for each task and the overall experience using a 5-point Likert scale, and the System Usability Scale (SUS) score was calculated as a percentage. The overall ease of use score was 3.78 (SD = 0.58), and the SUS score was 74.37% (SD = 13.33), reflecting moderate usability. The survey revealed that most tasks had mean scores above 3.66, except for reading the IFU (due to font size) and inserting the swab into the nostrils (3.00 and 3.08, respectively). Differences in ease of use were significant across age groups (p = 0.0026), education levels (p = 0.034), and occupation groups (p = 0.047), with older participants and those with lower education levels generally finding the test kits more challenging to use.
Male participants reported significantly higher SUS scores compared to female participants (75.41 vs. 73.11, p = 0.0049), and the number of prior experiences with the test kits significantly influenced both ease of use and SUS scores. Participants with more experience reported higher ease of use and SUS scores, suggesting that familiarity with the kits was a critical factor in determining overall usability.
The user experience survey involving 1,045 participants in South Korea provides insights into the usability challenges associated with COVID-19 home test kits. Addressing usability issues—such as improving instruction clarity, reducing component complexity, and ensuring ergonomic design—can significantly improve user compliance and reduce use errors. By tailoring the kits to accommodate users with varying levels of education and health literacy, manufacturers can enhance accessibility and effectiveness during health crises.
The associations between demographic characteristics and user performance highlight the need for targeted educational campaigns. Public health authorities can design training and support for those prone to errors, such as older adults or individuals with limited experience using these devices. Given that a majority of the population uses such devices during international disasters like pandemics, it is important to emphasize the need for better usability. The moderate System Usability Scale (SUS) scores also underscore the necessity for iterative testing and refinement, incorporating user feedback to ensure the final product meets user needs. These improvements are essential for individual safety, the reliability of results, and promoting the adoption of home testing as part of public health strategies.
To enhance the usability of self-test kits, manufacturers should focus on developing clear and easily understandable instructions, designing components with prominent visual cues, and conducting iterative usability testing. Clear instructions should feature quick guides with simple language appropriate for all education levels, with important notes and cautions presented before tasks to ensure user awareness. Illustrations should be listed in use order, easy to interpret regardless of age or experience. Small components optimized with visual cues can make their functions immediately apparent, minimizing the need to constantly refer to instructions, risk of losing, and enhancing overall satisfaction. By prioritizing these improvements, self-test kits can become more accessible and user-friendly, thereby enhancing individual safety, ensuring the reliability of results, and supporting public health strategies during pandemics and other international disasters.
Authors
Event Type
Poster Presentation
TimeTuesday, April 14:45pm - 6:15pm EDT
LocationFrontenac Foyer