Presentation
Improving Validation of Over-the-Counter Medical Device Selection
DescriptionThe U.S. Food and Drug Administration’s Center for Drug Evaluation and Research published guidance on self-selection studies for nonprescription drug products in 2013 (FDA, 2013). On the other hand, FDA’s Center for Devices and Radiological Health (CDRH) has not yet published guidance for the human factors / usability validation related to selection of over-the-counter medical devices. Yet, CDRH has begun to require such validation.
To date, the FDA’s only guidance in this matter has come in the form of comments in response to human factors / usability testing protocols during pre-submissions and HFE/UE summary reports during submissions and through presentations at Human Factors and Ergonomics Society meetings. (See for example, Reina, Wiyor & Shah, 2024).
This panel seeks to promote discussion of the FDA’s current expectations and present ideas for improving upon the entire process of validating the safe and effective selection of over-the-counter medical devices. The panel includes five consultants in human factors / usability engineering with experience preparing protocols for, and validating the safe and effective self-selection of medical devices. The panelists bring experience in presenting protocols through pre-submissions, responding to FDA comments and questions, and submitting successful 510(k) and de Novo applications that include self-selection of OTC medical devices. Based on their experience, the panelists will provide an outline of the FDA’s current expectations, tips and tricks for dealing with these expectations, and proposals for better ways of ensuring the safety and effectiveness of over-the-counter medical devices.
PANELISTS & POSITION STATEMENTS
Keith S. Karn (Moderator) is founder and principal consultant of Human Factors in Context, offering user research, human factors engineering, and physical / digital interface architecture. So far, his most memorable work days include flying a supersonic jet, studying operations in underground coal mines, and observing enteral feeding in Italian NICUs, all in the pursuit of product innovation. Before starting his consultancy, he led teams at Bresslergroup, Design Science, Xerox and Kodak. He holds degrees in engineering from Penn State and North Carolina State and a PhD in Experimental Psychology from University of Rochester.
Statement:
I argue that the current CDRH guidance–such as it is–fails to account for a significant part of the user population making medical device selection decisions for two reasons:
1) Currently, approximately 15% of all retail sales by US customers occur online (U.S. Department of Commerce, 2024). Many more people research products online and make selections prior to visiting a retail store. Furthermore, mobile apps can only be acquired online (e.g., through the Apple App Store and Google Play Store). The FDA’s requirement to test OTC selection decisions in person with the product and packaging in hand, does not account for such online selection decision making and is therefore inadequate for a large and growing percentage of OTC medical device selection decisions.
2) Many people make selection decisions for a family member, for example a child or older parent. As such, the notion of SELF-selection is inadequate to test the intent to select the product for a patient other than the decision maker.
Anthony D. Andre is the Managing Director of Interface Analysis Associates LLC, a healthcare human factors consultancy, located in Saratoga, CA. He is also a Professor of Human Factors and Ergonomics at San Jose State University. Dr. Andre is the Founder and Vice-Chair of the International Symposium on Human Factors and Ergonomics in Health Care and co-Editor in Chief of the journal Human Factors in Healthcare. He is passionate about making healthcare products safe, usable and accessible.
Statement:
I propose to radically change the FDA’s current approach to the validation of OTC medical devices. Based on collaborations with other consultants, the approach to the evaluation of self-selection I advocate more closely mirrors a traditional validation and equally emphasizes the risk of non-indicated users assuming they can use the product as it does indicated users not realizing they can use the product. My approach will provide more robust risk-related data than the current approach, will use fewer resources to conduct, and will be easier for sponsors and consultants to understand and follow.
Jennifer Soosaar is the Director of Human Factors at Core Human Factors, A Rimkus Company, where she manages the research team and recruiting team. Jennifer holds a Ph.D. from Yale University in Molecular, Cellular, and Developmental Biology with a focus in Immunology, and a BA from the University of Virginia in Biology. She is on the AAMI HFE committee. Jennifer’s favorite thing is to identify previously unanticipated potential use errors and then support the development of new mitigations.
Statement:
I challenge the need of an HCP to administer the Health Assessment Questionnaire (HAQ) as currently indicated by FDA. The labeling itself should clearly communicate to lay users what conditions or characteristics they should be aware of directly, since the premise of OTC is that they are making the decision to select the product without HCP intervention. In addition, the current study design proposed by the Agency does not validate if the mitigations against an unintended user choosing the OTC product are successful. I propose a study design that reduces the bias introduced by the HAQ, removes the need for HCP involvement, and provides data for unintended users.
Shannon Halgren is the president and chief human factors consultant at Sage Research + Design. Shannon has been helping clients improve the usability and safety of their products for over 20 years. She has extensive experience in medical device human factors across a wide variety of devices, users and use-environments. She received her M.A. and Ph.D. in Experimental Psychology from Rice University in Houston, Texas where she specialized in Human-Computer Interaction.
Statement:
I agree that self-selection is an important aspect of OTC medical device and pharmaceutical safe use. The FDA’s recent attention to this topic is welcome. However, there are shortcomings to the FDA’s recommended self-selection test methodology. For instance, the FDA does not seem to consider the unique nature of OTC software as medical devices (SaMDs) in their guidance or recommendations. Nor do they acknowledge or advise how to treat data from “defiant” users – those users who understand they are non-intended but chose to use the product anyway. I also agree with the other panel members that the FDA’s guidance about how to recruit for OTC studies is suboptimal and may not allow for a valid test of representative users.
Rachel Aronchick is a Managing Human Factors Specialist on Emergo by UL’s Human Factors Research & Design team and has been working as an HF consultant in the medical space for over 12 years. Rachel is a Certified Human Factors Professional and holds a BS in Human Factors from Tufts University and an MS in Interactive Design from Northeastern University. Rachel is co-author of a book titled Designing for Safe Use – 100 Principles for Making Products Safer.
Statement:
A thorough self-selection study method is critical for ensuring users can accurately self-select for OTC products. However, there are several aspects of the proposed (and still evolving) self-selection study methodology that deviate from long-standing HF validation methodologies and do not support a complete evaluation of risk controls. Furthermore, the suggested recruitment approach places undue burden on sponsors, consultants, and participants alike. I look forward to discussing concerns, sharing case studies of successful approaches, and brainstorming alternate methods with my fellow panelists.
To date, the FDA’s only guidance in this matter has come in the form of comments in response to human factors / usability testing protocols during pre-submissions and HFE/UE summary reports during submissions and through presentations at Human Factors and Ergonomics Society meetings. (See for example, Reina, Wiyor & Shah, 2024).
This panel seeks to promote discussion of the FDA’s current expectations and present ideas for improving upon the entire process of validating the safe and effective selection of over-the-counter medical devices. The panel includes five consultants in human factors / usability engineering with experience preparing protocols for, and validating the safe and effective self-selection of medical devices. The panelists bring experience in presenting protocols through pre-submissions, responding to FDA comments and questions, and submitting successful 510(k) and de Novo applications that include self-selection of OTC medical devices. Based on their experience, the panelists will provide an outline of the FDA’s current expectations, tips and tricks for dealing with these expectations, and proposals for better ways of ensuring the safety and effectiveness of over-the-counter medical devices.
PANELISTS & POSITION STATEMENTS
Keith S. Karn (Moderator) is founder and principal consultant of Human Factors in Context, offering user research, human factors engineering, and physical / digital interface architecture. So far, his most memorable work days include flying a supersonic jet, studying operations in underground coal mines, and observing enteral feeding in Italian NICUs, all in the pursuit of product innovation. Before starting his consultancy, he led teams at Bresslergroup, Design Science, Xerox and Kodak. He holds degrees in engineering from Penn State and North Carolina State and a PhD in Experimental Psychology from University of Rochester.
Statement:
I argue that the current CDRH guidance–such as it is–fails to account for a significant part of the user population making medical device selection decisions for two reasons:
1) Currently, approximately 15% of all retail sales by US customers occur online (U.S. Department of Commerce, 2024). Many more people research products online and make selections prior to visiting a retail store. Furthermore, mobile apps can only be acquired online (e.g., through the Apple App Store and Google Play Store). The FDA’s requirement to test OTC selection decisions in person with the product and packaging in hand, does not account for such online selection decision making and is therefore inadequate for a large and growing percentage of OTC medical device selection decisions.
2) Many people make selection decisions for a family member, for example a child or older parent. As such, the notion of SELF-selection is inadequate to test the intent to select the product for a patient other than the decision maker.
Anthony D. Andre is the Managing Director of Interface Analysis Associates LLC, a healthcare human factors consultancy, located in Saratoga, CA. He is also a Professor of Human Factors and Ergonomics at San Jose State University. Dr. Andre is the Founder and Vice-Chair of the International Symposium on Human Factors and Ergonomics in Health Care and co-Editor in Chief of the journal Human Factors in Healthcare. He is passionate about making healthcare products safe, usable and accessible.
Statement:
I propose to radically change the FDA’s current approach to the validation of OTC medical devices. Based on collaborations with other consultants, the approach to the evaluation of self-selection I advocate more closely mirrors a traditional validation and equally emphasizes the risk of non-indicated users assuming they can use the product as it does indicated users not realizing they can use the product. My approach will provide more robust risk-related data than the current approach, will use fewer resources to conduct, and will be easier for sponsors and consultants to understand and follow.
Jennifer Soosaar is the Director of Human Factors at Core Human Factors, A Rimkus Company, where she manages the research team and recruiting team. Jennifer holds a Ph.D. from Yale University in Molecular, Cellular, and Developmental Biology with a focus in Immunology, and a BA from the University of Virginia in Biology. She is on the AAMI HFE committee. Jennifer’s favorite thing is to identify previously unanticipated potential use errors and then support the development of new mitigations.
Statement:
I challenge the need of an HCP to administer the Health Assessment Questionnaire (HAQ) as currently indicated by FDA. The labeling itself should clearly communicate to lay users what conditions or characteristics they should be aware of directly, since the premise of OTC is that they are making the decision to select the product without HCP intervention. In addition, the current study design proposed by the Agency does not validate if the mitigations against an unintended user choosing the OTC product are successful. I propose a study design that reduces the bias introduced by the HAQ, removes the need for HCP involvement, and provides data for unintended users.
Shannon Halgren is the president and chief human factors consultant at Sage Research + Design. Shannon has been helping clients improve the usability and safety of their products for over 20 years. She has extensive experience in medical device human factors across a wide variety of devices, users and use-environments. She received her M.A. and Ph.D. in Experimental Psychology from Rice University in Houston, Texas where she specialized in Human-Computer Interaction.
Statement:
I agree that self-selection is an important aspect of OTC medical device and pharmaceutical safe use. The FDA’s recent attention to this topic is welcome. However, there are shortcomings to the FDA’s recommended self-selection test methodology. For instance, the FDA does not seem to consider the unique nature of OTC software as medical devices (SaMDs) in their guidance or recommendations. Nor do they acknowledge or advise how to treat data from “defiant” users – those users who understand they are non-intended but chose to use the product anyway. I also agree with the other panel members that the FDA’s guidance about how to recruit for OTC studies is suboptimal and may not allow for a valid test of representative users.
Rachel Aronchick is a Managing Human Factors Specialist on Emergo by UL’s Human Factors Research & Design team and has been working as an HF consultant in the medical space for over 12 years. Rachel is a Certified Human Factors Professional and holds a BS in Human Factors from Tufts University and an MS in Interactive Design from Northeastern University. Rachel is co-author of a book titled Designing for Safe Use – 100 Principles for Making Products Safer.
Statement:
A thorough self-selection study method is critical for ensuring users can accurately self-select for OTC products. However, there are several aspects of the proposed (and still evolving) self-selection study methodology that deviate from long-standing HF validation methodologies and do not support a complete evaluation of risk controls. Furthermore, the suggested recruitment approach places undue burden on sponsors, consultants, and participants alike. I look forward to discussing concerns, sharing case studies of successful approaches, and brainstorming alternate methods with my fellow panelists.
Event Type
Discussion Panel
TimeMonday, March 313:30pm - 4:30pm EDT
LocationHarbour A/B
Medical and Drug Delivery Devices (MDD)