Presentation
MDD23 - Off to the Races: Keeping Pace with Product Development
SessionPoster Session 1
DescriptionManufacturers are constantly challenged with the prospect of keeping up with the accelerated rate of technological advancement. The days of slowly and steadily releasing a product to the market have long since passed. Now, manufacturers need to begin product “version 2” development, including usability testing, before version 1 has even received de novo authorization.
Imagine having just received the good news of de novo launch! The entire company is thrilled but they have already been working on system updates and are pursuing an aggressive submission timeline for the next version. R&D teams are often already prepared to continue designing and readying updated product versions and software releases. But what happens when those advancements affect the critical tasks of the product? Is it true that manufacturers must prepare to re-validate the entire system? And, if so, should these validation tests include all user groups?
This presentation will define the different strategies manufacturers and consultants can take to balance the desire for conservative product development approaches with potentially expansive usability testing cycles, regulatory review timelines, and associated time and monetary “costs.” It will also explore these differences as they apply to varying submission types (e.g., 510(k), Special 510(k), letter to file).
In this presentation, we plan to:
- Define and provide examples of what types of product updates might require supplemental HF validation testing. These would include:
-- impacts to existing critical tasks
-- creation of new critical tasks – or of new essential tasks related to new or existing critical tasks
-- indication expansions
-- user group modifications
-- UI design changes, whether or not they are related to essential or critical tasks
- Explain how to leverage HF documentation to determine the scope for supplemental testing. This would include:
-- Leveraging a KPA or post-market surveillance data analysis to identify or draw comparisons between indications of use and/or users of similar product types
-- Conducting a literature review to identify or draw comparisons between indications of use and/or users of similar product types
-- Comparing the “original” risk analysis to the updated risk analysis to specifically determine which essential and/or critical tasks have been impacted
-- Analyzing any task analysis and/or intended workflow changes resulting from product updates and identifying where user interactions might differ with the new system
-- Identifying potential impacts to critical tasks and the resulting potential harm(s)
-- Analyzing data collected during previous usability tests – and most importantly validation test(s) – to determine if any findings observed during such tests are related to critical tasks and/or user groups impacted by the new system and/or its modified indication for use
- Provide strategies for navigating these different updates and, relatedly, how to determine the proper testing scope. These strategies might include:
-- Affected critical tasks, including new or modified essential tasks: consider only re-evaluating essential and critical tasks affected by design updates, and justifying that remaining critical tasks are not affected and, notably, have already been validated. Furthermore, consider evaluating these impacted tasks with a limited number of participants per user group (e.g., 8 rather than 15).
-- Indication expansions: consider justifying that the added indication does not affect the user population and/or workflow. For example, if the product is a surgical robotic system that is intended for general surgical procedures, consider justifying that the same user population would use the system in the same way even with another surgical procedure.
--- If the added indication does affect the user population or intended use specifications, prepare a list of sub-tasks that might be affected by the additional indication and develop a justification to solely evaluate those tasks.
-- User group modifications: Consider developing a justification that the altered and/or additional user group(s) are not different with regard to Human Factors. For example, the expanded indication of use might have distinct clinical applications, but those applications do not extend to usability (i.e., the user interactions with the system).
-- If there are distinct user interactions with the expanded indications of use, prepare a usability study that focuses only on tasks that would be completed by the altered and/or added user group(s).
-- UI design changes: consider justifying that these design changes either do not affect critical tasks, or that they can only serve to reduce harm/design out use errors for critical tasks. This is similar to the justifications of design changes that sometimes happen after HF Validation but before full submission. For example, something like changing the background from a pure while to a pale cream color on a GUI might not have a significant affect, but changing the color of a button or adding an indication light might.
- Discuss recent FDA feedback about testing devices undergoing updates
-- For indication expansions, manufacturers should focus on proving that individual procedures are still safe and effective, such as by demonstrating clinical backgrounds and applications do not impact user interactions with the system
-- Consider which user groups are “procedure-agnostic” and could be considered “proxy” participants for other user groups. For example, if there are some tasks that both sterile and circulating nurses are responsible for, consider whether including only one type of nurse is sufficiently representative for both user groups
Imagine having just received the good news of de novo launch! The entire company is thrilled but they have already been working on system updates and are pursuing an aggressive submission timeline for the next version. R&D teams are often already prepared to continue designing and readying updated product versions and software releases. But what happens when those advancements affect the critical tasks of the product? Is it true that manufacturers must prepare to re-validate the entire system? And, if so, should these validation tests include all user groups?
This presentation will define the different strategies manufacturers and consultants can take to balance the desire for conservative product development approaches with potentially expansive usability testing cycles, regulatory review timelines, and associated time and monetary “costs.” It will also explore these differences as they apply to varying submission types (e.g., 510(k), Special 510(k), letter to file).
In this presentation, we plan to:
- Define and provide examples of what types of product updates might require supplemental HF validation testing. These would include:
-- impacts to existing critical tasks
-- creation of new critical tasks – or of new essential tasks related to new or existing critical tasks
-- indication expansions
-- user group modifications
-- UI design changes, whether or not they are related to essential or critical tasks
- Explain how to leverage HF documentation to determine the scope for supplemental testing. This would include:
-- Leveraging a KPA or post-market surveillance data analysis to identify or draw comparisons between indications of use and/or users of similar product types
-- Conducting a literature review to identify or draw comparisons between indications of use and/or users of similar product types
-- Comparing the “original” risk analysis to the updated risk analysis to specifically determine which essential and/or critical tasks have been impacted
-- Analyzing any task analysis and/or intended workflow changes resulting from product updates and identifying where user interactions might differ with the new system
-- Identifying potential impacts to critical tasks and the resulting potential harm(s)
-- Analyzing data collected during previous usability tests – and most importantly validation test(s) – to determine if any findings observed during such tests are related to critical tasks and/or user groups impacted by the new system and/or its modified indication for use
- Provide strategies for navigating these different updates and, relatedly, how to determine the proper testing scope. These strategies might include:
-- Affected critical tasks, including new or modified essential tasks: consider only re-evaluating essential and critical tasks affected by design updates, and justifying that remaining critical tasks are not affected and, notably, have already been validated. Furthermore, consider evaluating these impacted tasks with a limited number of participants per user group (e.g., 8 rather than 15).
-- Indication expansions: consider justifying that the added indication does not affect the user population and/or workflow. For example, if the product is a surgical robotic system that is intended for general surgical procedures, consider justifying that the same user population would use the system in the same way even with another surgical procedure.
--- If the added indication does affect the user population or intended use specifications, prepare a list of sub-tasks that might be affected by the additional indication and develop a justification to solely evaluate those tasks.
-- User group modifications: Consider developing a justification that the altered and/or additional user group(s) are not different with regard to Human Factors. For example, the expanded indication of use might have distinct clinical applications, but those applications do not extend to usability (i.e., the user interactions with the system).
-- If there are distinct user interactions with the expanded indications of use, prepare a usability study that focuses only on tasks that would be completed by the altered and/or added user group(s).
-- UI design changes: consider justifying that these design changes either do not affect critical tasks, or that they can only serve to reduce harm/design out use errors for critical tasks. This is similar to the justifications of design changes that sometimes happen after HF Validation but before full submission. For example, something like changing the background from a pure while to a pale cream color on a GUI might not have a significant affect, but changing the color of a button or adding an indication light might.
- Discuss recent FDA feedback about testing devices undergoing updates
-- For indication expansions, manufacturers should focus on proving that individual procedures are still safe and effective, such as by demonstrating clinical backgrounds and applications do not impact user interactions with the system
-- Consider which user groups are “procedure-agnostic” and could be considered “proxy” participants for other user groups. For example, if there are some tasks that both sterile and circulating nurses are responsible for, consider whether including only one type of nurse is sufficiently representative for both user groups
Authors
Event Type
Poster Presentation
TimeMonday, March 314:45pm - 6:15pm EDT
LocationFrontenac Foyer
Digital Health (DH)
Simulation and Education (SE)
Hospital Environments (HE)
Medical and Drug Delivery Devices (MDD)
Patient Safety and Research Initiatives (PS)