Presentation
MDD20 - In Pursuit of Equity in Recruitment: Expanded Investigation of Discrimination in Participant Recruitment
SessionPoster Session 2
DescriptionSystemic discrimination in the design and evaluation of medical technology (MedTech) can result in medical devices that do not adequately meet the needs of all users and patients. For example, pulse oximeters and temporal thermometers have been found to be significantly less accurate in patients with darker skin tones, resulting in delayed medical diagnosis and treatment. Similarly, lack of consideration for female anatomical differences to males has led to a disproportionate number of female post-surgical complications from hernia repairs using surgical mesh.
Because human factors (HF) evaluations of MedTech assess if medical technologies are safe, usable, and meet intended users’ needs, they can play a pivotal role in the development and commercialization of biased MedTech. Specifically, participant sampling techniques for HF evaluations can be vulnerable to systemic discrimination such as under representation of minority groups. Whether intentionally or unintentionally, if this occurs, study samples are unlikely to reflect the demographic distribution of user populations, and the results of those studies are equally unlikely to reflect the behavior and opinions of actual end users. This limits the utility of conclusions drawn from those evaluations and can contribute to the development and commercialization of biased medical hardware and software, thereby disadvantaging minority groups, and catalyzing health inequity and disparity.
The desire to achieve design for equity is one of the core values at Healthcare Human Factors (HHF) and as one of the largest healthcare-focused human factors teams internationally, HHF is in a unique position to influence the equity of medical device design globally. Thus, in 2020, HHF began investigating if systemic discrimination affected our current recruitment practices.
This poster will share the methodology we used to continue our investigation of HHF’s participant recruitment practices, and the results of our evaluation. By sharing this information, we hope to focus attention on the potential for systemic discrimination in participant recruitment, the negative repercussion thereof, and to illustrate how this type of discrimination can be investigated within organizations. Furthermore, we hope to raise awareness of the availability of publicly available data standards and as a survey design resource, and disaggregated demographics as a resource for comparative analyses.
Previous Investigation
In a previous investigation, Ritvo and Ho (2024) found no statistical gender differences between MDs or RNs between the province and the HHF respondent pool, or between the HHF respondent pool and HHF study participants. Additionally, there was no statistical difference found between race-based data of the MDs and RNs in the HHF respondent pool and study participants. At that time, population level race-based data by occupation was not available. Therefore, an analogous evaluation of race-based data for MDs and RNs between the province and the HHF respondent pool was not possible.
Current Investigation
Statistics Canada recently published national and provincial race-based data by occupation, allowing us to complete the statistical comparison of race-based data for MDs and RNs between our HHF respondent pool and the province of Ontario. Additionally, we further extended our investigation of potential systemic discrimination in our recruitment practices to include Respiratory Therapists (RTs).
More specifically, for the current investigation, we asked the following questions:
• Is HHF’s respondent pool of MDs and RNs representative of the Ontario RN and MD population with regards to race categories?
• Is HHF’s respondent pool of RTs representative of the Ontario RT population with regards to race and gender categories?
• Is HHF’s RT usability participants representative of the HHF’s RT respondent pool with regards to race and gender categories?
Evaluation Method
Data for the following analyses was collected through the addition of a voluntary Racial and Social Equity section to HHF’s online participant recruitment surveys. This addition posed questions about respondents’ ethnic and cultural origins, indigenous identification, racial background, religious and spiritual affiliation, and gender identity. Answering these questions was not mandatory, and the information provided did not influence respondents’ eligibility to participate in studies.
Using one-proportioned, two-tailed z-tests, with a p < 0.05, we statistically compared this data to publicly available dissagregated demographic data.
Summary of Results
By statistically comparing data collected from our Participant Recruitment Surveys with publicly available, disaggregated demographic data, we determined there is no evidence of systemic discrimination in the way HHF recruits individuals for the HHF respondent pool, or in the way HHF selects participants for usability evaluations from its respondent pool.
Specifically:
• Gender proportions in the HHF respondent pool was not significantly different than the gender proportion of MDs, RNs, and RTs in Ontario.
• Gender proportion of MDs, RNs, and RTs in the HHF respondent pool was not significantly different than the gender proportion of HHF MD, RN, and RT usability participants.
• Race categories proportions of MDs, RNs, and RTs in the HHF respondent pool was not significantly different than the race categories proportions of HHF MDs, RNs, and RTs in Ontario.
• Race categories proportions of MDs, RNs, and RTs in the HHF respondent pool was not significantly different than the race categories proportions of HHF MD, RN, and RT usability participants.
Discussion and Next Steps
Statistical comparison of data collected from our Participant Recruitment Surveys with publicly available, disaggregated demographic data, do not show evidence of systemic discrimination in the way HHF recruits individuals for the HHF respondent pool, or in the way HHF selects participants for usability evaluations from its respondent pool.
This evaluation serves as an example of how systemic discrimination in participant recruitment practices can be investigated within organizations and highlights the need to do so. HHF hopes to continue our investigation of gender and racial discrimination in recruiting by examining other social equity factors including, ethnic and cultural origins, religious, and spiritual affiliations in Canada, and by conducing analogous investigations in other geographies (e.g., USA).
Knowledge Advancement and Target Audience
The goal of this poster is to continue to generate attention on- and discussion about- the risk of systemic discrimination in HF recruiting practices, as well as the importance of investigation of such discrimination within HF firms, and/or organizations that provide recruiting services. Moreover, we hope that this series of investigations serve as an example of how systemic discrimination in recruiting practices can be investigated within organizations quickly, easily, and economically. Without recruitment equity, HF evaluations are unlikely to reflect the behavior and opinions of true user populations, thus limiting the validity and utility of the evaluation results and conclusions. We hope that sharing our work in this space will generate awareness and discussion about this important topic within the HF community.
The learning objectives of this poster are:
• Emphasize the negative repercussions of systemic discrimination in HF recruitment practices;
• Provide a methodology for investigating systemic discrimination in recruitment practices within organizations that can be used by members of the target audience quickly, easily, and economically; and
• Raise awareness of the availability of publicly accessible data standards as a survey design resource and disaggregated demographics as a resource for comparative analyses.
Because human factors (HF) evaluations of MedTech assess if medical technologies are safe, usable, and meet intended users’ needs, they can play a pivotal role in the development and commercialization of biased MedTech. Specifically, participant sampling techniques for HF evaluations can be vulnerable to systemic discrimination such as under representation of minority groups. Whether intentionally or unintentionally, if this occurs, study samples are unlikely to reflect the demographic distribution of user populations, and the results of those studies are equally unlikely to reflect the behavior and opinions of actual end users. This limits the utility of conclusions drawn from those evaluations and can contribute to the development and commercialization of biased medical hardware and software, thereby disadvantaging minority groups, and catalyzing health inequity and disparity.
The desire to achieve design for equity is one of the core values at Healthcare Human Factors (HHF) and as one of the largest healthcare-focused human factors teams internationally, HHF is in a unique position to influence the equity of medical device design globally. Thus, in 2020, HHF began investigating if systemic discrimination affected our current recruitment practices.
This poster will share the methodology we used to continue our investigation of HHF’s participant recruitment practices, and the results of our evaluation. By sharing this information, we hope to focus attention on the potential for systemic discrimination in participant recruitment, the negative repercussion thereof, and to illustrate how this type of discrimination can be investigated within organizations. Furthermore, we hope to raise awareness of the availability of publicly available data standards and as a survey design resource, and disaggregated demographics as a resource for comparative analyses.
Previous Investigation
In a previous investigation, Ritvo and Ho (2024) found no statistical gender differences between MDs or RNs between the province and the HHF respondent pool, or between the HHF respondent pool and HHF study participants. Additionally, there was no statistical difference found between race-based data of the MDs and RNs in the HHF respondent pool and study participants. At that time, population level race-based data by occupation was not available. Therefore, an analogous evaluation of race-based data for MDs and RNs between the province and the HHF respondent pool was not possible.
Current Investigation
Statistics Canada recently published national and provincial race-based data by occupation, allowing us to complete the statistical comparison of race-based data for MDs and RNs between our HHF respondent pool and the province of Ontario. Additionally, we further extended our investigation of potential systemic discrimination in our recruitment practices to include Respiratory Therapists (RTs).
More specifically, for the current investigation, we asked the following questions:
• Is HHF’s respondent pool of MDs and RNs representative of the Ontario RN and MD population with regards to race categories?
• Is HHF’s respondent pool of RTs representative of the Ontario RT population with regards to race and gender categories?
• Is HHF’s RT usability participants representative of the HHF’s RT respondent pool with regards to race and gender categories?
Evaluation Method
Data for the following analyses was collected through the addition of a voluntary Racial and Social Equity section to HHF’s online participant recruitment surveys. This addition posed questions about respondents’ ethnic and cultural origins, indigenous identification, racial background, religious and spiritual affiliation, and gender identity. Answering these questions was not mandatory, and the information provided did not influence respondents’ eligibility to participate in studies.
Using one-proportioned, two-tailed z-tests, with a p < 0.05, we statistically compared this data to publicly available dissagregated demographic data.
Summary of Results
By statistically comparing data collected from our Participant Recruitment Surveys with publicly available, disaggregated demographic data, we determined there is no evidence of systemic discrimination in the way HHF recruits individuals for the HHF respondent pool, or in the way HHF selects participants for usability evaluations from its respondent pool.
Specifically:
• Gender proportions in the HHF respondent pool was not significantly different than the gender proportion of MDs, RNs, and RTs in Ontario.
• Gender proportion of MDs, RNs, and RTs in the HHF respondent pool was not significantly different than the gender proportion of HHF MD, RN, and RT usability participants.
• Race categories proportions of MDs, RNs, and RTs in the HHF respondent pool was not significantly different than the race categories proportions of HHF MDs, RNs, and RTs in Ontario.
• Race categories proportions of MDs, RNs, and RTs in the HHF respondent pool was not significantly different than the race categories proportions of HHF MD, RN, and RT usability participants.
Discussion and Next Steps
Statistical comparison of data collected from our Participant Recruitment Surveys with publicly available, disaggregated demographic data, do not show evidence of systemic discrimination in the way HHF recruits individuals for the HHF respondent pool, or in the way HHF selects participants for usability evaluations from its respondent pool.
This evaluation serves as an example of how systemic discrimination in participant recruitment practices can be investigated within organizations and highlights the need to do so. HHF hopes to continue our investigation of gender and racial discrimination in recruiting by examining other social equity factors including, ethnic and cultural origins, religious, and spiritual affiliations in Canada, and by conducing analogous investigations in other geographies (e.g., USA).
Knowledge Advancement and Target Audience
The goal of this poster is to continue to generate attention on- and discussion about- the risk of systemic discrimination in HF recruiting practices, as well as the importance of investigation of such discrimination within HF firms, and/or organizations that provide recruiting services. Moreover, we hope that this series of investigations serve as an example of how systemic discrimination in recruiting practices can be investigated within organizations quickly, easily, and economically. Without recruitment equity, HF evaluations are unlikely to reflect the behavior and opinions of true user populations, thus limiting the validity and utility of the evaluation results and conclusions. We hope that sharing our work in this space will generate awareness and discussion about this important topic within the HF community.
The learning objectives of this poster are:
• Emphasize the negative repercussions of systemic discrimination in HF recruitment practices;
• Provide a methodology for investigating systemic discrimination in recruitment practices within organizations that can be used by members of the target audience quickly, easily, and economically; and
• Raise awareness of the availability of publicly accessible data standards as a survey design resource and disaggregated demographics as a resource for comparative analyses.
Event Type
Poster Presentation
TimeTuesday, April 14:45pm - 6:15pm EDT
LocationFrontenac Foyer