Presentation
DH10 - Factors Associated With Different Levels of Trust for Artificial Intelligence Technologies in Clinical Practice Settings: A Review of Empirical Research
SessionPoster Session 2
DescriptionIntroduction: The potential for artificial intelligence (AI) to improve care processes throughout the healthcare continuum (e.g., screening, diagnosis, treatment) has been well documented. Trust has been consistently identified as a key factor influencing individuals’ intentions to use AI tools, including among healthcare professionals. This suggests that understanding the correlates of trust can represent an important strategy to optimize healthcare professionals’ adoption and use of AI tools.
Trust is a multi-level phenomenon that can be decomposed into three levels: purpose (i.e., automation is being used as intended), process (i.e., appropriateness of automation to task), and performance (i.e., automation outputs and their characteristics). This suggests that to build trust in AI performance, trust needs to be fostered in the process and purpose. Factors associated with trust may also differ based on which level of trust is being examined.
Although reviews have been published that examine the correlates of trust in general, to our knowledge, none has examined how correlates differ by level of trust. Thus, we aimed to conduct a review of existing literature to identify factors associated with purpose-, process-, and performance-related trust of AI tools among healthcare professionals. The findings can help guide developers who are optimizing the human-computer interaction of AI tools among healthcare professionals.
Methods: On September 28, 2024, we conducted an electronic search in MEDLINE, CINAHL, Scopus, and Web of Science for peer-reviewed literature using search terms and subject headings pertaining to healthcare professionals, AI, and trust. No additional search filters were applied. We also hand-searched bibliographies of included papers for additional studies.
Our inclusion criteria were papers that: 1) described empirical research (e.g., not commentaries or reviews), 2) were written in English, and 3) assessed for reasons for participants’ current level of trust in AI, and 4) the AI technology was being considered or implemented into care processes. We did not restrict articles based on the type of healthcare professionals. Since trust may systematically differ between healthcare professionals and non-healthcare professionals, we excluded articles that combined data from healthcare professionals and other types of respondents (e.g., hospital administrators, patients). However, we included articles that separately reported findings between healthcare professionals and other respondents.
The lead author screened articles’ titles and abstracts for initial eligibility and screened full texts to confirm final eligibility. The lead author also extracted the following information from each included study: 1) study type, 2) location (country), 3) care setting, 4) type of healthcare professionals, 5) AI technology involved, 6) stage of AI implementation, 7) care process(es) affected, 8) methods used, 9) sample size, 10) conceptual frameworks used, 11) type of trust, and 12) key findings. We used Lee and See’s model of trust in automation to define the categories for type of trust. This model suggests that trust can be dependent on performance, process, and purpose of the automation technology. The updated Consolidated Framework for Implementation Research was used to categorize factors that influence trust. Given the exploratory nature of this study, no study appraisal was conducted. Due to the heterogeneity in how trust was measured, we did not conduct a meta-analysis.
Results: We initially found 12,626 articles. This review ultimately included 75 articles after de-duplication and title, abstract, and full-text reviews. Most studies (n=45, 60.0%) used data from North America or Asia, and studied physicians’ perspectives (n=63, 84.0%). The care settings studied included emergency medicine, family medicine, radiology, and critical care. The most common type of AI technology studied was clinical decision support (n=42, 56.0%). Care processes that AI supported included diagnoses (n=39, 52.0%), treatment (n=19, 25.3%), and triage (n=3, 4.0%). Only 18.7% (n=14) of studies were conducted during or after implementation. Performance- and process-related trust were the most commonly studied levels of trust (n=70, 93.3%).
Overall, the included studies suggested that higher trust influenced other downstream outcomes, such as acceptability and behavioral intentions to use AI. Performance-related trust was often related to participants’ responses after being exposed to AI outputs. This level of trust may be influenced by factors pertaining to innovation (e.g., explainable AI, visibility of input variables), individual (e.g., performance expectancy), implementation process (e.g., dissemination of knowledge on how to interpret risk scores), inner setting (e.g., endorsement by peers), and outer setting (e.g., clinical trial evaluations on AI). Similarly, process-related trust often was related to differences in trust by task type and perceptions of how tools were developed. This level of trust may be influenced by factors pertaining to innovation (e.g., ability to support specific tasks), individual (e.g., type of healthcare professional), implementation process (e.g., time provided to healthcare professionals to familiarize themselves with the AI tool), inner setting (e.g., endorsement by peers), and outer setting (e.g., scientific evidence supporting the use of AI. There were limited evaluations on factors that contribute to purpose-related trust. The extant evidence has only focused on individual-level factors (e.g., age, education level).
Discussion: This review highlights several factors that may influence trust, such as performance expectancy and explainable AI, which have been previously reported in past reviews. However, when examining whether all three levels of trust were studied, we found a disproportionate focus on performance- and process-related trust. To our knowledge, only one study reported findings on purpose-related trust. Overall, for both performance- and process-related trust, characteristics of the innovation and individuals were the most commonly studied domains while fewer findings have been reported pertaining to implementation process, inner setting, and outer setting. This suggests that multi-level approaches are needed to support trust in AI tools as they are implemented into care delivery settings. Several limitations of this review provide insights for future research as described below.
Trust was often assessed through surveys with extensive variation in the types of author-developed questions used. Consequently, this limited the ability to make direct comparisons in study findings. This suggests a growing need for the development and use of validated instruments for measuring trust through surveys. For instance, the biomedical informatics community has developed the Theory of Trust and Acceptance of Artificial Intelligence Technology (TrAAIT), which consists of thirteen questions. Given that trust may differ based on whether it is related to the purpose, process, or performance of AI, there may also be a need for tools to be developed that can assess these constructs.
Different types of healthcare professionals have different clinical workflows, which has implications on information needs, resources to support decision-making processes, and optimal implementation of these resources. Despite this important nuance, to date, few studies have directly compared what factors contribute to trust levels among different types of healthcare professionals. With the growth of team-based care in healthcare delivery systems, there is a need for additional research to fully explore how different members of the care team respond to AI tools.
Lastly, trust can vary throughout the trajectory of task completion and may change in response to whether the end-user agrees with the AI outputs. Despite this, this review found few studies that used a longitudinal design to examine how trust changes over time. Future studies are needed that examine this to understand how trust changes in response to specific events.
Conclusion: This review found multi-level factors can influence trust in AI among healthcare professionals, especially for performance- and process-related trust. Further studies are needed to study purpose-related trust, develop validated measurement tools, examine differences between types of healthcare professionals, and understand how trust changes longitudinally.
Trust is a multi-level phenomenon that can be decomposed into three levels: purpose (i.e., automation is being used as intended), process (i.e., appropriateness of automation to task), and performance (i.e., automation outputs and their characteristics). This suggests that to build trust in AI performance, trust needs to be fostered in the process and purpose. Factors associated with trust may also differ based on which level of trust is being examined.
Although reviews have been published that examine the correlates of trust in general, to our knowledge, none has examined how correlates differ by level of trust. Thus, we aimed to conduct a review of existing literature to identify factors associated with purpose-, process-, and performance-related trust of AI tools among healthcare professionals. The findings can help guide developers who are optimizing the human-computer interaction of AI tools among healthcare professionals.
Methods: On September 28, 2024, we conducted an electronic search in MEDLINE, CINAHL, Scopus, and Web of Science for peer-reviewed literature using search terms and subject headings pertaining to healthcare professionals, AI, and trust. No additional search filters were applied. We also hand-searched bibliographies of included papers for additional studies.
Our inclusion criteria were papers that: 1) described empirical research (e.g., not commentaries or reviews), 2) were written in English, and 3) assessed for reasons for participants’ current level of trust in AI, and 4) the AI technology was being considered or implemented into care processes. We did not restrict articles based on the type of healthcare professionals. Since trust may systematically differ between healthcare professionals and non-healthcare professionals, we excluded articles that combined data from healthcare professionals and other types of respondents (e.g., hospital administrators, patients). However, we included articles that separately reported findings between healthcare professionals and other respondents.
The lead author screened articles’ titles and abstracts for initial eligibility and screened full texts to confirm final eligibility. The lead author also extracted the following information from each included study: 1) study type, 2) location (country), 3) care setting, 4) type of healthcare professionals, 5) AI technology involved, 6) stage of AI implementation, 7) care process(es) affected, 8) methods used, 9) sample size, 10) conceptual frameworks used, 11) type of trust, and 12) key findings. We used Lee and See’s model of trust in automation to define the categories for type of trust. This model suggests that trust can be dependent on performance, process, and purpose of the automation technology. The updated Consolidated Framework for Implementation Research was used to categorize factors that influence trust. Given the exploratory nature of this study, no study appraisal was conducted. Due to the heterogeneity in how trust was measured, we did not conduct a meta-analysis.
Results: We initially found 12,626 articles. This review ultimately included 75 articles after de-duplication and title, abstract, and full-text reviews. Most studies (n=45, 60.0%) used data from North America or Asia, and studied physicians’ perspectives (n=63, 84.0%). The care settings studied included emergency medicine, family medicine, radiology, and critical care. The most common type of AI technology studied was clinical decision support (n=42, 56.0%). Care processes that AI supported included diagnoses (n=39, 52.0%), treatment (n=19, 25.3%), and triage (n=3, 4.0%). Only 18.7% (n=14) of studies were conducted during or after implementation. Performance- and process-related trust were the most commonly studied levels of trust (n=70, 93.3%).
Overall, the included studies suggested that higher trust influenced other downstream outcomes, such as acceptability and behavioral intentions to use AI. Performance-related trust was often related to participants’ responses after being exposed to AI outputs. This level of trust may be influenced by factors pertaining to innovation (e.g., explainable AI, visibility of input variables), individual (e.g., performance expectancy), implementation process (e.g., dissemination of knowledge on how to interpret risk scores), inner setting (e.g., endorsement by peers), and outer setting (e.g., clinical trial evaluations on AI). Similarly, process-related trust often was related to differences in trust by task type and perceptions of how tools were developed. This level of trust may be influenced by factors pertaining to innovation (e.g., ability to support specific tasks), individual (e.g., type of healthcare professional), implementation process (e.g., time provided to healthcare professionals to familiarize themselves with the AI tool), inner setting (e.g., endorsement by peers), and outer setting (e.g., scientific evidence supporting the use of AI. There were limited evaluations on factors that contribute to purpose-related trust. The extant evidence has only focused on individual-level factors (e.g., age, education level).
Discussion: This review highlights several factors that may influence trust, such as performance expectancy and explainable AI, which have been previously reported in past reviews. However, when examining whether all three levels of trust were studied, we found a disproportionate focus on performance- and process-related trust. To our knowledge, only one study reported findings on purpose-related trust. Overall, for both performance- and process-related trust, characteristics of the innovation and individuals were the most commonly studied domains while fewer findings have been reported pertaining to implementation process, inner setting, and outer setting. This suggests that multi-level approaches are needed to support trust in AI tools as they are implemented into care delivery settings. Several limitations of this review provide insights for future research as described below.
Trust was often assessed through surveys with extensive variation in the types of author-developed questions used. Consequently, this limited the ability to make direct comparisons in study findings. This suggests a growing need for the development and use of validated instruments for measuring trust through surveys. For instance, the biomedical informatics community has developed the Theory of Trust and Acceptance of Artificial Intelligence Technology (TrAAIT), which consists of thirteen questions. Given that trust may differ based on whether it is related to the purpose, process, or performance of AI, there may also be a need for tools to be developed that can assess these constructs.
Different types of healthcare professionals have different clinical workflows, which has implications on information needs, resources to support decision-making processes, and optimal implementation of these resources. Despite this important nuance, to date, few studies have directly compared what factors contribute to trust levels among different types of healthcare professionals. With the growth of team-based care in healthcare delivery systems, there is a need for additional research to fully explore how different members of the care team respond to AI tools.
Lastly, trust can vary throughout the trajectory of task completion and may change in response to whether the end-user agrees with the AI outputs. Despite this, this review found few studies that used a longitudinal design to examine how trust changes over time. Future studies are needed that examine this to understand how trust changes in response to specific events.
Conclusion: This review found multi-level factors can influence trust in AI among healthcare professionals, especially for performance- and process-related trust. Further studies are needed to study purpose-related trust, develop validated measurement tools, examine differences between types of healthcare professionals, and understand how trust changes longitudinally.
Event Type
Poster Presentation
TimeTuesday, April 14:45pm - 6:15pm EDT
LocationFrontenac Foyer