Presentation
Mental Models, Information Architecture, and Medical Product Design
SessionHuman Systems Thinking (DH5)
DescriptionMedical product designers often need to organize large volumes of features, functions, and data into sensible groups that are easily discovered and understood. This activity, referred to as information architecture (Rosenfeld, Morville & Arango, 2015), focuses on the placement, grouping, and naming of interface items. Effective product design depends, to a large degree, on matching users’ mental models, matching expectations, making the most critical and frequent items prominent, grouping together the most related items, and naming items in ways that users understand. In short, effective information architecture requires eliciting, representing, understanding, and matching users’ mental models.
For purposes of design, mental models (Johnson-Laird, 1983; Gentner & Stevens, 1983) are internal knowledge representations of products we use. These models help us understand situations, make predictions, and simulate the results of our actions. The knowledge we have, and how it is organized, influences our expectations as well as how we experience products.
The challenge, however, is that these models are locked away in peoples’ minds and are difficult to uncover. This presentation introduces methods for eliciting, representing, and analyzing mental models so they can be used to design effective information architecture for medical products.
Eliciting Mental Models
Let us begin with the problem of eliciting knowledge (c.f., Cooke, 1994); that is, getting important knowledge out of peoples’ heads so that it can be understood by designers. Some methods, listed below, are new, and others are quite old:
• Pairwise comparisons (Thurstone, 1927) - The first method is pairwise comparisons. Research participants rate the relatedness of each pair of concepts on a Likert-type scale. Unfortunately, as the number of concepts to be rated increases, the number of ratings increases exponentially. For example, rating the pairwise relatedness of only 30 concepts requires 435 comparisons. Participants find this procedure arduous.
• Speeded comparison (Toteva, 2017) – This variation of paired comparisons is faster. Participants are shown each pair of concepts (one pair at a time) and asked to judge, as quickly and accurately as possible, whether the items are related. Fast “Yes” responses receive a rating of 4, slow “Yes” responses receive a rating of 3, slow “No” responses receive a rating of 2 and fast “No” responses receive a rating of 1. This leads to similar results as paired comparisons in a fraction of the time.
• Card Sorting (Fincher & Tenenberg, 2005) – This is the most common method. Each concept is printed on a separate card, and participants arrange related cards into groups. This is more engaging than paired comparisons but can still be quite time-consuming.
• Target Exercise (Tossel, Schvaneveldt, & Branaghan, 2010) – In this method, one concept is displayed in the bullseye of a target. The user then places all other concepts in concentric rings around that bullseye. The closer to the bullseye the more related to the target concept. This is repeated until all concepts have appeared in the bullseye. This method can be used to collect twice the data in the same time required by paired comparisons. Participants also rated it as more engaging.
• SpAM (Hout & Goldinger & Ferguson, 2013) – SpAM is short for the spatial arrangement method. It asks participants to arrange all concepts on a canvas so that highly related items are close to each other and unrelated items are far away. Again, this is faster and more engaging than paired comparisons.
• Concept Mapping (Moon, et al, 2011; Johnson & Coleman, 2020) – Participants create a diagram in which concepts (represented as nodes) are connected by labeled links describing the relationships among concepts. This captures how ideas are related and how they form a structured understanding of a particular topic or domain.
• LinkIt Constrained Concept Mapping (Branaghan, Schvaneveldt, & Winner, 2021) – LinkIt is like concept mapping but constrains the set of concepts to be rated. The experimenter provides the terms, so that participants cannot write in their own nodes. This enables the experiment to easily measure the similarity of knowledge structures.
Representing Mental Models
Knowledge elicitation typically results in a simple square matrix of relatedness ratings, which is pretty uninformative. Next, researchers use knowledge representation methods to visualize important relationships among items, enabling them to make design choices. There are three types of knowledge representation methods:
• Cluster analysis (Aldenderfer & Blashfield, 1984) – This method groups the most related items together. Sometimes, researchers simply choose a predefined number of groups to distribute the items (k-means clustering). Other times, for example in hierarchical clustering, researchers arrange these concepts into a tree structure or dendrogram.
• Spatial analysis (Kruskal & Wish, 1978) – Spatial analysis uses the original ratings data to distribute the items in n-dimensional space. For example, multidimensional scaling uses the least squares fit approach to place highly related concepts close together and unrelated items far apart. Unfortunately, sometimes the relationship between highly related items gets distorted (Branaghan, 1990) yielding a representation that is difficult to interpret.
• Network analysis (Schvaneveldt, Durso, & Dearholt, 1989) – These methods use the ratings data to generate a network in which each item is represented as a node, and each relationship is represented as a link between nodes. Highly related items are linked to each other whereas unrelated items are not linked. For example, Pathfinder (Schvaneveldt, 1990) yields a structure where only the most related items are linked, highlighting important relationships while suppressing unrelated relationships. Nearest neighbor networks, use a similar approach but may be more useful for highlighting the grouping of items.
Analyzing Mental Models
For many purposes, simply visually inspecting the model (dendrogram, spatial representation, or network) provides immediate insight. However, additional analysis can be useful. For example, network models enable you to determine the most important concepts because they are often the most central part of the network. That is, they are connected to the most other items (degree centrality), are on the most paths from one item to another (betweenness centrality) or represent the item with the shortest path to every other item (eccentricity). These items should be the most conspicuous in your design. Other analyses enable you to compare one person’s network to another. People with similar networks tend to think about the product similarly (Branaghan & Hildebrand, 2011).
For purposes of design, mental models (Johnson-Laird, 1983; Gentner & Stevens, 1983) are internal knowledge representations of products we use. These models help us understand situations, make predictions, and simulate the results of our actions. The knowledge we have, and how it is organized, influences our expectations as well as how we experience products.
The challenge, however, is that these models are locked away in peoples’ minds and are difficult to uncover. This presentation introduces methods for eliciting, representing, and analyzing mental models so they can be used to design effective information architecture for medical products.
Eliciting Mental Models
Let us begin with the problem of eliciting knowledge (c.f., Cooke, 1994); that is, getting important knowledge out of peoples’ heads so that it can be understood by designers. Some methods, listed below, are new, and others are quite old:
• Pairwise comparisons (Thurstone, 1927) - The first method is pairwise comparisons. Research participants rate the relatedness of each pair of concepts on a Likert-type scale. Unfortunately, as the number of concepts to be rated increases, the number of ratings increases exponentially. For example, rating the pairwise relatedness of only 30 concepts requires 435 comparisons. Participants find this procedure arduous.
• Speeded comparison (Toteva, 2017) – This variation of paired comparisons is faster. Participants are shown each pair of concepts (one pair at a time) and asked to judge, as quickly and accurately as possible, whether the items are related. Fast “Yes” responses receive a rating of 4, slow “Yes” responses receive a rating of 3, slow “No” responses receive a rating of 2 and fast “No” responses receive a rating of 1. This leads to similar results as paired comparisons in a fraction of the time.
• Card Sorting (Fincher & Tenenberg, 2005) – This is the most common method. Each concept is printed on a separate card, and participants arrange related cards into groups. This is more engaging than paired comparisons but can still be quite time-consuming.
• Target Exercise (Tossel, Schvaneveldt, & Branaghan, 2010) – In this method, one concept is displayed in the bullseye of a target. The user then places all other concepts in concentric rings around that bullseye. The closer to the bullseye the more related to the target concept. This is repeated until all concepts have appeared in the bullseye. This method can be used to collect twice the data in the same time required by paired comparisons. Participants also rated it as more engaging.
• SpAM (Hout & Goldinger & Ferguson, 2013) – SpAM is short for the spatial arrangement method. It asks participants to arrange all concepts on a canvas so that highly related items are close to each other and unrelated items are far away. Again, this is faster and more engaging than paired comparisons.
• Concept Mapping (Moon, et al, 2011; Johnson & Coleman, 2020) – Participants create a diagram in which concepts (represented as nodes) are connected by labeled links describing the relationships among concepts. This captures how ideas are related and how they form a structured understanding of a particular topic or domain.
• LinkIt Constrained Concept Mapping (Branaghan, Schvaneveldt, & Winner, 2021) – LinkIt is like concept mapping but constrains the set of concepts to be rated. The experimenter provides the terms, so that participants cannot write in their own nodes. This enables the experiment to easily measure the similarity of knowledge structures.
Representing Mental Models
Knowledge elicitation typically results in a simple square matrix of relatedness ratings, which is pretty uninformative. Next, researchers use knowledge representation methods to visualize important relationships among items, enabling them to make design choices. There are three types of knowledge representation methods:
• Cluster analysis (Aldenderfer & Blashfield, 1984) – This method groups the most related items together. Sometimes, researchers simply choose a predefined number of groups to distribute the items (k-means clustering). Other times, for example in hierarchical clustering, researchers arrange these concepts into a tree structure or dendrogram.
• Spatial analysis (Kruskal & Wish, 1978) – Spatial analysis uses the original ratings data to distribute the items in n-dimensional space. For example, multidimensional scaling uses the least squares fit approach to place highly related concepts close together and unrelated items far apart. Unfortunately, sometimes the relationship between highly related items gets distorted (Branaghan, 1990) yielding a representation that is difficult to interpret.
• Network analysis (Schvaneveldt, Durso, & Dearholt, 1989) – These methods use the ratings data to generate a network in which each item is represented as a node, and each relationship is represented as a link between nodes. Highly related items are linked to each other whereas unrelated items are not linked. For example, Pathfinder (Schvaneveldt, 1990) yields a structure where only the most related items are linked, highlighting important relationships while suppressing unrelated relationships. Nearest neighbor networks, use a similar approach but may be more useful for highlighting the grouping of items.
Analyzing Mental Models
For many purposes, simply visually inspecting the model (dendrogram, spatial representation, or network) provides immediate insight. However, additional analysis can be useful. For example, network models enable you to determine the most important concepts because they are often the most central part of the network. That is, they are connected to the most other items (degree centrality), are on the most paths from one item to another (betweenness centrality) or represent the item with the shortest path to every other item (eccentricity). These items should be the most conspicuous in your design. Other analyses enable you to compare one person’s network to another. People with similar networks tend to think about the product similarly (Branaghan & Hildebrand, 2011).
Author
Event Type
Oral Presentations
TimeTuesday, April 111:15am - 12:00pm EDT
LocationPier 2/3
Digital Health (DH)