Title: Shared Mental Models in Human-AI Team Decision Making

Date: Thursday, November 2, 2023

Time: 1:30 PM - 2:30 PM EST

Location:             (Virtual) Microsoft Teams Meeting

Meeting ID: 227 735 254 635
Passcode: NSBdUA

Sarah Walsh

Robotics Ph.D. Student

School of Aerospace Engineering

Georgia Institute of Technology

Committee:

Dr. Karen Feigh (Advisor) -- School of Aerospace Engineering, Georgia Institute of Technology

Dr. Frank Dellaert -- School of Interactive Computing, Georgia Institute of Technology

Dr. Sonia Chernova -- School of Interactive Computing, Georgia Institute of Technology

Dr. Ashok Goel -- School of Interactive Computing, Georgia Institute of Technology

Dr. Eugene Santos-- Department of Computer Science, Dartmouth College

Dr. Daniel Pless -- Research Scientist, Sandia National Laboratories

Abstract:

To effectively utilize AI-decision support tools, a human-AI team must be able to maintain a shared understanding of the environment, team goals, and problem constraints. Otherwise, the human-AI team performance will never outperform individual performance. In this dissertation, we investigate an approach to the interaction between humans and agents via a Shared Mental Model (SMM). SMMs provide meaningful information to and about both humans and AI-systems. Our goal is to demonstrate that joint human-AI systems which include a SMM increase accuracy and efficiency and reduce dissonance between human and AI systems in decision-making tasks. This work is comprised of two primary research thrusts: 1) developing methods to build accurate and useful SMMs in human-agent teams. 2) developing metrics to quantify the impact of partial and complete SMMs in HATs. We assess performance of these reduced-order SMMs by varying correctness of task mental models, and completeness of team mental models to determine if the onus of collaboration should primarily fall on the user, the AI agent, or be shared in HATs. This dissertation contributes three primary findings: 1) SMM in HATs improve decision making accuracy and speed, 2) Shifting the burden of team strategy between the user and AI-agent through unidirectional team mental models leads to both positive and negative impacts on team performance, while bi-directional team models improve team performance, and 3) AI-decision support using novel methodology can infer user decision making tendencies, abilities, and preferences. These and associated findings are then summarized as design recommendations for implementing SMMs in HAT dyads.