Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI

Alon Jacov,  Ana Marasović, Ana Marasović, Yoav Goldberg

Abstract: Trust is a central component of the interaction between people and AI, in that ‘incorrect’ levels of trust may cause misuse, abuse or disuse of the technology. But what, precisely, is the nature of trust in AI? What are the prerequisites and goals of the cognitive mechanism of trust, and how can we promote them, or assess whether they are being satisfied in a given interaction? This work aims to answer these questions. We discuss a model of trust inspired by, but not identical to, interpersonal trust (i.e., trust between people) as defined by sociologists. This model rests on two key properties: the vulnerability of the user; and the ability to anticipate the impact of the AI model’s decisions. We incorporate a formalization of ‘contractual trust’, such that trust between a user and an AI model is trust that some implicit or explicit contract will hold, and a formalization of ‘trustworthiness’ (that detaches from the notion of trustworthiness in sociology), and with it concepts of ‘warranted’ and ‘unwarranted’ trust. We present the possible causes of warranted trust as intrinsic reasoning and extrinsic behavior, and discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted. Finally, we elucidate the connection between trust and XAI using our formalization.

Keywords: trust, distrust, trustworthy, warranted trust, contractual trust, artificial intelligence, sociology, formalization

人工智能中信任的形式化： 人工智能中人类信任的前提、原因和目标

Alon Jacov, Ana Marasović, Ana Marasović, Yoav Goldberg

摘要：信任是人与人工智能互动的核心要素，因为 "不正确 "的信任水平可能导致技术的误用、滥用或废弃。但人工智能中信任的本质究竟是什么？信任认知机制的先决条件和目标是什么，我们又该如何促进这些先决条件和目标的实现，或评估在特定互动中这些先决条件和目标是否得到了满足？这项工作旨在回答这些问题。我们讨论的信任模型受到社会学家定义的人际信任（即人与人之间的信任）的启发，但与之并不相同。该模型基于两个关键属性：用户的脆弱性和预测人工智能模型决策影响的能力。我们纳入了 "契约信任 "的形式化，即用户与人工智能模型之间的信任是对某种隐含或明确的契约将成立的信任，以及 "可信度 "的形式化（脱离了社会学中的可信度概念），并纳入了 "有正当理由 "和 "无正当理由 "信任的概念。我们将有理由信任的可能原因归结为内在推理和外在行为，并讨论了如何设计值得信任的人工智能、如何评估信任是否已经显现以及是否有理由信任。最后，我们用形式化的方法阐明了信任与 XAI 之间的联系。

关键词：信任、不信任、值得信任、担保信任、契约信任、人工智能、社会学、形式化

来源：arXiv:2010.07487v3 [cs.AI] 20 Jan 2021