Overtrust in AI Recommendations to Kill

C Holbrook, D Holman, J Clingo, A Wagner

Abstract: This study explores prospective determinants of trust in the recommendations of artificial agents regarding decisions to kill, using a novel visual challenge paradigm simulating threatidentification (enemy combatants vs. civilians) under uncertainty. In Experiment 1, we compared trust in the advice of a physically embodied versus screen-mediated anthropomorphic robot, observing no effects of embodiment; in Experiment 2, we manipulated the relative anthropomorphism of virtual robots, observing modestly greater trust in the anthropomorphic agents. Across studies, when any version of the agent randomly disagreed, participants reversed their threat-identifications and decisions to kill in the majority of cases, substantially degrading their initial performance. Participants’ subjective confidence in their decisions tracked whether the agent (dis)agreed, while both decision-reversals and confidence were moderated by appraisals of the agent’s intelligence. The overall findings indicate a strong propensity to overtrust unreliable AI in life-or-death decisions made under uncertainty.

Keywords: artificial intelligence, human-robot interaction, human-computer interaction, social robotics, decision-making, threat-detection, anthropomorphism

过度信任人工智能的致命建议

C Holbrook, D Holman, J Clingo, A Wagner

摘要：本研究使用一个新颖的视觉挑战范例，模拟不确定情况下的威胁识别（敌方战斗人员与平民），探讨了信任人工代理有关杀戮决定的建议的前瞻性决定因素。在实验 1 中，我们比较了对实体拟人机器人和以屏幕为媒介的拟人机器人建议的信任度，结果发现实体拟人机器人不会产生任何影响；在实验 2 中，我们操纵了虚拟机器人的相对拟人化程度，结果发现拟人机器人的信任度略高。在所有的研究中，当任何版本的代理随机出现分歧时，参与者在大多数情况下都会推翻他们的威胁识别和杀人决定，从而大大降低了他们最初的表现。参与者对其决定的主观信心与代理是否（不）同意有关，而决定逆转和信心都受代理智力评价的调节。总体研究结果表明，在不确定的情况下做出生死攸关的决定时，人们倾向于过度信任不可靠的人工智能。

关键词：人工智能、人机交互、人机交互、社交机器人、决策、威胁检测、拟人化

来源：Holbrook,C. , Holman, D., Clingo, J., & Wagner, A.(2023). Overtrust in AI Recommendations to Kill.