CUED Publications database

Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty

Bhatt, U and Antorán, J and Zhang, Y and Liao, QV and Sattigeri, P and Fogliato, R and Melançon, G and Krishnan, R and Stanley, J and Tickoo, O and Nachman, L and Chunara, R and Srikumar, M and Weller, A and Xiang, A (2021) Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty. AIES 2021 - Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. pp. 401-413.

Full text not available from this repository.

Abstract

Algorithmic transparency entails exposing system properties to various stakeholders for purposes that include understanding, improving, and contesting predictions. Until now, most research into algorithmic transparency has predominantly focused on explainability. Explainability attempts to provide reasons for a machine learning model's behavior to stakeholders. However, understanding a model's specific behavior alone might not be enough for stakeholders to gauge whether the model is wrong or lacks sufficient knowledge to solve the task at hand. In this paper, we argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions. First, we discuss methods for assessing uncertainty. Then, we characterize how uncertainty can be used to mitigate model unfairness, augment decision-making, and build trustworthy systems. Finally, we outline methods for displaying uncertainty to stakeholders and recommend how to collect information required for incorporating uncertainty into existing ML pipelines. This work constitutes an interdisciplinary review drawn from literature spanning machine learning, visualization/HCI, design, decision-making, and fairness. We aim to encourage researchers and practitioners to measure, communicate, and use uncertainty as a form of transparency.

Item Type: Article
Uncontrolled Keywords: cs.CY cs.CY cs.HC cs.LG
Subjects: UNSPECIFIED
Divisions: Div F > Computational and Biological Learning
Depositing User: Cron Job
Date Deposited: 27 Nov 2020 20:10
Last Modified: 07 Sep 2021 02:18
DOI: 10.1145/3461702.3462571