When the stakes are high: new study finds people distrust single AI models and want human oversight when algorithms disagree
A new study by computer scientists at the University of California San Diego and the University of Wisconsin–Madison warns that relying on a single “best” machine learning (ML) model for high‑stakes decisions — from loan approvals to hiring — can undermine perceived fairness, and that ordinary people prefer human arbitration when equally good models disagree. The research, presented at the 2025 ACM CHI conference, explored how lay stakeholders react when multiple high‑accuracy models reach different conclusions for the same applicant and found strong resistance to both single‑model arbitrariness and to solutions that simply randomize outcomes; instead participants favored wider model searches, transparency and human decision‑making to resolve disagreements UC San Diego report and the authors’ paper Perceptions of the Fairness Impacts of Multiplicity in Machine Learning (CHI 2025) presents the detailed results and recommendations.