Ethics Analysis of Predictive Algorithm Affecting Public Safety
The challenge: How should our client identify ethical risks of implementing a predictive algorithm to complement part of their existing operations?
Design Team: Generation R
Main Client: Technical Safety BC
The case study and the full report could be accessed through the provided link.
The approach:
- Developed a novel method of investigating ethical risks of data-driven systems using a combination of algorithm analysis, activity system mapping and value-sensitive design
- Discovered potential challenges and tension points that our client could face as they deploy their data-driven system
- Provided technology, business, and policy design recommendations
Key learning or insights: There were many insights. As an example, one of the most interesting insights was understanding how values such as fairness are defined differently for employees in different roles (i.e. the inspectors versus vice president of operations).
The impact of findings: Our findings and recommendations have shaped the deployment of this technology and we are continuing to work closely with our client and learning as they implement our recommendations.
My specific role: I was fully involved with the entirety of this project. However, my key contributions included the development of a component of the process for tackling this challenge and conducting the qualitative study.