The Debiasing Paradox: Facial Recognition Technology and Biometric Identification Systems in the Artificial Intelligence Act

Abstract

Decision-making is no longer delegated to humans, but to algorithms to produce accurate results based on neutral presumptions. Biometric identification systems build on algorithms to produce decisions on an individual via identification or verification. One of the most common uses of biometric identification systems is Facial Recognition Technology (FRT) deployed by governments and authorities for law enforcement or by private companies in a myriad of applications. Despite its original design around the idea of accuracy, the real-life deployment of FRT (and biometric identification systems at large) has demonstrated that bias is inherently embedded into its functioning, with the potential to pose serious risks relating to direct and indirect manifestations of discrimination. Against this framework, the chapter considers the concept of the debiasing paradox to address the disconnect between the capacity to curtail bias in machine learning before the historical and systematic biases which may prevail even if debiasing is applied as a neutral solution. This chapter explains the functioning of FRT under the lens of the European Union legislator’s approach to regulating biometric identification systems. In light of the debiasing paradox, the chapter proposes a human rights-based approach to tackle bias in FRT both from the formal and substantive equality perspectives.