Algorithmic Bias in Criminal Justice

Algorithmic bias in criminal justice refers to the systematic and often discriminatory errors that arise when automated decision-making systems — including risk assessment instruments, predictive policing platforms, facial recognition technologies, and sentencing recommendation tools — produce outcomes that disproportionately affect individuals on the basis of race, ethnicity, socioeconomic status, or other protected characteristics. Within Cyber Criminology, algorithmic bias represents a critical concern at the intersection of digital technology and criminal justice, as the rapid adoption of data-driven tools across policing, courts, and corrections has introduced automated systems into decisions about liberty, punishment, and public safety that were previously made through human judgment alone. Research by ProPublica (Angwin, Larson, Mattu, & Kirchner, 2016), the AI Now Institute (Richardson, Schultz, & Crawford, 2019), and the National Institute of Standards and Technology (Grother, Ngan, & Hanaoka, 2019) has documented significant disparities in the performance of criminal justice algorithms across racial and demographic groups, raising fundamental questions about fairness, accountability, and the appropriate role of automated systems in the administration of justice. This article examines the sources of algorithmic bias, its manifestations across criminal justice domains, the competing definitions of algorithmic fairness, institutional responses, and policy proposals within the broader field of Criminology.

Introduction

The integration of algorithmic tools into criminal justice decision-making reflects a broader societal trend toward data-driven governance, motivated by the promise that quantitative analysis can improve the accuracy, consistency, and efficiency of decisions that have traditionally relied on human judgment. Risk assessment instruments predict the probability that a defendant will fail to appear for trial or commit new offenses, informing pretrial release and sentencing decisions. Predictive policing platforms forecast where crime is likely to occur, guiding patrol deployment. Facial recognition systems identify suspects from surveillance footage. Each of these tools applies statistical or machine learning models to data in order to generate predictions that inform consequential decisions about individuals’ lives.

The problem of algorithmic bias arises because these systems are not neutral: they are products of the data on which they are trained, the features they are designed to optimize, and the assumptions embedded in their design choices (Barocas & Selbst, 2016). When the data reflect historical patterns of discrimination — as criminal justice data inevitably do, given the well-documented racial disparities in policing, prosecution, and sentencing in the United States (Alexander, 2010; Tonry, 2011) — algorithms trained on those data may reproduce and amplify those disparities, embedding historical bias into systems that present themselves as objective and data-driven. The consequence is that algorithmic tools may systematically disadvantage racial minorities, low-income individuals, and other marginalized populations in decisions about arrest, detention, sentencing, and surveillance, potentially violating constitutional guarantees of equal protection and due process (Ferguson, 2017; Starr, 2014).

Sources of Algorithmic Bias

Data-Driven Bias

The most fundamental source of algorithmic bias in criminal justice is the data on which algorithms are trained and validated. Criminal justice data — arrest records, conviction histories, incident reports, victimization surveys — reflect the cumulative product of human decisions that are themselves shaped by racial bias, resource allocation priorities, and institutional practices that produce disparate outcomes across demographic groups (Lum & Isaac, 2016). If police departments have historically concentrated enforcement in communities of color — deploying more officers, conducting more stops, making more arrests for offenses (such as drug possession) that occur at similar rates across racial groups — the resulting arrest data will show higher crime rates in those communities even if the underlying behavior is not more prevalent. An algorithm trained on this data will predict higher crime risk in those communities and direct further enforcement there, creating a feedback loop that perpetuates and amplifies the original disparity.

Lum and Isaac (2016) demonstrated this dynamic empirically by applying a predictive policing algorithm to historical drug crime data from Oakland, California. The algorithm predicted crime hot spots that corresponded closely to areas of concentrated police enforcement — disproportionately communities of color — rather than to the distribution of drug use as measured by public health surveys, which showed relatively even distribution across racial groups. The analysis illustrated how predictive algorithms can transform enforcement patterns into predicted crime patterns, lending a veneer of scientific objectivity to decisions that perpetuate racial disparity.

The problem extends beyond policing data to the variables that risk assessment instruments use as predictors. Prior arrests, employment history, residential stability, educational attainment, and social network characteristics — all commonly used risk factors — are correlated with race and socioeconomic status through structural mechanisms that reflect systemic inequality rather than individual criminogenic risk (Harcourt, 2007). An algorithm that uses these variables as inputs will produce racially disparate outputs not because it explicitly considers race but because its input variables serve as proxies for racial and economic status — a phenomenon that computer scientists term “proxy discrimination” (Dwork, Hardt, Pitassi, Reingold, & Zemel, 2012).

Design and Optimization Choices

Algorithmic bias is also produced through the design choices that developers make in constructing, training, and validating criminal justice tools. The selection of the outcome variable — what the algorithm is trained to predict — shapes its outputs in ways that may embed bias. A pretrial risk assessment trained to predict “failure to appear” will produce different risk scores than one trained to predict “arrest for a new offense,” because the two outcomes have different distributions across demographic groups and different relationships to the input variables. The choice of which outcome to optimize reflects a policy judgment that algorithm developers make, often without transparent deliberation about the equity implications of the choice (Chouldechova, 2017).

The validation of algorithmic accuracy raises further bias concerns. An algorithm may be “accurate” in the aggregate — correctly classifying a high proportion of cases — while exhibiting significant accuracy disparities across subgroups. Chouldechova (2017) demonstrated mathematically that when base rates (the actual prevalence of the outcome being predicted) differ across groups — as they do for most criminal justice outcomes — it is impossible to simultaneously satisfy multiple widely accepted fairness criteria, including equal false positive rates and equal false negative rates across groups. This impossibility result means that any algorithm operating on data with different base rates across groups will produce disparate error rates for at least one group, regardless of how it is designed — a finding with profound implications for the use of algorithmic tools in a criminal justice system characterized by pervasive racial disparities in outcomes.

Manifestations Across Criminal Justice Domains

Pretrial Risk Assessment

Pretrial risk assessment instruments — including the Public Safety Assessment (PSA), the Virginia Pretrial Risk Assessment Instrument (VPRAI), and numerous proprietary tools — have been adopted by jurisdictions across the United States as alternatives to cash bail systems, with the goal of replacing wealth-based detention decisions with evidence-based risk assessments. The tools assign risk scores based on factors such as prior criminal history, age, and pending charges, generating recommendations about whether a defendant should be released pretrial and under what conditions.

The ProPublica investigation of the COMPAS risk assessment tool, published in 2016, brought algorithmic bias in criminal justice to national attention. Angwin et al. (2016) analyzed COMPAS scores for over 7,000 defendants in Broward County, Florida, and found that the tool was approximately twice as likely to incorrectly label Black defendants as high risk (false positives) compared to white defendants, while being approximately twice as likely to incorrectly label white defendants as low risk (false negatives). Northpointe (now Equivant), the developer of COMPAS, contested ProPublica’s analysis, arguing that the tool exhibited predictive parity — equal accuracy in predicting recidivism among those classified as high risk — across racial groups (Dieterich, Mendoza, & Brennan, 2016). The dispute illustrated the impossibility result demonstrated by Chouldechova: COMPAS could not simultaneously achieve equal false positive rates and equal predictive values across groups with different base rates, and the choice of which fairness metric to prioritize is a normative judgment rather than a technical determination.

Predictive Policing

Predictive policing systems, discussed in detail in the article on Digital Surveillance and Crime Control within this category, illustrate how algorithmic bias operates at the spatial level. Place-based predictive systems such as PredPol (now Geolitica) and HunchLab analyze historical crime data to generate geographic predictions about where crime is likely to occur. When the historical data reflects enforcement patterns rather than crime distribution — as Lum and Isaac (2016) demonstrated for drug crime — the algorithms direct police to communities that have been historically over-policed, generating additional enforcement activity that produces the data confirming the algorithm’s predictions (Brayne, 2021; Ferguson, 2017).

The Los Angeles Police Department’s discontinuation of PredPol in 2020, following a report by the Inspector General documenting racial disparities in the tool’s deployment and an audit by the RAND Corporation that found limited evidence of crime reduction effectiveness, represented one of the most significant institutional responses to concerns about algorithmic bias in policing (LAPD Inspector General, 2020). Other jurisdictions — including New Orleans, which discontinued its predictive policing program in response to community opposition — have similarly retreated from algorithmic policing tools, though many agencies continue to use these or similar systems.

Facial Recognition

Facial recognition technology introduces a distinct form of algorithmic bias that operates through differential accuracy across demographic groups rather than through racially disparate inputs. The NIST Face Recognition Vendor Test (Grother et al., 2019) evaluated 189 facial recognition algorithms from 99 developers and found that the majority exhibited significantly higher false positive rates for African American and Asian faces compared to Caucasian faces, and for women compared to men. The disparities were substantial: some algorithms exhibited false positive rates 10 to 100 times higher for African American faces than for Caucasian faces, creating a risk of misidentification that falls disproportionately on communities of color.

The documented cases of wrongful arrest resulting from facial recognition misidentification — including the cases of Robert Williams, Michael Oliver, and Nijeer Parks in the Detroit area — demonstrate the real-world consequences of algorithmic accuracy disparities. Each case involved a Black man who was identified by facial recognition as a suspect, arrested, and detained before the misidentification was discovered (Hill, 2020). These cases have prompted legislative action at the municipal level (bans in San Francisco, Boston, and Portland), state level (restrictions in Washington, Massachusetts, and others), and federal level (proposed but not enacted legislation), reflecting a growing recognition that facial recognition’s accuracy disparities create unacceptable risks of discriminatory harm.

Fairness Frameworks and Competing Definitions

Mathematical Definitions of Fairness

The computer science and legal scholarship on algorithmic fairness has produced multiple competing definitions of what it means for an algorithm to treat individuals equitably, with no consensus on which definition should govern criminal justice applications. The major fairness criteria include calibration (equal predictive accuracy across groups — among those scored as high risk, equal proportions actually recidivate), classification parity (equal error rates across groups — equal false positive and false negative rates), and individual fairness (similar individuals receive similar scores regardless of group membership) (Corbett-Davies & Goel, 2018).

Chouldechova’s (2017) impossibility theorem demonstrated that calibration and classification parity cannot be simultaneously achieved when base rates differ across groups — a condition that holds for virtually every criminal justice outcome. This mathematical result means that developers and policymakers must choose which fairness criterion to prioritize, a choice that involves normative judgments about the relative harms of different types of errors. A system that prioritizes equal false positive rates protects against the disproportionate over-prediction of risk for minority defendants but may sacrifice predictive accuracy. A system that prioritizes calibration maintains accuracy but tolerates disparate error rates. The choice between these approaches reflects competing conceptions of fairness that cannot be resolved through technical means alone (Barabas, Virza, Dinakar, Ito, & Zittrain, 2018).

Legal and Constitutional Dimensions

The constitutional implications of algorithmic bias in criminal justice are addressed through the Equal Protection Clause of the Fourteenth Amendment, the Due Process Clause, and statutory protections against discrimination. Equal protection challenges to algorithmic tools face the substantial hurdle of Washington v. Davis (1976), which requires proof of discriminatory intent rather than discriminatory impact for constitutional claims — a standard that is difficult to satisfy when the alleged discrimination results from an algorithm’s processing of facially neutral inputs rather than from deliberate racial targeting.

Due process challenges focus on the transparency and accountability of algorithmic decision-making. In State v. Loomis (2016), the Wisconsin Supreme Court upheld the use of COMPAS in sentencing while imposing procedural requirements: the court held that COMPAS scores could not be used as the determinative factor in sentencing, that defendants must be informed of the limitations of risk assessment tools, and that sentencing courts must consider the tool’s documented disparities. The Loomis decision represents the most significant judicial engagement with algorithmic bias in criminal sentencing and establishes a framework of cautious permissibility coupled with procedural safeguards — an approach that has influenced but not standardized judicial treatment of risk assessment tools across jurisdictions (Hamilton, 2019).

Institutional Responses and Policy Proposals

Transparency and Auditing

The demand for transparency in criminal justice algorithms — the ability of defendants, courts, and the public to understand how algorithmic tools generate their outputs — has become a central reform theme. Proprietary risk assessment tools such as COMPAS have been criticized for operating as “black boxes” whose internal logic is shielded by trade secret protections, preventing defendants from meaningfully challenging the scores used against them (Rudin, 2019). The tension between proprietary protection and due process transparency has generated litigation, legislative proposals, and advocacy for the adoption of interpretable (non-black-box) models that produce outputs whose logic can be explained and scrutinized.

Algorithmic auditing — the systematic evaluation of algorithmic tools for accuracy, bias, and compliance with fairness standards — has been proposed and in some cases mandated as a mechanism for ensuring accountability. New York City’s Local Law 144 (2021) requires bias audits of automated employment decision tools, establishing a precedent that may be extended to criminal justice applications. The federal government has not mandated algorithmic auditing for criminal justice tools, though the National Institute of Justice has funded research on the validation and fairness assessment of risk assessment instruments, and the Sentencing Commission has examined the role of algorithmic tools in federal sentencing.

Alternatives and Reforms

Reform proposals range from incremental modifications to existing algorithmic systems through to the elimination of algorithmic tools from criminal justice decision-making. Incremental approaches include the removal of racially correlated variables from risk assessment inputs, the adoption of fairness-constrained optimization that penalizes racial disparities in model outputs, and the implementation of ongoing monitoring and recalibration programs that detect and correct bias as it emerges. More fundamental proposals advocate for the replacement of algorithmic risk assessment with structured professional judgment — decision-making frameworks that guide human discretion through checklists and criteria without relying on statistical prediction — or for the elimination of risk-based decision-making in favor of presumptive pretrial release and sentence guidelines that do not consider individualized risk (Stevenson & Doleac, 2022).

The debate over algorithmic tools in criminal justice ultimately concerns the appropriate relationship between data-driven prediction and human judgment in decisions about liberty and punishment. Proponents argue that algorithmic tools, despite their limitations, produce more consistent and accurate predictions than unstructured human judgment, and that their disparities can be identified and addressed through technical and procedural reforms (Kleinberg, Lakkaraju, Leskovec, Ludwig, & Mullainathan, 2018). Critics respond that the translation of human decision-making into algorithmic form obscures the normative choices embedded in predictive systems, transfers responsibility from accountable human decision-makers to opaque technical processes, and lends an undeserved aura of objectivity to predictions that reflect and perpetuate structural inequality (Eubanks, 2018; O’Neil, 2016).

Conclusion

Algorithmic bias in criminal justice represents a challenge that is simultaneously technical, legal, and political — rooted in the data that reflect historical discrimination, the design choices that shape algorithmic outputs, and the institutional contexts in which algorithmic tools are deployed. The evidence for significant racial and demographic disparities in the performance of criminal justice algorithms is well-established across risk assessment, predictive policing, and facial recognition applications, and the mathematical impossibility of simultaneously satisfying competing fairness criteria means that no technical fix can fully resolve the problem. The most productive path forward requires the combination of transparency requirements that enable scrutiny of algorithmic tools, auditing mechanisms that detect and document bias, procedural safeguards that prevent algorithmic scores from becoming determinative of outcomes, and ongoing public deliberation about whether and under what conditions automated systems should inform decisions about liberty, punishment, and public safety.

References

Alexander, M. (2010). The new Jim Crow: Mass incarceration in the age of colorblindness. New Press.
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016, May 23). Machine bias. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Barabas, C., Virza, M., Dinakar, K., Ito, J., & Zittrain, J. (2018). Interventions over predictions: Reframing the ethical debate for actuarial risk assessment. Proceedings of Machine Learning Research, 81, 1–15.
Barocas, S., & Selbst, A. D. (2016). Big data’s disparate impact. California Law Review, 104(3), 671–732. https://doi.org/10.15779/Z38BG2H
Brayne, S. (2021). Predict and surveil: Data, discretion, and the future of policing. Oxford University Press.
Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153–163. https://doi.org/10.1089/big.2016.0047
Corbett-Davies, S., & Goel, S. (2018). The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023.
Dieterich, W., Mendoza, C., & Brennan, T. (2016). COMPAS risk scales: Demonstrating accuracy equity and predictive parity. Northpointe Research Division.
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (pp. 214–226). https://doi.org/10.1145/2090236.2090255
Eubanks, V. (2018). Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.
Ferguson, A. G. (2017). The rise of big data policing: Surveillance, race, and the future of law enforcement. NYU Press.
Grother, P., Ngan, M., & Hanaoka, K. (2019). Face Recognition Vendor Test (FRVT) Part 3: Demographic effects. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8280
Hamilton, M. (2019). The biased algorithm: Evidence of disparate impact on Hispanics. American Criminal Law Review, 56(4), 1553–1582.
Harcourt, B. E. (2007). Against prediction: Profiling, policing, and punishing in an actuarial age. University of Chicago Press.
Hill, K. (2020, June 24). Wrongfully accused by an algorithm. New York Times. https://www.nytimes.com/2020/06/24/technology/facial-recognition-arrest.html
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018). Human decisions and machine predictions. Quarterly Journal of Economics, 133(1), 237–293. https://doi.org/10.1093/qje/qjx032
LAPD Inspector General. (2020). Review of selected Los Angeles Police Department data-driven policing strategies. Office of the Inspector General.
Lum, K., & Isaac, W. (2016). To predict and serve? Significance, 13(5), 14–19. https://doi.org/10.1111/j.1740-9713.2016.00960.x
O’Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Crown.
Richardson, R., Schultz, J. M., & Crawford, K. (2019). Dirty data, bad predictions: How civil rights violations impact police data, predictive policing systems, and justice. New York University Law Review Online, 94, 15–55.
Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215. https://doi.org/10.1038/s42256-019-0048-x
Starr, S. B. (2014). Evidence-based sentencing and the scientific rationalization of discrimination. Stanford Law Review, 66(4), 803–872.
State v. Loomis, 881 N.W.2d 749 (Wis. 2016).
Stevenson, M. T., & Doleac, J. L. (2022). Algorithmic risk assessment in the hands of humans. IZA Discussion Paper No. 12853.
Tonry, M. (2011). Punishing race: A continuing American dilemma. Oxford University Press.
Washington v. Davis, 426 U.S. 229 (1976).

Related Articles