(In)Justice in the Age of Information

Note: Many of the ideas below were adapted from their original form in Cathy O’Neil’s “Weapons of Math Destruction”, which discusses the devastating impacts of statistical models. If you’re interested in learning more about how your life gets turned into data, I highly recommend it.

The Big Data revolution has promised changes in many domains, from the economy to education. Companies such as Google and Facebook use complex statistical models to tailor content towards individual users, and even the U.S. government is beginning to use data science to inform public policy and legislation.

But not all these changes are for the better. In some cases, reliance on statistical models contributes to gross injustices. This is particularly evident in the recidivism models used for prison sentencing.

What is Recidivism?

Recidivism is the likelihood of a convicted criminal to reoffend, once they are released from prison. A recidivism model, then, predicts how likely it is that any given individual will commit another crime if they are released; this likelihood is called a recidivism score. The model uses background information about the person (such as where they grew up), as well as their answers to a test (featuring questions such as, “how many of your family members have been or are currently incarcerated?”), to produce this score. A judge then relies on the recidivism score to determine the length and severity of an individual’s prison sentence.

The basic motivation behind measuring recidivism is reasonable. One of the functions of incarceration is ostensibly to protect society; thus, if it seems likely that someone will commit more crimes when they are released, it might make sense to forestall their reentry into society. Psychopaths, for example, are much more likely to reoffend (Harris et al, 1991), which generally results in longer prison sentences for individuals that score high enough on the Psychopathy Checklist[1]. So with the goal of protecting society in mind, it seems reasonable to tailor prison sentence length to an individual’s recidivism score – if, that is, we were confident about the accuracy of these recidivism scores.

The problem – as usual – is that it’s not that simple.

The Injustice of Justice

Let’s take a brief detour to discuss a more well-known, and highly controversial, topic: Stop-and-Frisk. Stop-and-Frisk is a policy in NYC that allows police to detain suspicious-looking[2] individuals and pat them down for evidence of illegal activity, such as a concealed weapon or illegal drugs. As with the recidivism model presented above, this might seem logical at face-value: after all, if someone is illegally carrying a concealed weapon, they might present a danger to the community. Thus, it makes sense to arrest them and confiscate the weapon.

The problem is that Stop-and-Frisk – and many implementations of “predictive policing” in general – is fundamentally racist. A meta-analysis of Stop-and-Frisk encounters showed that policemen were disproportionately stopping African-American and Latino men, resulting in disproportionate arrests of those individuals. Statistically, if police stop 1000 people in a day, there’s a high likelihood that one of them will be doing something illegal – and if primarily African-American and Latino men are being stopped, then it’s primarily African-American and Latino men who will be arrested[3].

Even worse, these new arrests actually feed into a policeman’s model – whether formal or informal – of who to detain. After all, if more African-American and Latino men have been arrested, then surely it makes sense to detain African-American and Latino men. This creates what Cathy O’Neil terms a pernicious feedback loop: a model whose biases can only be exacerbated, and which never learns from its mistakes. And its victims are then introduced to the revolving door of America’s justice system.

A Self-Fulfilling Prophecy

So now we return to the problem of recidivism scores. There are a number of problems built into the score, including the fact that ZIP code can be used as a proxy for race, and the types of questions asked in the recidivism exam are generally the types of questions that would be totally out-of-place in a courtroom. But the most glaring problem of the current usage of recidivism scores is that they effectively become a self-fulfilling prophecy.

Studies (Mueller-Smith et al, 2016; Mueller-Smith, 2015) reliably demonstrate a positive correlation between the time one spends in prison and the likelihood of re-offense. That is, the longer your prison sentence is, the more likely it is that you’ll commit another crime when you’re released. There are all sorts of explanations for why this could be. For example, it’s possible that spending more time among other convicted criminals affects your behavior and psychology, making you more of a “criminal” than before. More generally, it’s possible that the adverse conditions in many prisons could you to adopt a more aggressive attitude and become accustomed to solving conflicts through violence or intimidation. Or perhaps, after five to ten years in prison, your primary “social network” becomes other prisoners, each of whom is potentially involved in criminal activity, thus increasing your likelihood of becoming involved in criminal activity[4]. And even if you avoid all of these effects, you’re still reentering society with a felony on your record and 5-10 years of served prison time, meaning that many jobs are simply unavailable to you[5][6].

Any (or all) of the above seem like reasonable explanations for the relationship between time spent in prison and one’s likelihood to reoffend. Now, we still don’t know which is the strongest predictor, nor exactly how to characterize the relationship between sentence length and likelihood to reoffend. A counterargument would suggest the following: of course people with longer prison sentences have higher recidivism rates – they were given longer prison sentences because of the predicted recidivism score! This line of reasoning would argue, then, that the recidivism score is simply an accurate prediction. That means we should continue using the recidivism score to determine prison sentence length, right?

There are a couple of flaws with the counterargument. First of all, while it’s true that the direction of causality is extremely difficult to tease apart, the core of the counterargument is that the recidivism score is “correct” because people with higher recidivism scores end up being more likely to reoffend. But this ignores the potentially mediating variable: time spent in prison. If you think that time spent in prison could possibly affect actual recidivism, then that means you’re confounding the relationship between recidivism score (e.g. a “prediction”) and actual recidivism by explicitly taking action to affect that relationship (increasing the time spent in prison). This means you can’t be confident in the accuracy of your recidivism score. To be confident, you’d need to tease apart the three relevant variables:

  1. Recidivism score (the prediction)
  2. Time spent in prison (the potentially intervening variable)
  3. Actual recidivism (a binary variable: did they reoffend or not?)

One could do this experimentally, although that’d be extremely unethical. However, it’s possible that data currently exists to probe this relationship. One thing you could do is find cases in which a bunch of people with the same recidivism score received different prison sentences; this would allow you to better isolate the effect of sentence length on actual recidivism. Alternatively, you could find cases in which people with different recidivism scores received the same prison sentences; this would allow you to better analyze the accuracy of predicted recidivism[7].

The second problem with the counterargument is a bit deeper, and speaks to issues with how we view the justice system in general. Our treatment of convicted criminals reflects a particularly fixed attitude towards morality and behavior; that is, we tend to opt for incarceration as a means of protecting the rest of society, rather than rehabilitating the individual. This is true even when someone is released from prison. The word “felon” becomes a brand of sorts, and they are denied many of the basic rights that others in society enjoy. This begs the question: if someone is more likely to commit another crime, why is the correct solution to “lock them up”? Why do we not instead look at which systemic or individual factors led to the crime? Why do we automatically infer that they are a “bad person”, and not simply someone who made a mistake?

The truth is that we do sometimes consider criminal behavior through the lens of rehabilitation, but from my own observations, this seems to happen far more when the individual is relatively privileged in society (white, upper-income brackets, etc.). This speaks to an earlier point I made about the vastly differential treatment of so-called “white collar crime” by policemen; white collar crime, though devastating on the economy, generally receives little attention from law enforcement.

Actionability: Or, What Can We Do?

All that being said, is there anything we can do?

We can express opinions, of course – for example, I think it’s disgraceful that these recidivism scores currently play such a large role in determining prison sentence length, especially when we don’t really understand how accurate they are and what the effects of prison time are on actual recidivism. This path is effective insofar as there are people who are willing to listen to our opinion, and perhaps propagate it through their own social networks. In other words, it’s effective in the way that anything that stimulates discussion on a topic is effective.

But stimulating discussion, while perhaps a prerequisite for change, does not necessarily lead to it. More effective are organized movements, with clearly defined motivations and demands (such as “Ban the Box”), which ideally have some sort of ethical or economic leverage to elicit change. Leverage is key, because without it, those in power have little reason to pay any attention to your demands. Unfortunately, I’m by no means experienced with activism or the tools with which people acquire political leverage; on the other hand, I’ve gotten more and more exhausted with the trend of cynical defeatism and self-admitted impotence, which I think is probably its own self-fulfilling prophecy.

For me, as a scientist, the clearest line of actionability is for scientists to investigate more closely the effects of prison time on individual behavior. More research leads to a clearer understanding, which necessarily shapes the nature of our demands. And as a citizen, my current strategy primarily involves the “stimulate discussion” path, though I recognize that this is in many ways non-optimal. In terms of concrete activism, perhaps the strongest action one can take is to promote the “Ban the Box” movement; this isn’t directly related to problems with the recidivism model, of course, but since it’s possible that inability to find work can increase recidivism, it does seem relevant – and it’s an important cause regardless.

If this seems like an unsatisfying conclusion, that’s because it is. This is a real issue that affects real people, and it’s quite difficult to know exactly what to do to address the issue. If you have ideas, I’m all ears.


Harris, G. T., Rice, M. E., & Cormier, C. A. (1991). Psychopathy and violent recidivism. Law and Human Behavior, 15(6), 625–637. http://doi.org/10.1007/BF01065856

Mueller-smith, M., & Schnepel, K. T. (2016). Punishment and (non-) Deterrence: Evidence on First-Time Drug Offenders from Regression Discontinuities.

Mueller-Smith, M. (2015). The criminal and labor market impacts of incarceration.


[1] We’ll ignore, for the moment, the inherent difficulties in formalizing or quantifying a notion as elusive as “psychopathy” – or even non-normative pathology in general – as this is a discussion that surely merits its own post.

[2] Importantly, the policy replaces the notion of probable cause with reasonable suspicion.

[3] Note, too, that the kinds of illegal activity Stop-and-Frisk targets is generally associated with poorer neighborhoods, and often involves minor offenses like possession of marijuana. In Weapons of Math Destruction, Cathy O’Neil invites the reader to imagine “Stop-and-Frisk” applied to white collar crime: policemen infiltrating the offices of mortgage loan originators and Wall Street traders, demanding audits and uncovering fraud. Sadly, our justice system is out of balance; a marijuana dealer might receive time in prison and a felony on his record, while an investment banker – who played a role in the 2008 financial crisis, perhaps – walks away essentially scot-free. And sure, people groan at the injustice of it all, but there’s still a sense of normality to it, as if that’s the only way it could be.

[4] One of the most damning things about the recidivism score’s implementation is that one of the factors it takes into account is an individual’s social network. So now let’s imagine someone who is sentenced to 5 years in prison, and when he gets out, his main contacts are the community he built while in prison. Let’s say he gets busted for something relatively minor, like selling marijuana; now his social network consists of a bunch of other de facto criminals, so his recidivism score will likely go through the roof.

[5] Though the “Ban the Box” is attempting to change this, by restricting whether employers can ask about a person’s criminal history (e.g. “are you a convicted felon”?).

[6] Plus you can’t vote.

[7] I should note that neither of these studies, if carried out exactly as described, would likely pass peer-review at any reputable journals. There are all sorts of other tricky variables to control for, and it’s also possible that I’m missing something pretty fundamental in the design. It’s also possible that such a study has been done already, in which case I’d love to see the results.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s