Skip to main content
xYOU DESERVE INDEPENDENT, CRITICAL MEDIA. We want readers like you. Support independent critical media.

Algorithms are Just as Biased and Wrong as Humans

A new study looked at the accuracy and bias of algorithms being used to make risk assessments for criminal defendants. It concluded that algorithmic models are just as bad as humans.
US criminal justice system

Newsclick Image by Shoili

The US criminal justice system has increasingly been relying on algorithms to make risk assessments. The scores of these risk assessments go on to play an integral role in making crucial decisions about accused people, such as if an undertrial should be released on bail or not. One would expect these algorithms to be foolproof and free from bias for being chosen to make such important decisions. But a new study shows that these algorithms are just as biased and inaccurate as human beings with little or no experience in handing out sentences.

Julia Dressel, a computer science major at Dartmouth College, carried out this study. Julia recruited 400 research volunteers from Amazon Mechanical Turk. Using ProPublica’s database having records of 10,000 defendants awaiting trial and their arrest records for the next two years, Julia compared the accuracy of humans and machines. The volunteers were given profiles of 50 defendants each and were asked to predict if the defendant would be re-arrested in the next two years. The results were compared to the accuracy of COMPAS (Correctional Offender Management Profiling for Alternative Sanctions), a commonly used risk assessment tool in US, known for its bias against people of colour.

Julia found that both humans and COMPAS had similar accuracies. The research volunteers were correctly able to predict if a defendant will be arrested in the next two years 63-67% of the time, compared to COMPAS having an accuracy of 65%.

To make the risk assessment, COMPAS selects six factors from a list of 137 questions. The answers to these questions are taken from criminal records, or answered by the defendants. The list includes questions such as if the defendant’s parents were ever incarcerated, or how many of the defendant’s friends take illegal drugs, or how often did the defendant get in fights in school. They do not include the defendant’s race. Julia and her co-author, Hany Farid, created another model which used only two factors: a defendant’s age and previous convictions. The accuracy of their new model (67%) was about the same as COMPAS.

COMPAS’ scores incorrectly label black people as being a higher risk about 40% of the time; the algorithm incorrectly labels white people as being of lower risk about 48% of the time. This was again similar to how many times humans made biased decisions in Julia’s study. The volunteers incorrectly judged black people to be a higher risk 37% of the time, and white people to be of lower risk 40% of the time. While race is not a factor being explicitly considered, a bias has been coded into the algorithm.

These risk assessment scores are not meant to be used in passing sentences, they are only meant to help in deciding which defendants need probation or treatment programs. But judges have ended up using risk assessment scores in determining sentences, often handing out harsher sentences for someone with a higher risk score. What are the odds of someone wrongly getting a longer sentence because of their COMPAS score? Quite high, seeing the low accuracy of the algorithm. Then why is the use of such algorithms being encouraged?

According to Cathy O’Neil, data scientist and author of Weapons of Math Destruction, this is because people tend to trust and fear mathematics at the same time, which takes away their ability to question the results of algorithms. “People should feel more entitled to push back and ask for evidence, but they seem to fold a little too quickly when they’re told that it’s complicated,” she says.

O’Neil also explains why algorithms have the same biases that humans do in her book, “Our own values and desires influence our choices, from the data we choose to collect to the questions we ask. Models are opinions embedded in mathematics.

Get the latest reports & analysis with people's perspective on Protests, movements & deep analytical videos, discussions of the current affairs in your Telegram app. Subscribe to NewsClick's Telegram channel & get Real-Time updates on stories, as they get published on our website.

Subscribe Newsclick On Telegram

Latest