This page highlights several examples or case studies that we have considered in designing the workshop. Each example is chosen to highlight some aspects of the problem we consider important.
This page highlights several examples or case studies that we have considered in designing the workshop. Each example is chosen to highlight some aspects of the problem we consider important.
One example of a case where machine learning decision making systems have come under scrutiny is predicting criminal recidivism, that is whether someone accused of a crime is likely to commit another. These prediction systems are used in many states to decide whether to allow defendents out on bail or even in sentencing decisions.
Pro Publica extensively reported on this issue, and has found that even defendents often disagree with the assessments (even when it benefits them):
Sometimes, the scores make little sense even to defendants.
James Rivelli, a 54-year old Hollywood, Florida, man, was arrested two years ago for shoplifting seven boxes of Crest Whitestrips from a CVS drugstore. Despite a criminal record that included aggravated assault, multiple thefts and felony drug trafficking, the Northpointe algorithm classified him as being at a low risk of reoffending.
“I am surprised it is so low,” Rivelli said when told by a reporter he had been rated a 3 out of a possible 10.
In addition to challenges around their accuracy and the limited context that the prediction scores incorporate, one of the major problems with the machine learning predictions is the fact that defendents are not allowed to contest the prediction that is shared with judges:
Defendants rarely have an opportunity to challenge their assessments. The results are usually shared with the defendant’s attorney, but the calculations that transformed the underlying data into a score are rarely revealed.
The companies that manufacture the recidivism prediction systems claim that the calculations are proprietary information.
This case serves as an example that demonstrates the complexities of these decision making systems: the need to explain decisions (whether to the defendents themselves or experts such as judges and lawyers), the need to protect intellectual property and prevent gaming the system, and most fundamentally the need to maintain human rights.
One example of a case where machine learning decision making systems have come under scrutiny is predicting criminal recidivism, that is whether someone accused of a crime is likely to commit another. These prediction systems are used in many states to decide whether to allow defendents out on bail or even in sentencing decisions.
One example of a case where machine learning decision making systems have come under scrutiny is predicting criminal recidivism, that is whether someone accused of a crime is likely to commit another. These prediction systems are used in many states to decide whether to allow defendents out on bail or even in sentencing decisions.