Researchers warn predictive policing could reinforce discrimination

Researchers have warned that predictive policing could reinforce discrimination if systems treat records of where officers have looked as a reliable map of where crime occurs. The warning appears in a BBC adaptation of an award-winning essay, with new comments from researchers. Its principal policing studies were published in 2017 and 2019; they identify risks, rather than measuring a rise in discrimination caused by a current police AI system.

The concern is a feedback loop. If a forecast sends officers to an area already heavily patrolled, they may record more incidents there partly because they are present to see them. Those records can then influence the next forecast. Crime reported by the public and crime detected through patrols need not reflect the same pattern.

In a 2017 case study of the Los Angeles Police Department, sociologist Sarah Brayne documented a person-based scoring system in which police contact raised someone’s risk score, while a higher score could draw further police attention. She also described how selective recording could affect forecasts of crime locations. Her research did not measure whether those location forecasts increased discrimination; she called the effect of data-driven surveillance on inequality an open empirical question.

A separate 2019 study by Rashida Richardson, Jason Schultz and Kate Crawford examined 13 US jurisdictions selected because predictive policing overlapped with documented investigations or oversight of unlawful or biased police practices. In nine, records produced during the periods in question were available to train or inform predictive systems; the researchers could not reach a conclusion for the other four. Nine out of 13 is therefore not a rate for police forces generally, nor proof that every system used those records. The authors found stronger evidence of their use in Chicago.

The BBC article also draws an analogy with a 2019 healthcare study. There, a system predicting future care costs gave Black patients the same risk scores as White patients who were, on average, less ill. The researchers calculated that, under a simulated selection without that health gap, the Black share of patients automatically identified for extra support would rise from 17.7% to 46.5%. That was not an observed change in care, and the study was not about policing. The original essay misdescribed the figures as percentages of Black and White patients receiving help; the BBC adaptation describes them as shares under different selection scenarios.

Carolyn McKay, a University of Sydney researcher quoted by the BBC, called for scrutiny of the data and human oversight of policing decisions. Dimitrios Kollias of Queen Mary University of London agreed that historical bias is a serious concern but said it is not inevitable: systems can be tested for unfair outcomes and their data and objectives improved. Whether such safeguards work in a particular police deployment remains a question for evaluation, not a conclusion established by the older studies.