The Avon and Somerset police and the Bristol City Council have stopped using at least two AI models that assessed the risk of crimes against children. The decision was made due to low accuracy, and verifying the systems proved nearly impossible—independent auditors could not find the source code or a list of variables, according to WIRED.

WIRED, along with the civil rights group Liberty Investigates, local media Bristol Cable, and the nonprofit Lighthouse Reports, analyzed hundreds of pages of documents obtained through freedom of information requests. This report comes amid the launch of PoliceAI—a national center for testing and scaling AI tools in police forces across England and Wales.

How Bristol Collected Data

The focus is on the Think Family Database—a database established by the Bristol City Council in 2016 to assist families and children who may need support. According to WIRED, it could include records on nearly 500,000 city residents, although the project page currently states it covers about 55,000 families. This discrepancy may be due to the difference between individual records and family profiles.

The Think Family Database combined police and social data: housing status, mental health information, teenage pregnancies, participation in parenting courses, school absences, and eligibility for free school meals. The publication states that this information was collected without the direct consent of residents, using legal grounds for information sharing among government entities.

One police data specialist described the approach as mixing different data sets.

“I put all of this into a big bucket,” he said.

Using the Think Family Database, law enforcement and authorities built machine learning models that assigned risk scores to adults and children. Journalists learned of at least 23 models from the Avon and Somerset police—ranging from predicting burglaries and court absences to assessing the risk of missing persons and the likelihood of becoming a victim of domestic violence.

Simultaneously, the Offender Management App operated, targeting around 300,000 individuals in the region. One senior officer referred to it as the basis for a “leaderboard” of the most dangerous offenders.

Reasons for Discontinuation

One of the early models assessed the risk of crimes against children. According to WIRED, it included data from the police, city council, and other government bodies, as well as anonymized information from the charity Barnardo's on 1,000 children who had already been victims of such crimes.

The scoring was also influenced by:

  • the child's status as needing assistance;
  • frequent school absences;
  • mental health issues.

Another model considered housing support, rental arrears, and free school meals.

In 2016, the police ethics committee warned that the selected data and variables could lead to algorithmic bias. They recommended using the system cautiously and explaining to the public why and how such analytics were applied.

Later, the British nonprofit consulting organization Social Finance evaluated the project. In their review, they identified risk scoring as the weakest element, with low accuracy undermining the practical value of the models. By the time of the review, two models assessing the risk of crimes against children had already been discontinued, according to WIRED.

Social Finance linked the decline in model quality to changes in the data set. The police attempted to scale the approach across Avon and Somerset, which encompasses five local councils, but failed to reach agreements on data sharing with all local authorities. As a result, the models retained primarily police data without the previous social indicators.

According to journalists, Bristol city service employees complained that vulnerable children were not included in the results. One worker noted that minors who had recently become victims of crimes could receive lower scores than individuals involved in burglary cases. Other staff members expressed their reluctance to rely on the assessments due to the opaque methodology.

Social Finance also could not fully verify the models: they were unable to locate the source code and list of variables. According to WIRED, neither the police nor the Bristol City Council retained documents regarding the decision to discontinue the two child crime risk assessment models by June 2023.

Audit Findings

Separately, WIRED obtained over 36,000 performance assessments from the police for 13 models used or tested between 2017 and 2024. The data was forwarded to the auditing firm Eticas, which concluded that most models exhibited low accuracy in positive identifications. This means a significant portion of individuals were incorrectly flagged as risks by the system.

According to this data, the model for identifying potential burglars showed a positive identification accuracy of less than 10% over three years: fewer than one in ten individuals flagged by the system actually committed such a crime. Auditors also noted that such performance metrics are atypical for well-managed models in operational use.

The police told WIRED that some models, including the burglary tool, were never implemented. They explained the presence of long-term performance assessments as a result of an automatic check of a static file that was not deleted after the decision not to implement it.

Separately, the police website states that some of the department's tools use AI, and the results of algorithms are applied only as advisory signals for officers. The police emphasized that the models do not make decisions automatically.

The Bristol City Council stated that it currently only uses the NEET risk model—an assessment of the likelihood that a child will not engage in education, employment, or training after leaving school. According to officials, this tool does not replace professional judgment.

PoliceAI

This story unfolds against the backdrop of expanding AI applications in the UK’s law enforcement through PoliceAI. On June 10, the UK Home Office launched a center for testing and scaling AI tools across 43 police forces in England and Wales. The project has a budget of £75 million over three years.

In its first year, PoliceAI will focus on tools for analyzing, solving, and summarizing digital evidence. Trials are set to take place in ten forces in 2026-2027, followed by scaling to all police forces.

PoliceAI operates under the College of Policing—a professional body responsible for standards and training for police officers in England and Wales. It is led by former Chief Constable of Avon and Somerset Police, Andy Marsh.

WIRED noted this connection: in the region where controversial AI analytics were developed, the head of the structure now involved in scaling AI for police previously worked. Against this backdrop, the Bristol case illustrates that the risks associated with such models are not only linked to algorithm accuracy but also to data quality, documentation retention, and the possibility of independent verification.

Previously, the publication reported on Maryland resident Alonzo Sawyer, who spent nine days in jail due to a false match in a facial recognition system.

Earlier, London Mayor Sadiq Khan blocked a nearly £50 million contract between the British division of Palantir and the Metropolitan Police. The deal aimed to implement an AI-based analytical system called Unified Operational Analytics to expedite criminal investigations. However, the mayor vetoed the agreement, citing “serious breaches” of procurement procedures.

It is worth noting that in 2023, it was revealed that the FBI, in collaboration with the Pentagon, had secretly developed and tested a facial recognition system for years.