Investigation launched into algorithmic bias

Written by Peter Walker

The potential for bias in the use of algorithms in crime and justice, financial services, recruitment and local government is set to be investigated by the Cabinet Office’s Centre for Data Ethics and Innovation (CDEI).

The government body stated that algorithms have huge potential for preventing crime, protecting the public and improving the way services are delivered, but decisions made in these areas are likely to have a significant impact on people’s lives, so public trust is essential.

Professionals in these fields are increasingly using algorithms built from data to help them make decisions, the CDEI explained. But there is a risk that any human bias in that data will be reflected in recommendations made by the algorithm, so the investigation will aim to ensure those using such technology can understand the potential for bias and have measures in place to address.

In crime and justice, algorithms could be used to assess the likelihood of re-offending and inform decisions about policing, probation and parole. For example, some police forces have already started to use algorithms to feed into their decision-making – such as the Harm Assessment Risk Tool in Durham to assist officers in deciding whether an individual is eligible for deferred prosecution based on the future risk of offending.

The establishment of the CDEI supports the government’s wider Industrial Strategy, to make sure data-driven technologies and artificial intelligence are used for the benefit of society. It will partner with the Race Disparity Unit to explore the potential for bias based on ethnicity in decisions made in the crime and justice system.

Digital secretary Jeremy Wright said: “I look forward to seeing the Centre’s recommendations to government on any action we need to take to help make sure we maximise the benefits of these powerful technologies for society.”

Roger Taylor, chair of the CDEI, added: “We want to work with organisations so they can maximise the benefits of data driven technology and use it to ensure the decisions they make are fair – as a first step we will be exploring the potential for bias in key sectors where the decisions made by algorithms can have a big impact on people’s lives.”

The CDEI will also explore the opportunities for data-driven technology to address the potential for bias in existing systems and to support fairer decision-making. This may include increasing opportunities for those in the job or credit markets in existing recruitment and financial services systems.

In recruitment, computer algorithms can be used to screen CVs and shortlist candidates. This could help potentially limit the impact of unconscious bias, where people discriminate against candidates because of their background, while there have also been reports of such technology inadvertently exacerbating gender bias.

And in financial services, data analysis has long been used to inform decisions about whether people can be granted loans. But the rise of data and AI machine-learning presents increased issues about the transparency and fairness of such decisions.

The CDEI’s will also investigate how data is used to shape online experiences through personalisation and micro-targeting – for example where you search for a product and then adverts for similar products appear later in your browser. It is launching a series of nationwide workshops to investigate public views on the acceptability of micro-targeting.

Both policy reviews will publish interim reports in the summer with final reports out early next year.