Ministers may unlock economic growth and enhance public services by integrating data from different government departments, according to Sir Robert Chote, the chair of the UK Statistics Authority.

Chote has raised concerns about the prevalent “siloed” nature of government data, where departments and agencies keep their information isolated, potentially stifling innovation and masking underlying issues.

Chote pointed out that these silos often exist because government bodies fear that exposing their data might reveal flaws or lead to inconvenient conclusions.

“During the pandemic, the sense of national emergency provided an urgent impetus for sharing data. This enabled valuable insights, such as identifying where to target communication campaigns to increase vaccine uptake.”
Sir Robert Chote

However, Chote noted that since then, the momentum for data-sharing has waned, leaving many opportunities unrealized.

He highlighted the “unrealized potential” in linking and sharing data, which could be instrumental in generating new ideas and improving government policies.

An example of this potential was seen in the Better Outcomes through Linked Data program, which revealed that offenders with substance abuse issues were ten times more likely to die by suicide than the general population in England and Wales.

By linking data from the Ministry of Justice, the Department of Health and Social Care, and the Ministry of Housing, Communities, and Local Government, the program suggested that accelerating access to treatment could reduce these tragic outcomes.

Despite such successes, Chote emphasized that these instances of effective data integration remain rare.

He stressed that departments often prioritize protecting privacy and guarding against reputational damage over the potential benefits of data sharing. There is a hesitation to expose weaknesses in their data or risk narratives that might challenge the status quo.

Budget Cuts, Low Survey Rates Strain Data Collection

The reluctance to share data is compounded by the growing difficulty in collecting reliable information. Chote warned that budget cuts and declining public participation in surveys are making it harder for official bodies like the Office for National Statistics (ONS) to gather accurate data.

“It is becoming increasingly difficult and expensive to maintain the quality of the household surveys that we rely on to understand important parts of the economy and society because of falling response rates and higher collection costs, a trend seen by other statistics agencies across the world.”
Sir Robert Chote

This challenge was illustrated last year when the ONS halted its monthly publication of jobs data from the labor force survey due to concerns about the reliability of the figures, as response rates plummeted.

The labor force survey is crucial for estimating unemployment and job data, which in turn influences the Bank of England’s decisions on interest rates.

In response to these challenges, the ONS has begun sourcing data from private companies to supplement its official statistics.

For instance, it now uses anonymized data from supermarket scanners to monitor inflation and taps into data from Auto Trader and the Rail Delivery Group to track car prices and train fares, respectively.

Anonymized Visa data also helps the ONS monitor credit and debit card spending to inform its travel and tourism research, while live PAYE data from HM Revenue and Customs provides a more accurate picture of employment levels.

Yet, as Chote noted, these examples remain exceptions rather than the rule. To truly harness the power of data, Chote suggests that the government must overcome its hesitancy and embrace a more integrated approach to data sharing.

This, he believes, could lead to more informed decision-making, better public services, and a stronger economy.