AI Safety & Risk: What You Need to Know Right Now
By Saghir Hussain ·
The government's AI Safety Institute has spent £100 million of your money and still can't tell you if the chatbot your kid uses for homework is harvesting their data or the algorithm denying your Universal Credit is biased. That is the scandal hiding in plain sight.
The Institute That Marks Its Own Homework
Launched with fanfare at Bletchley Park, the AISI was supposed to be our independent watchdog. Instead, it operates inside the Department for Science, Innovation and Technology — literally marking the homework of the same ministers who approve the contracts. Its "evaluations" are voluntary. Frontier labs like OpenAI, Anthropic, and Google DeepMind decide whether to participate, what models to submit, and which results get published. When Anthropic's Claude 3 Opus was tested pre-release, the findings were shared privately. The public got a press release.
This isn't oversight. It's a PR operation funded by the taxpayer.
Meanwhile, real harm is accumulating. In Tower Hamlets, a benefits automation system wrongly flagged over 1,000 claimants for fraud between 2021 and 2023 — disproportionately targeting Bangladeshi and Somali households. The DWP knew the algorithm had "disparate impact" but kept it running. No minister resigned. No procurement officer was sacked. The system was only paused after a Freedom of Information request by the Public Law Project exposed the scale.
In Glasgow, NHS boards piloted an AI triage tool that routinely deprioritised elderly patients with multiple comorbidities — because the training data treated "complex needs" as a proxy for "poor outcomes." Clinicians raised flags internally for months. The pilot expanded anyway.
Small Businesses Carry the Risk, Big Tech Writes the Rules
While ministers pose for photos at AI summits, a graphic designer in Birmingham gets sued because Midjourney regurgitated a copyrighted illustration in a client deliverable. A solicitor in Manchester faces disciplinary action after a junior associate cited hallucinated case law from ChatGPT. The liability falls on the user every time. The model providers? They have indemnity clauses, lobbyists, and direct lines to Number 10.
The UK's "pro-innovation" framework means exactly what it says: innovation first, accountability maybe never. The EU's AI Act bans social scoring and mandates transparency for high-risk systems. Here, we get a voluntary code of practice drafted by the very companies it's meant to regulate. The Data Protection and Digital Information Bill — currently limping through the Lords — weakens automated decision-making safeguards just as public sector deployment accelerates.
Who Actually Pays?
You do. Every time a council outsources child risk scoring to a US vendor that refuses to disclose its training data. Every time a recruitment platform filters out your CV because you went to a comprehensive in Middlesbrough rather than a private school in Surrey. Every time your energy bill spikes because National Grid's "smart balancing" algorithm prioritises industrial contracts over domestic supply during peak demand.
The AISI's latest annual report celebrates "building evaluation capacity." It does not mention a single enforcement action, a single model withdrawn, a single company fined. Because it has no powers to do any of those things.
Next month, the government publishes its AI Opportunities Action Plan. Watch for the language: "growth," "leadership," "unlocking potential." Ask who unlocks what for whom. Then ask your MP why the only safety body with teeth is the one protecting Big Tech's margins.
When does the taxpayer get a watchdog that bites?