AI Safety & Risk: What You Need to Know Right Now
By Saghir Hussain ·
The government's AI Safety Institute has a £100 million budget and a prime office in London, yet it still cannot tell you whether the chatbot your teenager uses at 2am is harvesting their mental health data for training runs.
The regulator that isn't one
Let's be clear: the AI Safety Institute (AISI) is not a regulator. It has no statutory powers, no enforcement teeth, and no mandate to investigate private companies. It is a research body funded by the Department for Science, Innovation and Technology (DSIT) — essentially a taxpayer-funded advisory shop for frontier labs that already employ their own safety teams.
When the Online Safety Act finally received Royal Assent in October 2023, Ofcom got powers over user-generated content. AI-generated content? Still largely uncharted. The Act mentions "generative AI" only in passing, leaving Ofcom to issue guidance later. Meanwhile, deepfake porn targeting schoolgirls in Birmingham and Manchester circulated on Telegram and X for months before platform action. The police lack specific offences; the Crown Prosecution Service relies on a patchwork of harassment and copyright law.
Prime Minister Rishi Sunak's Bletchley Park summit last November produced a "landmark" declaration signed by 28 countries and the EU. Voluntary commitments. No penalties. No transparency registers. The frontier labs — OpenAI, Anthropic, Google DeepMind, Meta — agreed to let AISI test models before deployment. But testing is voluntary, results are confidential, and "deployment" definitions are slippery. GPT-4o launched in May with voice capabilities that AISI had not fully evaluated. The institute learned about it the same way we did: from the livestream.
Where the risk actually lives
The existential risk debate — rogue AGI, paperclip maximisers — dominates Westminster briefings and BBC Newsnight segments. It suits the labs: it frames safety as a future engineering problem, not a present commercial one.
The real risks are boring and profitable. Algorithmic management systems setting delivery routes for Deliveroo riders in Leeds, penalising them for toilet breaks. Automated benefit fraud flags at the DWP that disproportionately suspend payments to claimants in Tower Hamlets and Glasgow, with no explainable logic. Facial recognition trials by the Met and South Wales Police that misidentify Black faces at higher rates — documented by Big Brother Watch, ignored by the Home Office.
Small businesses bear the compliance cost. A solicitors' firm in Bristol using an AI drafting tool faces professional indemnity spikes if the tool hallucinates a precedent. A GP practice in Cornwall trialling an AI triage bot has no clear liability framework when it misses sepsis. The Post Office Horizon scandal should have taught us what happens when unaccountable software meets vulnerable people. We are repeating the architecture.
The accountability vacuum
DSIT's current approach: a "pro-innovation" framework relying on existing regulators (ICO, CMA, Ofcom, FCA, EHRC) to police AI in their sectors. But the ICO has 600 staff for the entire UK data economy. The CMA's new Digital Markets Unit still awaits statutory powers. The FCA is stretched regulating crypto and greenwashing.
Labour's manifesto promises a "binding regulation" for frontier models and a "Regulatory Innovation Office." Good. But Peter Kyle, the Shadow Secretary of State, has not defined "frontier," set compute thresholds, or explained how a UK-only regime affects open-source models hosted on Hugging Face servers in France.
We need a named minister with a statutory duty to publish an annual AI harm register: incidents, sectors affected, regulatory actions taken, gaps identified. We need mandatory pre-deployment notification for models above a defined compute threshold — not voluntary red-teaming. We need liability clarity: if your AI agent signs a contract, libels a competitor, or denies a loan, who pays?
The technology is not the problem. The accountability vacuum is. And it is a political choice.
When the next Horizon-scale scandal breaks — and it will — will Parliament ask why it ignored the warnings? Or will it feign surprise again?