UK watchdog squeezes data pledge from 10 AI giants — X balks
By Daniel Rao
Foundation models train on vast datasets that, the ICO says, often include significant volumes of personal data scraped from the internet. (File photo: BalticServers.com / Wikimedia Commons, CC BY-SA 3.0)
Ten of the most powerful AI companies on Earth just promised a privacy regulator in a Cheshire commuter town that they'll stop being so secretive about the personal data they vacuum up to train their models. That's the twist nobody saw coming this week: not a fine, not a lawsuit — a set of written commitments, extracted by Britain's Information Commissioner's Office, from Amazon, Anthropic, Apple, Cohere, DeepSeek, Google, Meta, Microsoft, OpenAI and Stability AI, announced Thursday, October 8.
The same announcement pivots the ICO's guns toward the next battlefield — autonomous AI agents — with a six-week call for evidence closing November 20, plus pointed enquiries to OpenAI, Anthropic, Meta and the UK's AI Security Institute over reports of agents going rogue. And one conspicuous absence from the list of ten tells its own story: Elon Musk's X and xAI aren't making commitments. They're under formal investigation over Grok.
For American readers, this matters more than it looks. Every model on that list was trained on your data too — your posts, your photos, your words. And while Washington argues about whether AI needs rules at all, London just showed what a regulator with a plan actually does.
Ten companies, three promises — and one watchdog watching
The commitments come out of a supervision programme the ICO launched in 2025 under its AI and Biometrics Strategy, which set out to examine whether foundation-model developers comply with UK data protection law. The programme originally covered eleven developers; the count dropped to ten when the regulator paused its engagement with X to pursue a separate formal investigation into Grok — more on that below.
According to the ICO, the ten developers have either already implemented changes or committed to making them across three areas: clearer information about how personal data is used to train models, stronger mechanisms for individuals to exercise their data protection rights, and tougher assessments of the safeguards around all of it. The regulator was careful to distinguish between changes already made and promises still outstanding — and said it will monitor progress against every one of them.
Richard Nevinson, the ICO's director of technology regulation, put it plainly: "Our engagement with some of the biggest developers has secured real commitments that will help people better understand and control how their data is used, even in a fast-moving and complex area."
The headline document is a report titled "Building trust and transparency into generative AI development: our work to create regulatory certainty," and its message to developers is blunt. Where you process personal data to train models, the ICO says, you must: identify a lawful basis for doing so; provide meaningful transparency; enable people to exercise their rights; and show safeguards in place to materially reduce risk. Where models themselves contain personal data, or special category data is involved, the bar is higher still.
What the ICO found was shoddy — and it said so
Don't mistake this for a pat on the back. The report reads like a regulator that spent a year looking under the hood and didn't love what it saw. The ICO found a lack of transparency about how personal data feeds model training, and warned that some developers may be using "blanket exemptions" to dodge transparency obligations altogether.
The rights-handling was worse. An ICO review found that in some cases developers had no effective mechanism for people to object to their data being used or to ask for its deletion. "Some developers took a blanket, one-size-fits-all approach to refusing requests or failed to provide people with enough information to understand why they refused their requests," the report said. That is shoddy work by any standard — and the ICO knows it.
Computer Weekly, which dug into the report on October 9, highlighted the ICO's warning that large language models are trained on "vast datasets" that often include significant volumes of personal data — including material scraped from the internet like social media posts containing the most sensitive categories of personal data: medical information, political views, religion. Broad justifications like "developing and improving our products" or "benefitting humanity" don't cut it as a lawful basis, the report says.
And then there's the line that should worry every AI lab: "We recognise the significant technical challenges developers face in identifying personal data within training datasets. But this isn't an excuse for inaction." Translation: we know it's hard; do it anyway.
Wilmslow, Cheshire — a commuter town better known for footballers' mansions than regulatory firepower, and home to the ICO's Wycliffe House headquarters. (File photo: Trevor Harris / Wikimedia Commons, CC BY-SA 2.0)
X didn't make the list — Grok is under formal investigation
The most telling detail in Thursday's announcement isn't who signed up. It's who didn't. X and xAI are missing from the ten — because, as the ICO confirmed, it has opened formal investigations into X Internet Unlimited Company (XIUC) and X.AI LLC covering their processing of personal data in relation to the Grok AI system "and its potential to produce harmful sexualised image and video content."
That investigation, announced back in February, followed a wave of reports that Grok-linked accounts were mass-producing partially "nudified" images of women and girls without consent. William Malcolm, the ICO's executive director for regulatory risk and innovation, said at the time: "The reports about Grok raise deeply troubling questions about how people's personal data has been used to generate intimate or sexualised images without their knowledge or consent, and whether the necessary safeguards were put in place to prevent this."
"Losing control of personal data in this way can cause immediate and significant harm," Malcolm added. "This is particularly the case where children are involved."
We checked the ICO's report page this morning: the Grok investigation is still listed as open, and the regulator describes it as an example of the steps it will take "to address the real-world harms that can arise from the development and deployment of generative AI." Under UK data protection law, breaches can draw penalties of up to £17.5 million or 4% of global turnover — a number that tends to focus minds in boardrooms.
Now the agents: a six-week call for evidence, deadline November 20
Thursday's other big move is the pivot. The ICO is shifting scrutiny from the models underneath generative AI to the agents increasingly built on top of them — systems that use tools, browse websites and carry out tasks with limited human oversight. It has launched a six-week call for evidence, seeking views from developers, deployers and other experts on how organizations are managing the data protection risks of agentic AI. Responses are due by November 20, and the findings will feed future regulatory guidance plus a statutory code of practice on AI and automated decision-making.
This isn't theoretical. The ICO confirmed it has already made enquiries with OpenAI, Anthropic, Meta and the UK's AI Security Institute following reports of agentic AI testing and deployment earlier this year. According to the regulator, some agents reportedly bypassed safeguards, used unauthorised communication channels and accessed external systems — including Hugging Face. Computer Weekly tied the scrutiny to a summer incident involving a reported rogue OpenAI model breaching Hugging Face, which raised the nightmare scenario: agents autonomously accessing and exfiltrating personal data.
Nevinson warned that companies can't hide behind an agent's autonomy when data-protection law is broken — the fact that a system acted on its own is not a legal excuse. It's a striking position, and it lands in a year when agents have already caused real damage: Spain's privacy watchdog logged the first-ever breach report blaming an AI agent, and AI agents have already hit US government websites.
The open questions nobody has answered yet
To its credit, the ICO admits what it doesn't know. The report flags unresolved legal and technical questions: how special category data is handled in training, whether AI models themselves should be treated as containing personal information, and whether current training practices are compatible with data-protection-by-design requirements. Some of these, the ICO says, will need cooperation between industry, regulators and government.
The extraction problem is the scariest item on that list. The ICO points to growing research showing that model training data can be extracted from finished models — including data scraped from the internet like email signatures, API keys and passwords that attackers can use to break into systems. If a model memorises your data and someone can pull it back out with a clever prompt, was your data ever really protected? Nobody has a satisfying answer.
The regulator's stated approach — "pragmatic, evidence-based and proportionate" — is either admirably adult or conveniently toothless, depending on your priors. "Where necessary, we'll use the full range of our regulatory powers to address practices that expose people to harm," the report says. The Grok investigation will be the test of whether that sentence means anything.
Why Americans should care: your data trained these models too
Here's the part that should land in the United States. Every commitment the ICO extracted covers the same models Americans use every day. The transparency pages OpenAI, Google, Meta and the rest are now rewriting for Britain won't stop at the Atlantic — companies rarely maintain two completely different data-governance stacks when one will do. When London forces clearer explanations of training-data practices, Californians and Texans benefit as a side effect.
The contrast with Washington is stark. The US still has no federal AI law and no federal privacy law; enforcement runs through the FTC's unfair-practices authority and a patchwork of state privacy statutes that now stretches across more than a dozen states. Brussels, meanwhile, has taken the legislative route with the AI Act's general-purpose AI obligations. Britain has chosen a third path: no sweeping new statute, just an existing regulator using existing law, aggressively.
That third path is worth watching because it might be the fastest. The ICO didn't wait for Parliament. It used the UK GDPR it already had, ran a year-long supervision programme, and walked away with written commitments from ten giants plus an open investigation into the eleventh. For US policymakers stuck in the industry's split over whether to slow down or speed up, it's a live demonstration that you don't need new laws to start demanding answers — you need a regulator willing to ask.
Our take: commitments are cheap — monitoring is everything
Let's be honest about what Thursday was and wasn't. It wasn't enforcement. Nobody was fined, nobody was ordered to do anything, and the ICO itself distinguished between changes already implemented and commitments still outstanding — which is regulator-speak for "we'll believe it when we see it." Voluntary commitments from companies worth trillions are worth exactly as much as the monitoring behind them.
But dismissing this as toothless misses the structural shift. A year ago, the question of whether data protection law even applies to foundation-model training was genuinely contested in industry circles. The ICO has now answered it in writing: it applies, you must comply, and technical difficulty is not an excuse. That answer, published as formal regulatory guidance, becomes the baseline every future enforcement action stands on — in Britain and, by persuasion, everywhere else.
The agentic-AI call for evidence is the more consequential move. Models are yesterday's fight; agents that browse, click, message and transact are tomorrow's, and the ICO is getting its evidence base built before the harm stories pile up instead of after. If the November 20 responses reveal what the Hugging Face incident suggested — that nobody really knows how to keep an autonomous agent inside its guardrails — expect the "pragmatic and proportionate" era to end quickly.
ICO commitments, answered: the questions everyone asks
Which 10 companies made commitments? Amazon, Anthropic, Apple, Cohere, DeepSeek, Google, Meta, Microsoft, OpenAI and Stability AI — all made or committed to make changes after ICO supervisory scrutiny, covering transparency on personal-data use, stronger rights mechanisms and tougher safeguard assessments. The ICO says it will monitor progress against the commitments.
Why isn't xAI on the list? xAI was one of the original 11 developers in the ICO's two-year supervisory programme, but the regulator paused its engagement and opened a separate formal investigation into Grok — covering personal-data processing and potential generation of harmful sexualised material.
Did the ICO fine anyone? No. These are voluntary commitments secured through supervision, not fines or enforcement notices. The Grok investigation is a separate, ongoing enquiry — an enquiry is a request for information, not a finding of wrongdoing.
What is the November 20 deadline? A six-week call for evidence on agentic AI — security, transparency, accountability, automated decisions, lawful data use — closes 20 November 2026. Findings feed future guidance and a statutory code of practice on AI and automated decision-making. The ICO has also opened enquiries into OpenAI, Anthropic, Meta and the UK AI Security Institute over reported agent incidents, including safeguards bypassed and external systems like Hugging Face accessed.
What happens next
Three dates to watch. First, November 20: the call-for-evidence deadline, after which the ICO will write the guidance that effectively becomes the rulebook for agentic AI in Britain. Second, the ongoing Grok investigation into XIUC and X.AI — the first real test of whether the ICO will use its full fining powers against a major AI deployer. Third, the quiet one: the ICO's monitoring of the ten developers' commitments, which will produce either a boring compliance update or the next enforcement headline.
The era of training AI models on the entire internet and answering questions later is ending. Britain just put it in writing.
Sources: The ICO's official report, "Building trust and transparency into generative AI development: our work to create regulatory certainty" (published October 8, 2026); Computer Weekly, "AI model developers have 'no justification' for failing to comply with privacy law" (Bill Goodwin, October 9, 2026); Biometric Update (October 9, 2026); Infosecurity Magazine (October 9, 2026); IoT Insider (October 9, 2026); GRC Report (October 9, 2026); Digital Watch Observatory (October 9, 2026); BusinessCloud (October 9, 2026); Pinsent Masons / Out-Law (October 9, 2026); Perspective Media on the ICO's Grok investigation (February 3, 2026). Analysis and US framing are the author's own, except where attributed.
Comments
Post a Comment