The right to be forgotten needs rewriting.

Article 17 of the GDPR gives you a right to erasure, and the wording is worth reading rather than paraphrasing:

"The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay and the controller shall have the obligation to erase personal data without undue delay…"

One thing to flag before going any further. That right assumes the data subject gave consent in the first place, which is something the frontier labs largely skipped. So there is already a problem underneath the problem I want to write about. For the rest of this piece I am going to assume a world where models are trained only on data people actually agreed to hand over, because otherwise the erasure question never gets a fair hearing.

The drafters had a clear picture in mind: your data sits somewhere, a controller can find it, and on request they remove it and can show you it is gone. That picture holds for nearly all software ever written, and the right has worked reasonably well as a result.

A trained language model breaks all three at once. Your data does not sit anywhere in particular, no controller can locate it, and nobody including the controller can demonstrate afterwards that it is gone. (Section 04 comes back to what the statute actually says, once the technical picture is on the table. I am deliberately not arguing law before the machine.)

What the industry ships in response is a bundle of four things described as deletion, only one of which touches the model at all. Everything downstream of this page follows from that gap: the law promises an outcome the technology cannot produce, so the outcome gets redefined at the point of delivery, without anyone announcing it.

The usual framing is "database versus AI", and I think that framing is the problem. Once "training" is finished a model no longer needs the dataset at all, because everything it learned now lives in the parameters. Deleting your row from the original corpus changes nothing, in the same way that taking the sugar back out of your coffee is not a thing you can do. You can still taste it. You just cannot retrieve it.

But a model is not the only place your data ends up. Between a row in a table and a frontier model there are at least four intermediate homes, and each one has its own deletion cost, its own verifiability, and its own body of law already attached.

Reading the ladder in order makes a point that gets lost in the impossibility discussion. Five of the six rungs are fine, with deletion that is cheap and provable. The problem sits entirely on the sixth, and putting personal data there was a decision somebody made rather than a fact about machine learning.

One objection has to be dealt with before going further. If your data is diluted across billions of numbers, is any of you still in there in a sense that matters? Sometimes not. But models do "memorise", and researchers have pulled verbatim training text out of production systems: names, phone numbers, addresses, long passages of copyrighted books. The strongest predictor of whether something can be pulled back out is simply how often it appeared.

Which makes the coffee analogy worse than it first looks. It is not only that you cannot get the sugar out. Somebody else may be able to, if they ask the right way.

This is why the EDPB declined to treat models as automatically anonymous in December 2024. Their bar: a model is anonymous only if the chance of extracting someone's personal data, directly or through careful prompting, is negligible for every person in the training set, and most large models cannot clear that bar.

So we arrive at the strange world of "unlearning". I am not an ML researcher, and there are far more methods here than I have read, but the shape of the field is clear enough. It has a decade of published work behind it and some of it ships: labs apply these methods to open-weight releases to strip hazardous capabilities before publishing, because you cannot put a filter in front of a model people download. None of that is in dispute, and my argument is not that unlearning does nothing.

The dispute is about what these methods actually remove. Positions on the map below are roughly cost against strength of guarantee. Two methods reach the top of the guarantee axis, and both sit at the far right of the cost axis, so neither is ever run for one person. (That gap is the whole story, and it is the reason the cheapest option is also the most popular one.)

So what do they remove? Mostly, they suppress the behaviour while leaving the knowledge in place. The fact stays encoded and the model learns not to say it out loud, which passes the check a compliance process would run and fails almost everything else. What convinced me was not the theory but how easily the suppression comes undone.

People collapse all of this into the word "impossible," which is doing too much work. Three separate claims hide inside it, and only the last is immovable.

It is easy to attack a version of the statute that does not exist, so let me start with what it actually says. Article 17 is technology-neutral, attaching to "personal data" and "processing" rather than to files or rows. And 17(2) already anticipates that data spreads: a controller who has published data must take reasonable steps, accounting for available technology and cost, to tell other controllers to erase links and copies. Whoever drafted that was thinking about replication, which is more foresight than the "written for databases" line allows them.

Is erasure even unconditional? It is not. Article 17(3) carves out freedom of expression, compliance with a legal obligation, public interest and public health, archiving and research, and the defence of legal claims. So anyone arguing that the law demands the impossible (me, a few paragraphs ago) should concede first that it already bends in five named situations.

So the difficulty is not that Article 17 was written for databases. It is that the remedy assumes a controller can locate what it is asked to remove, and beneath that sits a question still unanswered: is a trained model a record of the people in its training data, or something closer to a speaker that has read about them? The distinction decides everything, because records get access, rectification and erasure, while speakers get regulated on what they say and do, and no legal system has ever granted a right to compel a speaker to forget.

I went looking for a settled answer to that and did not find one. Two European authorities read the same instrument in opposite directions. Hamburg's DPA argued in July 2024 that LLMs do not store personal data at all, on the basis that weights encode numerical correlations between tokens rather than information identifying a person; on that reading Article 17 cannot point at the model, only at inputs and outputs. The EDPB's Opinion 28/2024 went the other way. It is also hedged into near-uselessness, with one analysis counting "case by case" sixteen times and "may" or "might" a hundred and sixty-one times, which more or less guarantees divergence between national regulators.

Two more regulators have since answered differently again. The UK's ICO holds that erasing one person's data will usually not require touching the model, but that where a model contains personal data or enables its inference, retraining or deleting the model may be necessary. France's CNIL goes furthest: rights against a training dataset raise ordinary problems, rights against the model itself present "particular and unprecedented difficulties," and the answer is proportionality. Output filtering, documented suppression logic and audit-trail design can constitute compliance provided the reasoning is recorded, timelines can flex, and in some cases cost or technical impossibility can justify refusing the request outright.

So which is it? Four regulators, four incompatible answers to one question. Hamburg says the model holds no personal data. The EDPB says possibly, decided case by case. The ICO says sometimes the model must go. CNIL says filter the output and write down why.

The speaker side is no more settled. In Walters v. OpenAI a Georgia court granted summary judgment in May 2025, reasoning partly that disclaimers meant no reasonable person would take the output as an assertion of fact.

Stack those together and the position is incoherent in a fairly specific way. If the model is not a record of you, data protection cannot reach the weights. If the output is not an assertion of fact, defamation cannot reach the speech. A thing that is neither sits in a gap where neither body of law arrives.

There is also a remedy sitting in Opinion 28/2024 that needs no unlearning algorithm at all, and I think it is the one labs should actually be worried about. Where unlawfully processed data cannot be removed, enforcement can extend to erasure of the entire dataset and, explicitly, of the model itself. The US Federal Trade Commission has been ordering exactly that since 2019 (Cambridge Analytica, Everalbum, Weight Watchers and Kurbo, Ring) under the name algorithmic disgorgement.

Every argument that erasure from the weights is impossible therefore doubles as an argument that the only available unit of deletion is the whole model, which is an awkward destination for three years of industry explanation about why unlearning is intractable.

The obvious objection to this whole piece is that the ICO and CNIL have just demonstrated the opposite of its thesis: the framework is adapting, regulators are reading Article 17 sensibly against a new technology, and no rewrite is required. It is a fair objection (it nearly changed my mind) but I can see three problems with it.

CNIL is permitting refusal on grounds of cost or technical impossibility, and Article 17(3) is an exhaustive list on which technical difficulty does not appear. A supervisory authority creating a sixth exception through guidance is legislating without legislating, and a data subject refused on that basis has lost a statutory right to a document that binds no court.

Then there is harmonisation, which was the entire point of a Regulation rather than a Directive; and a right whose content depends on whether you file in Hamburg, Paris or London is being negotiated locally rather than interpreted. The opinion meant to settle this says "case by case" sixteen times and "may" or "might" a hundred and sixty-one times, which is a regulator declining to decide in the register of having decided.

And the EU has, I think, already conceded the argument next door. When it determined that general-purpose models needed obligations around copyright and training data it did not reinterpret an existing instrument; it wrote Article 53 of the AI Act, gave the Commission investigatory powers and attached a penalty of 3% of worldwide turnover. Three years of adaptation through guidance has meanwhile produced one enforcement decision, since annulled, and a compliance practice that filters outputs while telling people their data was deleted. At some point an interpretation this far from the text is more honestly described as a different rule.

Compelled forgetting turns out not to be a new problem at all, which I did not expect when I started looking. Several mature institutions have confronted some version of "make it not known", discovered that erasure was unavailable, and landed on the same substitute: stop trying to remove the knowledge, and regulate what may be done with it instead.

None of these fields was reading the others, and that is what makes the convergence worth something.

Retrofitting Article 17 onto model weights has given us three years of guidance, one annulled fine, and a compliance practice that amounts to hiding outputs. So what would I ask for instead? Not weaker protection, but protection that describes something a system can actually do and an auditor can actually check.

The remaining objection is that reframing is exactly what the industry wants: erasure is a bright line, use restrictions are a surface that erodes through a decade of carve-outs, and a reframe risks ratifying collection that was unlawful when it happened. The fourth move is the answer to it, keeping the sharpest remedy in the law rather than trading it away, and biting on provenance, which is precisely what was unlawful. A bright line nobody can measure is not a bright line anyway.

What you give up is the promise that you can make a system un-know you. That was never deliverable against any mind, biological or otherwise, and the law has never granted it anywhere else, so giving it up is closer to a clarification than a defeat.