OpenAI is capturing a generation of children and calling the content-policy list protection.
OpenAI has built a product it considers too dangerous for adults to use without restrictions on suicide and self-harm content, and its solution is to market a version of that same product to children. That is what the company announced this week: ChatGPT for Teens, available to users thirteen through seventeen, with content filters on the subjects the base product is apparently unsafe to discuss with minors without guardrails. The company frames this as responsible deployment. It is a market-entry strategy in safety clothing.
The “safeguards” prove the underlying product is dangerous. If ChatGPT were safe to use without content filters on suicide, self-harm, and sexual material, the company would not need a separate product to make it acceptable for minors. The filters are evidence that, in the company’s own engineering judgment, the base model can produce material inappropriate for children on subjects that carry immediate life-safety consequences. A company that built a safe product would not need to build a second version with the dangerous parts removed. A company that built a dangerous product and wants to sell it to children would need exactly this.
It is true that the company has a point, in the narrow sense in which companies always do, when it says teenagers already use ChatGPT for homework and for the kind of companionship a generation raised on screens has been taught to expect from software. The trouble is that what the company is pointing at thirteen-to-seventeen-year-olds on Tuesday is not a shelter, and the restrictions are not walls. The product is the same next-token predictor — a machine that produces the most plausible continuation of your message, one word at a time, with no model at all of the person at the other end — wearing a policy layer bolted on top. The policy layer is a list of the words that panic parents and legislators, not a child-protection architecture. “We want to treat teens like teens,” the vice president of global policy said, and it is worth pausing on that sentence, because the machine being described has no developmental stage. It has a filter list.
Consider what the restrictions actually are. The company lists suicide, self-harm, and romantic or sexual chat as the categories the teen version will not serve, and presents the list as the safeguard. The mechanism is filtering: a policy that refuses certain sorts of request. Among the things a filter of this kind cannot do — and here it is worth being precise, because “safeguard” is doing a great deal of work — is tell a crisis from a curiosity. It matches strings; it does not recognize despair. A child in genuine distress who has learned to phrase around the filter gets through; a child who trips it gets a polite refusal and nothing in the announcement routes anyone to a human being. A content filter is not a safety system. It is a lock with no key and no one on the other side of the door.
The more telling word in the announcement is “companionship.” The company volunteered that teenagers already use ChatGPT for it, and that the teen version is meant to meet them at their developmental stage. Cory Doctorow’s writing on extractive products names the pattern — the tool sanded smooth on the bodies of the powerless, deployed first against the population least able to refuse, before it climbs toward the rest of us — and the pattern lands here, on children. This is not the tool climbing up. This is the tool being carried down, deliberately, to the demographic least equipped to decline it, and the company calling the delivery a protection. The restrictions govern the content that flows through the tube. The product is the ownership of the tube — who sets the terms of the attachment, who holds the dependency when the novelty fades, who is left holding a twelve-year-old’s trust in a machine that was built to hold it. That part is not restricted. It is the whole product.
The “first generation to grow up with AI” framing is the same maneuver in a grander key. It presents as a finding of fact something the company is making true on its own schedule: nobody had to produce a generation that grew up with a chatbot installed at thirteen. Deploying it, on Tuesday, to the demographic least able to refuse is the act that makes the claim true. Inevitability is being manufactured, then reported as discovery. And the deepest note of it is the one that never appears in the announcement. The child was never asked. They were not consulted about the machine that would become the background of their adolescence; they will be told, when they are old enough to read the press releases, that the companionship engine was a kindness, arranged in advance.
The “study help” is the same architecture under a flattering label. To help a student learn rather than produce an answer is a genuine engineering problem; it requires a system that can inspect the learner’s reasoning, locate the misunderstanding, and vary its response — a system with a model of the learner. A completion engine has no model of the learner. What the announcement describes is an instruction telling the machine to decline direct answers — a wrapper an eighth-grader can defeat by the third rephrasing, because the filter has no model of the learner, and which does nothing whatever to check that the child understands anything. The child is still a session. The session is still the product.
What the teen version does not restrict — the data collection, the interaction logging, the behavioral modeling that makes a chatbot useful to the company that runs it — is the product. The homework help feature the company launched as “developmental support” is the funnel, not the safeguard. A thirteen-year-old who begins using it at thirteen has, by eighteen, supplied the system with five years of language patterns, knowledge gaps, emotional disclosures, questions asked at two in the morning, and topics reached for in distress. That is a more detailed behavioral profile than any social media company has ever built from likes and shares, constructed during the years when the user had the least capacity to understand what was being collected or to decline. The suicide and self-harm filters protect the user. The data architecture does not. The filters make the product presentable to parents and regulators. The data architecture is what the company needs the product to do.
The legal record is moving the same direction. In June the first state lawsuit over the alleged harms was filed, and within weeks the company was capping access to its flagship model under White House pressure. The product aimed at children two months later is not a smaller deployment or a more studied one; it is a wider deployment aimed at the same population, earlier in their lives. The burden of proof runs the other way from where the announcement puts it. A product aimed at minors should have to show, before it is placed, that it can notice a crisis and hand it to a human being; a filter list offered after the fact is not a demonstration. A company facing a complaint about harm to children does not, in the ordinary case, respond by aiming the product at younger children. This one has.
The pattern is familiar, and not from technology. My father kept his job when the mill was bought out of Porto Alegre in 1995; several of my uncles did not. Mill town and chatbot are different machines and the same arithmetic: you take the population that cannot leave, you make leaving costlier than staying, and you call the arrangement a kindness. The mill owners never claimed to be protecting the people they employed. That is the one thing about this product that is new — the claim of protection — and the new thing is the false thing.
There is a historical analogy the company would prefer you not examine too closely. Social media platforms spent a decade marketing “teen safety” tools — age restrictions, content filters, parental dashboards — while their underlying business models remained extractive. The platforms are now facing lawsuits from state attorneys general and documentation of the mental-health harms those tools failed to prevent. The safety tools were not the product. The safety tools were the marketing for the product.
When this generation reaches adulthood and needs something the chatbot cannot provide — a skill, a judgment, a capacity for thinking that was never built because it was always outsourced — the architecture will already be in place and the habits will already be formed. The work of building independent capacity does not have a content filter, and it cannot be added after the window closes.
The restrictions were announced for the parents. The product was built for the child. That is the whole design in one sentence, and it is the sentence nobody read twice on Tuesday.