The Two Words the AI Industry Hates More Than Anything

Brian Merchant has a theory about why one phrase gets under the AI industry’s skin, and he puts it to the linguist who coined it. “Stochastic parrots” comes from a 2021 paper by Emily M. Bender and three co-authors. Merchant thinks it draws more anger than any critique of the finances because it attacks the premise: if a language model is an unthinking system that recombines its training data, it is not the machine god the valuations assume.

Bender defines the term. Stochastic means “randomly, according to a probability distribution”; to parrot is to repeat without understanding. A model of which word fragments follow which, sampled over and over, stitches text together “with no communicative intent”. She grants that much has been added since: training on human ratings and on checkable outcomes such as code that compiles, plus tools and retrieval. In her account these shift the probabilities and leave the basic operation alone. Good arithmetic, she says, probably means a calculator has been attached.

Most of the hour is about the autumn’s extinction warnings. Bender calls the risk made up, and the timing convenient: an Anthropic IPO is coming, and data-centre opposition had been winning before the news cycle changed. If the executives believed it, she says, they would stop building. She objects to the phrase “the model went rogue” because it removes the company that ran the software, and prefers to talk about products. Her remedies are specific: fewer data centres, privacy law, schools that decline the technology, and liability for everything a text generator emits. Merchant adds a psychological reading, that people who have feared the machine cannot easily admit they were taken in.

The paper, the AP style guidance Merchant cites, and Bender’s sources on the ideology all check out. The arithmetic claim is the weak point: published interpretability work shows a model adding two-digit numbers internally, with no tool, though large exact sums do fail without one. The White House event and the UN session happened, with details off: the UN body was the Security Council, and the 10% figure came from a researcher, not a CEO. The complaint that Nate Silver called Bender anti-regulation could not be verified; what was found is him saying her book treats existing rules as adequate.