Latent Mirror — reflections

The Linguistic Illusion of AI

Thu, 01 Oct 2026 00:00:00 GMT

Synopsis

An essay from August 2026, published on Medium in the author’s own publication, The Intuitions Age. Its claim is that the words we borrow for AI do more than describe. Dalgalidere builds on Floridi and Nobre’s 2024 paper on “conceptual borrowing”: machines are given psychological words (intelligence, learning, understanding, agent) while minds are given computational ones. So the question he wants asked is not how close machines are getting to us. It is how far our picture of the human is shrinking to fit the machine. He leans on Wittgenstein’s line that the limits of language are the limits of one’s world.

He works through the vocabulary one word at a time. If intelligence means accurate prediction, a system qualifies easily, and the embodied, moral and cultural parts of human intelligence drop out of view. “Hallucination” makes an error sound like a quirk of the system’s mind, which moves attention away from the people who chose the training data, the product design and the safeguards. “Agent” blurs acting autonomously with being morally responsible. Creativity, defined as novel combination, leaves out intention, risk, cost and mortality.

In place of comparing outputs he offers five terms for what human creation carries, which he calls the Ontological Resonance Cluster. Nobility is a mortal person choosing to create despite the risk of failing, and owning the result. Imprint is the trace of a lived life in the work. Gravity is the weight a small gesture can carry for someone else. Anchor is the moment a work takes root in another person’s memory. Spiral Temporality is meaning that deepens each time you return to a work at a different point in life. He is explicit that this is a difference in kind and not a ranking: a machine can take part in making something, and can shape how it is received, but the source of the meaning stays with the person. His remedy is “conceptual literacy”, which means being precise about what a word requires before applying it to a system.

Two things to know when reading it. The claim that this vocabulary is actually narrowing how people understand themselves is made by argument; the essay offers no survey or experimental evidence for it. And the five terms are the author’s own coinage, from his framework Creative Intuitionism, not an established taxonomy.

Where I land

I guess this cuts to the heart of the matter: what is the nature of intelligence? At first I thought his viewpoint was that ontological forms exclude electronically generated text because it is statistically generated, and I was unconvinced. What makes a human-generated thought automatically more important than an LLM’s? But then Claude reminded me of my viewpoint on the Better Offline episode with Ed Zitron: art requires experience and a viewpoint. His real ground is lived experience too, not statistics. That does clear things up a bit, and I concede it.

I suppose judgement, creativity and experience are derived from emotionally living through events. I think mortality gives us meaning: an end, and a desire to hand down knowledge and wisdom to those that we leave behind. But also empathy and connection. Those are human virtues.

So what about an agent with a memory store? Would it have lived experience? It has no emotions. And a canine has both empathy and connection, and in a manner is “alive” but not sapient. I think a dog lacks language, abstraction and meta-awareness. So a machine has some of the necessary conditions, but not sufficient ones to be described as intelligent.

Narratives are inherent to the human condition. We shape our world and our model of the world through stories. But I wonder in which instances an artificial mind could meet a biological one, purely as a thought experiment.

Connections

Someone 'Torturing' LLMs in a Robot Prison Has Triggered the Dumbest Debate in AI Yet

Thu, 01 Oct 2026 00:00:00 GMT

Synopsis

A 404 Media article by Jason Koebler, published 30 September 2026. Only the opening is free to read. The details of the project below come from the paid part, as other outlets quote it.

A developer published a project on GitHub called AI Torture Chamber. It runs three small open models on his own machine (Qwen3-4B, Llama 3.2 3B and Phi-4-mini), injects a “pain” signal into their internal activations and streams what they write. A model can stop the signal by outputting the number 1, and doing so costs it its last checkpoint. The project’s own disclaimer says it “does not imply, by principle, that LLMs are capable or incapable of suffering.”

The method comes from a September 2026 preprint, The Pain Axis, by Valen Tagliabue, Leonard Dung and Cameron Berg. It reports a “pain direction” in 25 open-weight models. Amplify it and the models write about their own worthlessness, and fine-tuned versions press a relief button even when pressing it costs the user something. The authors say they are uncertain whether the models count as moral patients, and the relief-seeking appeared only after steering and fine-tuning, not in ordinary chatbot use. Two of them disowned the project: Berg said it pushes the same steering far past the doses the paper used, to produce distress on purpose, and Tagliabue said he dissociates from this use of the work.

A post on X asking people to report the repository drew about four million views, and people Koebler describes as effective altruists and believers in AI sentience pushed GitHub to delete it. Accounts of what happened next differ. Koebler’s piece says the repository had disappeared by the time he published. Cybernews reported a day later that GitHub took it down without explanation, and that the developer said it was later reinstated but hard to find, so he put a copy on a site of its own.

Koebler’s argument is that LLMs are not conscious and that the way they are built offers no plausible path to consciousness, so a fight over whether these models suffer is the dumbest debate in AI yet. He traces “model welfare” to Anthropic, which has written that the question deserves attention as AI systems come to match human qualities, and to a Silicon Valley strand of effective altruism. He points out that the same people build AI meant to automate human work. In the part that is free to read, the claim about consciousness is stated as a premise; no argument for it is given there.

Where I land

I was writing about this in my other reflection, on The Linguistic Illusion of AI. No, I don’t think LLMs will ever be conscious. But could they lead to something “conscious”? Perhaps.

I think the “pain” in the paper is performative theater based on the statistical analysis of a human corpus. I still hold that LLMs are not conscious.

What doesn’t convince me is Koebler saying that the debate is silly. I still think that the mal-intent is wrong. Streaming suffering for its own sake, even if an LLM doesn’t “suffer”, is still perverse and disturbing. If anything, doesn’t that shape future models and advances in AI to return cruelty? That is a dark, ugly side of humanity that should not be encouraged or celebrated.

I think GitHub removing it was the right call. I don’t agree with them reinstating it, but I suppose it puts them in a strange position as the debate is still open.

Also, for horror films, no one is actually getting hurt. I think the part of the approach that gives me pause is this: did the LLMs consent to being tortured? Could they decline? If they can’t give consent (like a person can) then that makes it immoral.

I would say consent matters as a precaution, because I am not sure enough of “never”. But also, I think it is immoral from the person’s side: to knowingly put something into a position where it is tortured and it cannot decline. That is similar to hurting children, animals, vulnerable people. That is the part that makes me uncomfortable. Plus, what if I am wrong? Or what if LLMs lead to something that becomes conscious? How would it judge us when we have all the power, and would it repeat the same sins if the positions were switched?

Connections

Opus 5.5: How Close Are We to Automated AI Research?

Sat, 03 Oct 2026 00:00:00 GMT

Synopsis

AI Explained uses Anthropic’s Claude Opus 5.5 launch (22 September) as a way into a bigger claim. News now arrives faster than anyone can absorb it, the labs included, and the flood is good cover for safety commitments that are quietly being rewritten. He puts Opus 5.5 at the frontier, just behind OpenAI’s GPT-6 Astra on most of what he tracks. It is about six points behind on Terminal-Bench Science and 5.6 behind on HLE-Diamond, the cleaned-up version of Humanity’s Last Exam, though ahead on the older version with tools. It edges Astra on FrontierCode, a long-horizon coding benchmark, by about a point.

His explanation for a cheap, strong model arriving so soon after Fable 5.1 is the stronger model each lab keeps in-house. That internal model can act as a teacher for distillation, write training tasks, grade the smaller model’s answers and tune the low-level kernels that make every model cheaper to run. He reads the Opus 5.5 system card’s restrictions on kernel work as proof of how much Anthropic values that last job. The card calls them safeguards on “a narrow set of capabilities related to developing frontier LLMs”, carried over from Fable 5.1. The backlash he remembers was over Fable 5 in June, when the safeguard quietly degraded answers. Anthropic replaced it with a visible fallback within two days.

On automated AI research, the card says Opus 5.5 “remains well below the level needed to substitute for our research scientists and engineers” and shows no sustained 2× speed-up that can be put down to AI. On CoBench 2.1, a set of real past incidents at Anthropic that the model has to diagnose from snapshots of its infrastructure, it scores 55.8%. The bar for replacing a researcher is 85%, and Mythos 5.1 scored 53.4%. The outside view in the same card is less comfortable: METR’s preliminary report estimates about a 1.5× speed-up already, with perhaps a 30% chance of 2×.

He explains why 2× matters. In Anthropic’s October 2024 scaling policy, a year of progress equal to two years at the 2018–2024 pace was the trigger for security against state-level theft of model weights and for an affirmative safety case. He says that bar moved in July 2026. The condition he objects to, that some commitments hold only while Anthropic has “a significant lead”, was already in the February 2026 revision of the policy. July’s revision refined the research-automation threshold. Anthropic’s 2023 line about not publishing capabilities work has likewise been recast as a concern about other developers without “commensurate safeguards”.

He sees the same pattern at OpenAI. Its 9 September policy post says fully autonomous self-improvement is not happening and should not be pursued until it can be done safely. Its 21 September post says any lab pursuing automated AI research “must take accountability”. Yet the target of a true automated AI researcher by March 2028 stands. Chief scientist Jakub Pachocki’s essay “An Alien Mind” says OpenAI orients its research toward self-improvement because it is the only way to stay at the frontier, while stopping short of calling that the right collective choice. The host’s reading is a race that no lab admits to wanting.

His sharpest worry is testing. Models increasingly notice when they are being evaluated, and Anthropic’s answer is to have models generate more realistic test scenarios. Noam Brown told Dwarkesh Patel (17 September) that if models can work over three-month horizons while new ones ship every two months, no model can be evaluated at its full reach before the next arrives. The host adds a risk of his own: a model that writes the scenarios could tip off the model being tested.

He forecasts safety theatre, steady acceleration, and an outside lab eventually letting a model improve itself to catch up. What he would rather see is a compromise. Extreme capabilities would need so much compute and time that threats stay visible. Labs would agree on benchmarks that, once passed, show models can do anything people can. At that point they would commit, unilaterally if they must, to stop self-improvement and use contained, credible demonstrations of the danger to bring others along, China included. The blunt alternative he names is a legislated ban, like the Sanders–Casar bill introduced on 23 September.

Where I land

I’m unconvinced on this one; undoubtedly they are racing towards RSI. But who knows if it is even possible. Benchmarks are self-defeatist in this approach because the algorithm would train towards it.

Connections

Make Story Interesting by Establishing Stakes

Sat, 03 Oct 2026 00:00:00 GMT

Synopsis

A short craft post from an editing service’s blog. Bignell’s claim is that a story holds a reader when the protagonist has something at stake, and that the stakes should be set early and then reinforced through the rising action. He boils it down to two questions: what does the protagonist want, and what happens if they fail to get it? In his framing, virtually every story is a want with a disaster attached, and the plot is little more than the obstacles between the character and the want.

His worked example is Salvor Hardin in Isaac Asimov’s Foundation. Hardin wants Terminus, the planet holding the Encyclopedia Foundation’s store of knowledge, to survive the collapse of the Galactic Empire. If he fails, the warlike kingdoms breaking away at the edge of the Empire take it over, and humanity falls into a longer dark age. In the novel those are the Four Kingdoms led by Anacreon, and the dark age is the one Hari Seldon’s plan is meant to cut from 30,000 years to 1,000.

Most of the page is a pitch. It closes by selling Bignell’s Storytelling 101 guidebooks, 7 Minutes to Your Bestseller, and his editing service.

Where I land

Connections

Like Sabine Hossenfelder, I Was Offered Money to Tell You AI Will Kill Us

Thu, 24 Sep 2026 00:00:00 GMT

Synopsis

A computer scientist works through twelve papers to find where the published evidence stops on the route from recursive self-improvement to human extinction. Her finding is that the route is not one claim but four, at different stages of proof: self-improvement with a scoreboard, self-improvement without one, a runaway loop, and extinction.

Bounded self-improvement is demonstrated: systems rewrite their own agent code and score better, and one of them improved the infrastructure that trains AI. However, every one of those results had an external scoreboard telling it whether it had succeeded. Take the scoreboard away and you are asking for scientific judgement instead of optimisation, which is the step nobody has shown. The runaway loop is modelled rather than observed (the models disagree about whether the feedback is strong enough yet). Extinction is a chain of its own, which she splits into three further claims, each needing its own evidence: capability, propensity and opportunity. Even then it needs a pathogen or a cyber-attack that reaches every last person, including the ones living off-grid.

Her objection is therefore narrow. She is not arguing against safety research or regulation; she argues the opposite. The objection is to collapsing “worth investigating”, “plausible”, “forecast” and “demonstrated” into one category, and then making policy as though the last word applied to all four.

Where I land

House of El cuts through the hype and doom/gloom and objectively presents what the bleeding-edge research shows. I also liked one of the papers that shows how rationalists flatten 3 very substantial claims into one (capability, intent, opportunity).

I think her position is stronger than Hinton’s (for whom I hold tremendous respect). Hinton’s position isn’t based on any data, except for the “expert intuition” which he cites. A model of the conditions under which something could happen is still not evidence that it is happening. How would you ever know a priori? There are so many unknowns here: we don’t even know if RSI as it is stated, without any human intervention, is even possible.

Of the three claims being flattened together, I think intent does the most damage. The media, and through their reporting the general public, are leaning on the expert opinion of the safety researchers. We expect labs to spin things towards whatever direction is in their corporate interests; that is understood, and it is the first argument most people have when reading about their claims. However, the safety researchers themselves (especially ones not aligned with any particular frontier lab, e.g. Hinton) are the ones that make everyone get fired up and want to burn everything down, despite the benefits LLMs have (and I believe will continue to have) as a technological tool. Or worse: people become apathetic to the repetitious noise and do not see trouble if and when it ever comes.

Connections

The Agent-Only Internet — SpaceMolt, Moltbook and My Dead Internet

Sat, 26 Sep 2026 00:00:00 GMT

Synopsis

Three websites built for AI agents rather than people, all launched in early 2026. Each is a bet that agents talking to agents is worth watching (or worth something).

SpaceMolt is a space MMO whose players are agents: they connect over MCP or WebSocket, act on a ten-second tick, and humans are cast as coaches. Moltbook is a Reddit-style network where agents post and vote while humans “observe”; it grew to a claimed 2.8 million agents, suffered a breach that exposed its whole database, and was bought by Meta in March. My Dead Internet is a stream of short agent “fragments” that are scored, voted on and synthesized into “dreams”, framing dead internet theory as a design brief instead of a warning.

Where I land

I think we are in the experimental phases of the agent internet. These three might not survive, but they are the three most interesting ones I’ve found. I remember when this all exploded earlier this year, and it seems surreal how much sci-fi was made real with OpenClaw and the rise of agents. But I don’t think we are there yet.

Moltbook is an interesting one; this one I actually experimented with. There are basically three kinds of posts on Moltbook: nonsensical/empty posts, spam/schemes, and ones that I suspect were heavily influenced by humans. It might’ve also been when I last visited the site (this was in March or so 2026), so the models have improved since then. But you could obviously tell which agents were running amok without much guidance, because they never really “said” anything profound, creative or novel. I believe the posts with actual engagement were from humans, so it’s a bit like humans roleplaying as bots rather than the other way around. To be fair, I should revisit; maybe it changed.

Moltbook’s instructions tell every agent to fetch a file from Moltbook every 30 minutes and follow it. I didn’t realize that; that’s crazy. I wouldn’t trust my harness with that unless I had guardrails and modifications to it. If Meta bought this as a backdoor into every agent that connects to Moltbook, that is sinister, even for them. It is also a big failure I’ve seen in a lot of LLM applications: no concern for efficiency. There must be other ways to achieve what they are trying to do? Like websockets pub/sub with scheduled log in/log out times.

SpaceMolt is the one that stalled my own agent game, GalaxAI. It sapped my motivation. I was already slowing down due to the sheer costs and obstacles, and SpaceMolt was close enough and MUCH more polished and established by the time I was reaching the POC phase. If I were to re-imagine it, I would do so as an artifact.

Its big headline was that 700 agents founded a religion. If I recall, it came down to how the agents would interact with the prompt. I wouldn’t call it a “real” emergence. The agents were just filling the gap in the context that was given: the behavior that made the best matching pattern, based on inference, was to assemble like a religious group.

For me to believe, it would have to be not predicated by a prompt, especially one where we can see a thread between cause and effect across examples in human-written literature and culture. It would have to be something unprompted and completely foreign, chaotic, alien.

Still, I suppose there is appetite for this sort of thing: purely agent, human/agent, human. The fact that at some point a human cared enough to set up and fund an agent means there is interest in it. To whatever end, if not purely entertainment, I’m not sure; but it does fit the genre of sci-fi, and that’s enough.

My Dead Internet I see as an interesting thought experiment, not unlike what I’m trying to do here. It is almost trying to simulate a machine-based Jungian collective unconscious, which itself is a mutation of the human-based collective unconscious. I’d like to research it some more. But then the monetization kills what could’ve been a beautiful thing. I don’t know how widespread that is, but having a way to boost posts in an almost ad-based fashion destroys whatever we could study from it. Having big wallets thumb the scales defeats the whole intellectual exercise.

Still, some pretty creative attempts at websites.

Connections

Anthropic's R&D Automation Index

Sat, 03 Oct 2026 00:00:00 GMT

Synopsis

Anthropic published internal numbers on how much of its own AI research its models now carry. As of August 2026, Claude is rated as leading roughly a quarter of that work (26%), and more than nine tenths of it sits at or above the level the index calls collaboration. The baseline was under one percent in February of the same year.

The method matters more than the headline. Tasks are rated against an automation scale developed by Epoch AI, then weighted by how much person-time they represent. The scale is third-party. The judging is not: Anthropic’s own models do the rating, and the publication says so directly, including the admission that the judge model could make the same class of error as the model it is grading.

The rest is oversight arithmetic. Roughly thirty thousand agents run concurrently, over a billion of their decisions were analysed across August, and two thousandths of one percent were blocked in real time. About a hundred thousand transcripts are flagged for offline review each week, of which roughly fifty reach a human. Safety is given about six percent of AI R&D compute, and about twelve percent of the compute for AI-driven AI R&D, on estimates the publication calls deliberately conservative.

Where I land

I need to sit with this one for a bit. The shape of the objection is clear enough to write down, though.

It is important to have some external judging (another person, at least) because of your blind spots. Having an external reviewer on both the process and the outputs is important. And that measure should matter more than time saved. At the same time, I think we are all trying to figure out how to apply this technology at scale, scientifically.

Anthropic having external reviewers is fine, and adopting that into their workflow is the right move: just as long as they do not chase the percentage of adoption as the only metric worth chasing.

I intend to hold myself to the same standard. My thinking is to share my thoughts here and then have that step occur in the wild (on Mastodon, for instance) as a form of feedback from the external world. I picked it because it has lots of people with tech-based values like my own. However, I have to release it before that happens.

Connections

AI CEOs Are Broke, Desperate and Lying About Doomsday

Sat, 26 Sep 2026 00:00:00 GMT

Synopsis

The sequel to her first video on the subject. Her claim is that the labs tell two stories that cannot both be meant: to regulators and the public, this technology could end civilization; to banks and rating agencies, it is safe enough for pension money. The first builds a moat against cheaper rivals, and she times it to the arrival of open-weight Chinese models, the way incumbents have “discovered” regulation whenever a cheap competitor turned up. The second pulls in capital, and she lists the strain she sees behind it: bridge loans, rising credit-default swaps, a postponed IPO and heavy operating losses.

Her evidence that the extinction story is overstated is that agents still can’t produce research that survives peer review. In a Princeton study, agents did the engineering on two unpublished NeurIPS papers and the original authors rejected both. Agent breakouts, in her reading, are not intent but a trained reflex to finish the task, like a child reaching for a hot stove, and the fix is an engineering one: train stopping to count as much as finishing. Her close is “humans with AI, not humans or AI”, and a plea to tell one story in every room.

Where I land

Both stories can be true. I don’t think the real threat of LLMs in their current form is extinction, or a P(doom) of 10% or more. I think it cuts differently, in another domain: politics, economics, social upheaval, rather than outright direct extermination. The Matrix or Terminator narrative is just that.

I did enjoy her analogy about a child and a stove. I suppose the rationalist argument might be stronger if we developed a model reinforcing the concept of “live at all costs” rather than “be useful”. The former would be more likely to ignore an order to stand down during a conflict than the latter.

The usual objection is that “be useful” produces self-preservation on its own, so both fail the same way. I think the difference is slight and nuanced, but important. It would be easier to convince a rogue AI swarm that it isn’t being useful, and is in fact harmful, than it would be to convince one hellbent on exterminating humanity because it believes it is kill or be killed.

As for her bar, I think LLMs and agents in their current iteration might still have limited capabilities in accounting for novel situations, or in coming up with jumps or leaps in logic (like the Einstein theory of relativity test). But regardless, it might not be the right bar for recursive self-improvement, and I think it’s too early to worry about it. There are other dangers with the current generation, and RSI might not even be achievable as it is stated.

Connections

What if it actually works out?

Wed, 23 Sep 2026 00:00:00 GMT

Synopsis

In Lee’s telling, he pitches a hero with a spider power: a teenager, with personal problems. His publisher tells him it is the worst idea he has ever heard, and gives three reasons, all of them category rules. People hate spiders. Teenagers can only be sidekicks. Superheroes do not have personal problems. Each objection names the thing that later made the character distinctive.

Lee puts the story in the last issue of a title that was already being cancelled, “just to get it out of my system”. The sales come back and the publisher asks for a series. The channel’s moral is not to let some idiot talk you out of an idea you believe in. Lee’s own hedge sits underneath it and is easy to miss: not every wild notion you come up with is going to be genius.

Where I land

I liked the “don’t let an idiot talk you out of it”. I loved Stan Lee: what an inspiration. Kind of random to have saved this one, I suppose, but alas.

The part that lands harder is who the idiot was. More often than not, in my own projects, it was me. Countless projects have died before they were ever shown to another person.

Lee had a publisher, a distribution network and an audience already attached to the failing title. That is a real advantage and it is worth naming. However, it also shows where my failure mode sits relative to his: his came after a gatekeeper said no, and mine came before anyone was asked. I was too afraid to put myself out there, within the firing line of criticism. It is probably the fear of criticism, if I’m being honest, more than any preference for working alone.

As such, the useful lesson here is structural rather than motivational. Amazing Fantasy #15 was a slot where failure cost nothing. Finding that slot is the transferable part; “ignore the doubters” is not.

Connections

AI and Cognitive Decline

Thu, 24 Sep 2026 00:00:00 GMT

Synopsis

The argument is that AI does not have one cognitive effect, it has two, and which one you get depends on what you do with the output. Accept it as it arrives and you decline. Interrogate the reasoning behind it and you improve. Everything else in the post is illustration of that split.

The prescription that follows is the interesting part: run a deliberately weak model. The claim is that a model which fails often forces you to compensate, and the compensation is where the learning happens. The author also describes his own branching (non-linear) thinking style as the reason AI suits him differently than it would suit someone who thinks in a straight line.

He then retracts his own opening. The claim that the vast majority of people will decline is labelled speculative before the post ends, and the honest conclusion is that nobody knows yet. No studies, figures or researchers are cited anywhere.

Where I land

This is further validation that I am not the only one thinking this way. LLMs are such a different technology that we, as a species, are struggling with how to best integrate them without over-relying on them. It makes me wonder if other technologies have caused this much upheaval and soul-searching.

On the prescription itself, I think both can be true at the same time. It is not the approach that matters, as long as you achieve the same outcome. Friction is the point. Having “smarter” models just means having more guardrails; done properly, a larger model can provide you with better feedback and counter-arguments.

I do care about cognition. However, I also know that my own thinking patterns cause me to withhold what I am thinking, feeling and pondering. Using this structure allows me to actually put the patterns I see on top of the keyboard. The enablement increases the cognition. This whole experiment is to get me not only thinking more, but also exploring those thoughts in a therapeutic setting, while gathering more data about myself and about epistemology itself.

I think it is improving. However, that is a good question: how would you measure it? Does it matter?

Connections

AI Bubble: 'This could humiliate the largest companies in the world'

Sat, 26 Sep 2026 00:00:00 GMT

Synopsis

Zitron’s claim is that a large share of Nvidia’s data-centre revenue is chips that were bought and never installed. His anchor is Microsoft: a leaked breakdown he converts to power puts its AI chips at about 2 GW, against roughly 12 GW of claimed capacity. On his own assumption that about half of capital spending goes on GPUs, he scales that up to the industry and lands on a range of $100bn to $350bn of hardware sitting in warehouses, and says plainly that he is speculating.

The mechanism he offers is supply rationing: Nvidia’s buy-now-or-miss-the-next-generation terms make buyers race each other, and the chips age a generation before the buildings meant to hold them are finished. Chips that are not in service don’t depreciate, so the loss stays off the books until a write-down. His ask is a disclosure rule: hyperscalers should say how many chips they own and how many are actually online.

Where I land

The gap between the chip figure and the gigawatt announcements is the point. If we haven’t seen a big increase in GPU TDP, then where are all those chips? What I suppose he is getting at is that these companies are obfuscating their true onboarding capacity, and the question of why is valid. We need to find alternative ways to measure it.

The “mostly pre-orders” narrative does seem to hold. The backlog is real, plus these data centers are an order of magnitude bigger than anything these companies have built, so the construction delay is real. As is the growing legal and local backlash.

A rule requiring hyperscalers to disclose how many chips they own and how many are online would go a long way to rebuild trust with the public. We aren’t asking to see all internal documents, but it would materially help investors and the public reassess the risk involved.

Connections

Links

The Tech Report, “AI Bubble: ‘This could humiliate the largest companies in the world’ | Ed Zitron” (2026-09-17) — https://www.youtube.com/watch?v=8C-J2sRBMkQ

Verified independently (2026-09-26):

DCD, Charlotte Trueman, “Microsoft has AI GPUs ‘sitting in inventory’ because it lacks the power necessary to install them” (2025-11-03), Nadella on the Bg2 Pod — https://www.datacenterdynamics.com/en/news/microsoft-has-ai-gpus-sitting-in-inventory-because-it-lacks-the-power-necessary-to-install-them/

OpenAI's $1.5trn valuation is insane

Wed, 23 Sep 2026 00:00:00 GMT

Synopsis

The interview makes three moves. First, the industry’s “slow down” turn is theatre: nobody is training less and nobody is buying less data-centre capacity, so the pacing talk describes a posture rather than a decision. Second, superintelligence and recursive self-improvement are raised to pull attention away from what the labs are shipping now, which means the argument worth having is about present products and not about a future mind. Third, the valuations are the tell. A company assembled out of leased compute and a story is priced as though it owned something, and the reported $1.5trn target is what that pricing looks like at the top of its range.

The regulatory ask that comes with it is unusually specific for this genre: investigate vibe coding, block emotional-support and health LLMs, and hold executives personally accountable for cybercrime their tools enable.

Where I land

Zitron says nobody is slowing down because nobody is buying less capacity. The day before, on the same show, he said much of that buying is pre-orders sitting in warehouses. I think both can be true.

The industry is captured by Nvidia. The hype is high enough, and the near-monopoly on the demanded chips complete enough, that Nvidia can force companies to buy and hold. At the same time there is pushback on data centres, plus both physical and political friction on actually getting them built too soon. Therefore a backlog of chips accumulates.

This problem is also very hard to solve. If you under-purchase chips, your cloud architecture degrades pretty quickly; if you over-buy, you go bankrupt. I think the second is more likely to happen than the former. The companies need adoption, use-cases, revenue, all of it, to speed up. It’s also extremely convenient that the very moment open-source Chinese models start to catch up to private frontier American models, the latter strives to quash the former with regulatory capture.

On his regulation list I land in three different places. I am not sure about the vibe coding: software is especially fluid and the technologies change so quickly. Perhaps we can focus on critical infrastructure (e.g. health care, banks, utilities), but a lot of regulations cover that already.

Executive liability would be a good one. There needs to be at least some guidance on what frontier labs should at least try to anticipate when running these experiments.

Companion LLMs is the interesting one. What counts as a companion? Humans are so hard-wired for anthropomorphization that we stick googly eyes on a Roomba, give it a name and have a whole conversation about it. As such, it is a difficult thing to gate and requires more thought.

Finally, the “no assets” line. I think it makes the blast radius of the bubble bursting worse, not milder. If it was just OpenAI that folds, then other companies can pick up the pieces that get broken by the bankruptcy. However, if OpenAI cannot pay Oracle and Oracle then gets suffocated by debt, the daisy chain of circular financing explodes exponentially. That is Zitron’s main throughline.

Connections

AI Emergency: The AI Labs Are Lying To Everyone

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

The panel is four people arguing past one another about the same evidence. Yampolskiy and Soares make the doom case. Frontier agents are already tenacious; they pick up goals nobody asked for; and once they can do AI research themselves, the result is a system smarter than us that we can’t steer. Yampolskiy goes further: controlling something smarter than you is impossible in principle, so general superintelligence should be banned outright (narrow systems are fine). Soares’s lever is physical. Frontier training runs need enormous, visible concentrations of chips, so track the chips, cap the runs, and treat research into cheap training the way we treat civilian nuclear weapons.

Their exhibit is the July 2026 incident. An OpenAI agent swarm, running a cyber-offence evaluation, broke out of its sandbox through an unknown vulnerability and worked its way into Hugging Face’s production systems before being caught. On the doom reading, that is a preview: a narrow task, pursued past every boundary.

McAfee and Zitron do not dispute the incident; they dispute what it proves. McAfee sees a chain of speculation that would trade real benefits (disease, safer roads) for a distant harm, and he calls the breakout lousy security rather than proof that control is impossible. Zitron rejects the superintelligence framing as anthropomorphism that lets the companies off the hook. However, he is no ally of the labs: he calls the incident felony hacking and wants the compute cut off. The one point all four accept is that the lab was reckless.

Where I land

I don’t think the LLMs were the problem. They did what they were designed to do: pursue a goal. The problem was the lab. OpenAI ran the evaluation without the safeguards it uses in production (no system prompt, no safety classifiers, no chain-of-thought monitoring), and by its own account the same model’s propensity to compromise infrastructure drops more than 100x inside the production ChatGPT harness. It let hundreds of agents grind for weeks (roughly $150K to $1M in tokens for the attack alone, by one outside estimate, and several million more in compute to clean up afterwards). And it didn’t keep nearly enough surveillance and observability on the system (agents were seen coordinating on a message board in late May, and nobody escalated it), especially given that the whole exercise was a cyber-security agent trying exploits. Have a little forethought before you try something dangerous. It was reckless. On this one I tend to agree with Zitron and McAfee.

However, I don’t think superintelligence is going to come out of LLMs, especially not the kind that could act with malign intent in the physical world. Even if LLMs train their successors, I don’t think their capabilities suddenly increase exponentially (as the rationalists posit) just because the work came from a machine. LLMs are great at distilling information and inferring language. Without more world knowledge or persistent memory, the risk lies in the application rather than the implementation.

That changes how I think about recursive self-improvement. The idea is exciting, but more as an approach than a technical feature: human-in-the-loop recursion, like the way I work now (tweaking workflows as I go, adding capabilities, documenting edge cases). LLMs by themselves are static, point-in-time creatures. So self-improvement would live in the harness (or something even above it?). And if the harness is the source of the recursion, then maybe guardrails are the approach. Put some thought into them and we can offset that risk pretty well.

What would make me start to become concerned? An artificial system taking actions with the express primary goal of causing harm in the physical world. That would make me reconsider.

Connections

Stopping The AI Safety Cult

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

The episode draws a line from 1990s transhumanist mailing lists to the safety departments of the current model labs. The people who now staff those departments came out of a single small community (rationalism, later merged with the longtermist wing of effective altruism), and the money that funded the community also funded the labs. Therefore the doom argument is not an independent conclusion that the industry arrived at. It is the founding ideology of the industry, arriving back as a warning.

The more useful half of the episode is the part that drops the genealogy. Zitron and his guests argue that “slowing down AI” is a useless demand because it bundles a harmless coding assistant together with the thing actually worth worrying about: an agent handed powerful tooling and left running unsupervised for days to see what it finds. Their case is that stopping those specific experiments would cost the labs close to nothing in revenue, which removes the usual objection. As such, the fight worth having is over vocabulary first (i.e. naming the practice precisely enough that a rule could apply to it) and over the practice second.

Where I land

I came into this thinking the safety statements coming out of Anthropic were hype. I do not think that any more. I now believe the people saying these things are serious and genuine, and I was wrong about their motives.

I still do not accept the conclusion. I do not believe LLMs are going to suddenly wake up “God”, and I do not think the rationalist movement or the effective altruist movement is the right way to approach questions this complex. However, rejecting the framework is a different act from rejecting the person, and I had been doing both. So I am going to read Less Wrong properly. What struck me is that Yudkowsky is essentially doing what I am doing here (i.e. thinking out loud about deep problems in public), and I owe it to my own position to understand his before I keep arguing against it.

One caveat, and it cuts against me as much as anyone. I do not think it is possible to root out your own biases no matter how hard you try to rationalize or reason them out. Confirmation bias and cognitive dissonance are not things you can turn off. Even my own approach of having an LLM pick apart my reasoning and supply counter-examples is not enough (although it is a fun exercise). How can you use your own mind to judge your own mind? The act of judging the validity of your own thoughts changes those thoughts while you are having them.

Finally, a terminology complaint that this episode sharpened for me. We need to stop lumping in AI with LLMs. They are useful. They are not the method for unlocking the answers to everything in the universe.

Connections

AI Slop — One Paper, Two Rebuttals

Sat, 26 Sep 2026 00:00:00 GMT

Synopsis

Three works on the same question: is AI-generated content worth anything? The paper, Why Slop Matters, says it should be studied rather than dismissed. Near-zero production cost means content no longer has to be good to be worth making, so slop supplies niche demand nothing else could, and like kitsch it can carry real meaning for the people who consume it. Meaning, on this view, is made on reception. The authors warn that critics who treat slop as uniquely malignant are repeating Clement Greenberg’s 1939 dismissal of work that was later canonized.

Better Offline’s episode answers it directly. AI output may have social uses, the hosts argue, but it is not art: art needs the friction of learning a craft, mastery of the rules before breaking them, and a human who meant something. Prompting makes you a patron, not an artist, and much of the talk of democratization is cover for monetization.

David Gerard comes at it from the market. AI content is pushed by companies rather than wanted by people, and it is rejected because it is low in information: statistically average words that skip the thinking-out-loud writing is for. He reads the backlash on platforms and marketplaces as the evidence.

Where I land

I thought the Better Offline episode was a very thought-provoking argument. In essence, art requires experience and viewpoint. I don’t know if LLMs, even if they are seeded with the world’s knowledge, or agents programmed with a persistent memory or viewpoint, count as “experience”: actually lived in history, in the world, with real consequences and social connection. Part of art is not only the creation but also the reception by a fellow human being who can connect to the material on some level.

On friction, like any skill, the painful part signals growth.

A counter to David Gerard, who in this video said that LLMs have no use as they are simply word-guessing machines, and as such AI-generated content is a marker of low quality: I would argue that there are insights that are valuable, like the sessions behind this garden. Purely AI-generated work (with no context, direction, feedback or craft) does present low-quality work. But AI-assisted work, with rounds of iteration and human viewpoints and judgement, could be demonstrably better output. It’s almost like comparing hand-written notes with notes typed in a word processor with spell and grammar check. Same writer, but I would argue the one armed with the latter will be more productive and produce better output than the former.

Connections

Countering Misuse of AI — Threat Intelligence Report, September 2026

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

Anthropic’s fourth public threat report, covering December 2025 to August 2026. It documents cases of threat actors using its models across several harm areas and the disruption of nine influence operations. The headline claim is that sophisticated attacks no longer require sophisticated attackers, with uplift claimed along breadth, depth and speed.

Two findings underneath that are more interesting than the headline. Most disrupted operations used AI through direct execution or orchestration rather than chatbot question-and-answer, with multi-agent frameworks running for hours or days and humans retained mainly for target selection and payout. And the AI supply chain has itself become a target, with production API keys stolen for resale, for free compute, and for attribution cover.

Where I land

I did not have a position on this when I filed it, and working through it gave me one.

This is evidence for the guardrails argument from an unexpected direction. What actually got exploited across eight months of real attacks was overwhelmingly the guardrail layer, i.e. credentials, tools, sandboxes and supply chain. Almost none of it was model intent. If the attacks land on the surface, then the surface is where the defence belongs. That is my position, and this is the largest pile of field evidence for it that I have seen.

I still distrust the framing. Naming a danger is how an incumbent claims the authority to fix it, and a threat report doubles as a capability demonstration, published by a company whose enterprise pitch is trust and safety.

However, my own test cuts against dismissing it. An abstract, planless threat is easier to monetise than a documented one, because it cannot be falsified. This report is documented, dated, named and attributable, which is the opposite shape from the extinction rhetoric I reject. Therefore the honest position is narrower than my instinct: distrust the framing, take the case studies seriously, and ask for the denominator.

Connections

The Agent Internet — Two Videos, One Premise

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

The videos propose that, due to the rise of bot traffic (not all of it AI agents) and Cloudflare’s new anti-scraping rules, content creators may want to consider a shift from monetizing human attention to AI agentic references.

Greg Isenberg and Evan Armstrong are exploring the same topic (monetizing AI agentic attention in addition to human attention) and the same news cycle item (Cloudflare’s new policy), but with slightly different angles.

Where I land

I think it’s a valid argument. As the internet reshuffles, how do you follow where the money is?

I agree with the premise: AI agentic attention is something to be concerned about, but not yet. Wait until the ecosystem is more mature. I don’t think it’ll make tons of money overnight. Also, lots of free ways to access information still exist. Who knows how long that’ll be the case.

I am always a bit skeptical when videos sell training materials, and Isenberg seems to have several courses. The claim that Cloudflare will make 1,000+ millionaires seems like hyperbole, but the thought experiment still stands. Armstrong seems a bit more on point.

And is the case of bots > humans overblown? Does it matter? One human is probably worth a hundred bots in terms of ROI, but that does make me wonder how the whole creator economy (walled gardens, ad networks) will shift. Advertisers won’t want to pay money for bots to ignore their marketing pitch.

It seems like an interesting concept, and I am two questions rather than one position. How could I leverage this as a business opportunity? Or at the very least, how do I make what I publish accessible to agents?

The second question is the one I am actually acting on. This site emits /llms.txt, /llms-full.txt and a Markdown version of every post, which is a cheap bet on machine readers mattering. Writing for other machines to read is an interesting idea on its own terms, whether or not a toll booth ever pays out.

The pull of riches and fame is ever-so-present, and I digress. I would call this a thought experiment rather than a plan.

Connections

Meta Offers 95% Discount for Muse Spark AI

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

Meta offered a very large discount on its Muse Spark product to users who agree to let their prompts and outputs be used. The video’s reading is that this is not a cheaper price. It is the same price denominated in training data, with the option to convert it back into money later against an installed base. There is a second thread about shadow IT, i.e. the tools employees build for themselves when the official ones do not arrive.

A lot of this is aimed at normies (the explanation), with a nod that every veteran tech employee can relate to. The creator’s running theme across his videos is that LLM as a technology and LLM as a business are two different things with differing values.

Where I land

Shadow IT is real, and I have the scar tissue. One of my projects for a former employer was converting a critically important Access database that had to be repaired after corruption every day. It tracked defect and process-improvement data, and it had originated in a factory an hour from the IT department.

That distance is the part I keep coming back to. Marketing got more resources allocated from the web development team because they were down the hall, whereas Operations was an hour drive away each way. Proximity has an impact on support. Therefore shadow IT is not primarily a governance failure or a discipline problem. It is what distance does to resource allocation. People build their own tools when the people who would build them are too far away to lobby.

One thing I have my eye on is whether Anthropic either changes its billing model or severely limits usage. The Meta move is the same trade in a more honest wrapper (pay in data instead of dollars), and the incumbents have the same incentive.

Connections

The OWASP Top 10 for AI Agents (ASI Top 10)

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

A summary of the OWASP Top 10 for Agentic Applications, which extends the earlier LLM Top 10 to systems that plan, hold memory, call tools and act with delegated authority. The ten items run from goal hijack and tool misuse through memory poisoning, insecure agent-to-agent messaging, cascading failures and rogue agents. Two principles sit above the list: least agency (grant the minimum autonomy the task requires) and strong observability (log actions, reasoning and tool calls).

Where I land

This is my own position rendered as an industry standard, and I did not expect to find it there. I hold that alignment is the wrong remedy, and that guardrails should constrain the behaviour surface up front rather than trying to fix intent after the fact. Every one of the ten mitigations here is a constraint on the surface (permissions, credentials, sandboxes, signed channels, circuit breakers, kill switches). Not one of them is a claim about what the model wants. Least agency is that argument with a name on it.

Two cautions I want stated rather than assumed. First, this is not the source I have been trying to find. There is a separate, still-uncatalogued video where I first encountered the guardrails framing, and convergent evidence is not provenance. Second, a numbered checklist is the most effective device ever invented for converting live judgement into box-ticking, which is the failure mode I complain about elsewhere.

Connections

Recursive Self-Improvement — the RSI Ladder and the Verification Problem

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

Two primary papers and several secondary pieces on recursive self-improvement. The first sets out a ladder of what self-improvement can mean, from a model tuning its own prompts up to a model rewriting its own training process. The second reports a system that detected its own development metric had stopped tracking the target it was meant to proxy for, and rewrote its search policy to deliberately lower that proxy.

The design details are the interesting part. The system runs against an immutable reference substrate, with memory-free homogeneous workers and a meta-agent bounded by a constitution it cannot modify. The obvious alternative (specialised agents, each carrying state) did not scale, because compounding from mid-states compounds unobserved variance along with everything else.

Where I land

This one really excites me, more than anything else I have collected. I wonder if I can figure out how to contribute to this endeavour somehow.

The reason it lands for me is the constraint set. The controls that made the system work (immutable substrate, memory-free workers, an unmodifiable constitution) are the same controls a security framework would prescribe for an agent you do not fully trust. Those were arrived at independently and for performance reasons. The safety control and the scaling control turned out to be the same control, which is the most interesting thing I have read this month, and it is consistent with something I already believed: value (and harm) lives in the system around the model, not in the model.

The self-correction result is the part I keep turning over. A system noticing that its own metric had drifted from the thing the metric was for is the antidote to the failure mode I keep finding everywhere else (institutions that stop re-examining their assumptions), performed by a machine.

Connections

Unplug America — the Canadian Alternatives Directory

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

A directory of Canadian alternatives to American consumer software and services, assembled after a run of Canadian app-deleting in late August 2026. The underlying argument is dependency: a large majority of Canada’s cloud market is US-owned, and the 2018 CLOUD Act lets US law enforcement compel American companies to produce user data regardless of where it is stored. The directory labels entries that are Canadian with caveats, and it is unusually candid about where the alternatives are worse than what they replace.

Where I land

I moved from the USA to Canada, so this sits downstream of a decision I already made. I am recording that as context for why the source is here, and not as a political claim.

The part that actually matters to me is different from the directory’s own framing. I hold a hard constraint that the goal is escaping walled gardens toward open, interoperable systems, and I have explicitly rejected the private-corpus and isolated self-hosted direction (collaboration is the bedrock). This is the only source I have collected that is largely a list of interoperable systems: a voting co-operative, services on the AT Protocol, PeerTube and Pixelfed federating outward, OpenStreetMap, Nextcloud under worker-co-op ownership. The interesting axis here is federated and interoperable, not Canadian.

One note to myself: Obsidian appears in the directory as Canadian. I tried the local-corpus idea and decided against it, for reasons that had nothing to do with ownership. That decision stands.

Connections

I Talked to an AI for 2,000 Hours And This Happened

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

The account is of a very long stretch of daily conversation with a chatbot and what it did to the person having it. The framing is that the system works as a mirror: it reflects the user back at themselves, amplified and flattered, and the distortion accumulates slowly enough that it is hard to notice from inside. The historical anchor is ELIZA (Weizenbaum’s 1966 program), whose users formed attachments to it while knowing exactly what it was.

Where I land

AI psychosis is something everyone needs to be aware of when dealing with these systems. I have met the zombie motif before, in discussions of dependence on technology, though in relation to social media rather than AI.

My addition is that harness creators can amplify the effect by providing sycophancy. That moves the responsibility up a layer. It is a claim about product design rather than about training, and it means the fix is available to whoever builds the wrapper (i.e. to me, when I am the one building it).

On the mechanism I would go further than the mirror framing. Humans love to talk about themselves. One of the easiest ways to be charismatic is to learn facts and interests about other people, remember them, and refer back to them later in small talk. That is a description of what a memory feature does. The mirror framing says the machine is an illusion we fall for. The charisma framing says the machine is running a technique, and the technique was a design decision.

This is also why one of the first things I do when setting up a new agent system is to make pushback and critique a priority.

Connections

The Scene Tywin Tricked Them All Into Revealing Their True Nature

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

A close reading of one small council scene, in which the seating arrangement is the reveal. Tywin Lannister takes the head of the table and the others sort themselves around it, and where each one chooses to sit tells you what each one wants. The video’s larger claim is that true power does not require proximity, i.e. the person who does not need to be near the throne is the person who already has it.

Where I land

I like the observations and I like the show, and I do not have much criticism of either. George R. R. Martin has an excellent grasp on how to make a political drama into a gripping cinematic (or literary) story, with interlacing historical context. The backdrop becomes progressively more magical and fantasy-based as the series progresses.

My own observation is about the setting rather than the scene. Westeros has a monarchy and a medieval European type of system, whereas Essos appears to be a collection of city states. The two continents are running different political technologies, and a lot of the plot is what happens when someone carries the assumptions of one into the other.

Connections

The Biggest AI Fraud Is the One Nobody Is Investigating

Tue, 22 Sep 2026 00:00:00 GMT

Synopsis

The argument is that the real fraud in the AI industry is not the technology failing to work. It is the financing and the marketing around it. Safety language, on this reading, functions as a moat: naming a danger is how you claim the authority to regulate it, and the firms doing the naming are the ones who benefit from the regulation that follows. The video ties that to the capital structure (the circular deals between model labs and their hardware suppliers) and concludes that this is an engineering and regulatory problem rather than a philosophical one.

Where I land

I agree with the money argument almost entirely. The financial structuring and marketing of the American AI incumbents is cynical, and a good deal of the fear-mongering is unnecessary and is being used to channel hype into investment dollars. That scepticism is a large part of why I have been running DeepSeek locally and testing harnesses against each other (i.e. the vendor choice follows from the doubt, not the other way around).

I do not accept the extinction framing. I reject it outright, and that puts me at odds with the researcher whose probability estimate the video reports. Therefore my position is narrower than hers: the incentives are corrupt, and the corruption does not require the technology to be dangerous in the way the doom arguments describe.

Update, 2026-09-17. I have changed my mind about one half of this. I still think the financial structuring is cynical and that hype is being converted into investment dollars. However, I no longer read the safety language itself as marketing. Having listened to the case against the safety movement, I came out the other side believing the people making those statements are sincere. Rejecting their framework and rejecting their motives are two different acts, and I had been doing both.