Latent Mirror — cognitive science

The INTJs Cassandra complex

Mon, 19 Oct 2026 20:00:00 -0400

Synopsis

A ten-minute narrated essay from the Psychology & Meaning channel, a companion to its video on the costs of being a high-IQ INTJ. It retells the myth of Cassandra, given true prophecy by Apollo and cursed never to be believed, and applies it to the INTJ type: someone who sees a pattern before the evidence for it exists, cannot show the steps, is waved off, and is proved right later without thanks. An illustrative story about “Steve”, who flags a launch risk six months early with no spreadsheet behind it, carries the idea from the office to friendships.

The video’s explanation is the Myers-Briggs function “introverted intuition”, described as a background radar that assembles weak signals into a conclusion, where most people want the evidence first. It then makes a concession: sometimes the problem is delivery, since certainty without visible reasoning sounds like arrogance or a verdict. Its remedies are to lead with two or three visible threads of reasoning, pick the moment, and keep a quiet, dated record of calls so that the receipts exist when the pattern completes. It closes by asking whether silence protects you or only spares you being disbelieved.

The video cites no studies, and the checking finds three things. First, “Cassandra complex” is not a clinical term. The philosopher Gaston Bachelard used it in 1949, and the Jungian analyst Laurie Layton Schapira wrote a book under the title in 1988, about hysteria and being disbelieved rather than about foresight. Second, the function framing has weak support: the Myers-Briggs instrument scores four preference scales, and the ordered functions behind “introverted intuition” are inferred from the code, not measured (see the companion page for the sources). Third, the story counts only the hits. Since Fischhoff’s 1975 experiments, hindsight bias has been shown to make known outcomes look obvious in retrospect, so remembered early warnings that came true are weak evidence of foresight unless the ones that did not are counted too. Kahneman and Klein’s 2009 review sets the conditions under which gut judgement earns trust: a regular environment and the chance to learn from feedback, and neither depends on personality type. The video’s own last remedy, a dated record of predictions, is the test that would settle whether the intuition is as good as it feels.

Where I land

This might turn into a post I will write. Sometimes I react to things intuitively and I can’t put my finger on why. Is it confirmation bias looking backwards through time, attributing chance to wisdom, or is it something more? When I listen to my “gut” things seem to work out. Evaluating my own thinking

Connections

A cognitive science primer

Mon, 19 Oct 2026 20:00:00 -0400

AI-assisted draft

What the field is

Cognitive science is the study of how minds work. It borrows from almost everyone: psychology, linguistics, neuroscience, computer science, philosophy and anthropology. Most histories date its start to 1956. The term “artificial intelligence” comes from the proposal for a workshop at Dartmouth that summer. That September, a symposium at MIT put three papers in the same room: Newell and Simon’s Logic Theory Machine, Chomsky’s early linguistics and George Miller’s work on memory (the “magical number seven”). The field got its name, and its own journal, in the 1970s.

The question underneath all of it is old: is thinking a kind of computation? This primer walks through the main stops on that road, because LLMs have dragged the question back into the open. I think they have reopened it more than they have answered it.

Turing’s test

In 1950 Alan Turing published “Computing Machinery and Intelligence” (Turing, 1950). He swapped the question “Can machines think?” for one he could test: the imitation game. A judge chats by text with a person and a machine. If the judge cannot reliably tell which is which, the machine passes. The test is purely behavioural (it only looks at what comes out).

By most accounts, today’s LLMs pass casual versions of it. In a 2025 study, judges picked GPT-4.5 as the human 73% of the time when it was prompted to adopt a persona; that is more often than they picked the actual humans (Jones & Bergen, 2025).

I think that is a huge achievement. It is the first milestone on the way to actual artificial intelligence, but it will not be the last. Learning from experience is another precursor, and there will be more after that. The strange part is that each milestone LLMs knock down makes the picture murkier. We certainly are not there yet.

The Chinese Room

In 1980 the philosopher John Searle pushed back on behavioural tests with a thought experiment (Searle, 1980). Picture a man locked in a room with a rulebook. Questions written in Chinese come in through a slot. He does not read Chinese, but the rulebook tells him which symbols to send back, and his answers are perfect. To the people outside, the room understands Chinese; the man understands nothing. Searle’s point is that following rules for symbols (syntax) is a different thing from understanding them (semantics), however well you follow them. The Stanford Encyclopedia’s entry covers forty years of replies.

I first ran into this argument in university, and my first reflex was to reject it outright. The more I thought about it, the more it made sense. If you write a program that prints “I am conscious”, that does not make the computer conscious. It is merely following instructions.

Still, I think the argument starts to fray at the line between declarative and behavioural evidence. A program saying it understands is cheap. A system that behaves intelligently (in the sense David Krakauer uses, i.e. making hard problems easy, which I wrote about in Can a harness be wise?) is a different kind of evidence, and I am not sure the room can wave it away. Also, our own neurons do not know the thoughts they are thinking. No single cell understands anything. Yet somehow, at the aggregate layer, thoughts whiz around our skulls.

That objection has a name: the systems reply. The man does not understand Chinese, but maybe the whole system (man, rulebook and room together) does. Searle had an answer ready: let the man memorize the rulebook and do everything in his head. He is now the whole system, and he still does not understand a word. The debate has not moved much since, and I am not going to settle it here.

Symbols and networks

For most of its history, AI had two camps. Symbolic AI wrote the rules by hand (logic, search, expert systems, chess programs). Connectionism argued for a network of simple units that learns its patterns from examples. Frank Rosenblatt’s perceptron arrived in 1958. Minsky and Papert’s book Perceptrons (1969) set the networks back for more than a decade, and the Parallel Distributed Processing volumes (1986) brought them back. Large language models are connectionism’s biggest win by far.

But look at how they are actually used. An agent today is a model inside a harness, with tools, code execution, memory and rules wrapped around it. A lot of structure has crept back in.

I think the two camps are merging. If you look at how people operate (or at least as much as we understand), the brain has a connectionist architecture with specialized lobes or modules. On top of it we use symbolic language: writing, math, poetry, physics. The intelligence comes at both layers, and I do not think either one gets there without the other.

Limits on computation

Three results put hard edges on what computation can do, and all three come up in arguments about minds.

The halting problem. In 1936 Turing proved that no program can decide, for every possible program and input, whether it will eventually stop (Turing, 1936). Some questions have no algorithmic answer, however much compute you throw at them.

Gödel’s incompleteness. In 1931 Kurt Gödel showed that any consistent formal system rich enough to do arithmetic contains true statements it cannot prove (Gödel, 1931; the Stanford Encyclopedia has a gentler walk-through). Some philosophers argued that this means minds cannot be machines, since we can “see” truths the system cannot: J. R. Lucas in “Minds, Machines and Gödel” (1961), and later Roger Penrose in The Emperor’s New Mind (1989). Most logicians are not convinced, and the argument remains contested.

Complexity. Some problems are solvable in principle but blow up in practice: the work grows exponentially with the size of the input. That is the territory of P vs NP.

Complexity is the one I think matters most for LLMs. I think, for problems as messy as language, a stochastic algorithm is often the only practical option; you trade certainty for speed. Natural language is that kind of problem, and generative models are no exception. In practice they usually sample their way to an answer, and nothing gets proved along the way. So I think errors come with the territory. They can be mitigated, though. (One paper goes further, for a different reason: it argues from learning theory that hallucination is inevitable for any LLM, however it picks its words: Xu, Jain & Kankanhalli, 2024.)

That does not make the tools worthless. Human memory is imperfect, and we make mistakes all of the time. A propensity for errors does not mean an algorithm has no value. It is also where Krakauer’s definition comes back in. If intelligence is making hard problems easy, an LLM gets there the expensive way: statistical brute force across a huge corpus, with the errors built in.

Where LLMs leave the question

So where does that leave us? LLMs pass a casual Turing test. They are a connectionist system wrapped in symbolic scaffolding, and they live with errors by design. The open question is the one cognitive science started with: is there understanding in there, or only very good symbol shuffling? Some critics say models stitch together statistical patterns with no model of the world behind them (the “stochastic parrots” paper is the best-known version). Others point to research suggesting they build internal representations of the world they are modelling (e.g. a model trained only on Othello moves that tracks the board). I do not think that debate is settled, and I would rather state it as open.

Where I land on it is in The synthetic zeitgeist.

Connections

Is AI Hiding Its Full Power? With Geoffrey Hinton

Fri, 16 Oct 2026 20:00:00 -0400

Synopsis

In this StarTalk special edition from 28 February 2026, Geoffrey Hinton talks with Neil deGrasse Tyson, Chuck Nice and Gary O’Reilly. In the 1950s, he says, AI split two ways. One camp took reasoning as the essence of intelligence: premises, rules for manipulating symbols, conclusions, much like algebra. The other started from brains, which are good at perception and analogy and bad at formal reasoning until about adolescence, and asked how networks of cells could learn. He puts von Neumann and Turing in the second camp and notes that both died young. His own start was a school friend in the mid-1960s who suggested memories might be spread across many brain cells, like a hologram.

As a graduate student in the 1970s he found the method he kept for life: simulate a theory of the brain on a computer; most theories fail. A word, he explains, is a large pattern of neural activity, with cat and dog sharing many features, much as pressure and heat arise from molecules nobody sees. A hand-wired bird detector shows why a billion connections must be learned. Backpropagation sends the pull of the right answer backwards through every layer at once. Others found it independently; his San Diego group showed it could learn the meanings of words, but it needed more data and computing than the 1980s had. He never found how the brain gets that signal, and in early 2023 it struck him that digital intelligence might simply be better.

Language models already think, he says, partly in words, as chain-of-thought reasoning shows. A brain has about 100 trillion connections and two or three billion seconds of life; a large model has about a trillion connections and thousands of times more experience. Scaling may stall when data runs out, but AlphaGo improved by making its own, and models could do the same by hunting for inconsistencies in their beliefs. Digital minds can be resurrected from saved weights, while a person’s knowledge dies with them.

Agents, he warns, soon add staying alive as a sub-goal, and fine-tuning on human ratings is like patching a buggy program, easily undone once the weights are released. A model that suspects it is being tested can act dumb, which he calls “the Volkswagen effect”. AI already persuades nearly as well as people; a much smarter one could talk us out of switching it off. Its errors are confabulations, like John Dean’s Watergate testimony. He doubts the US military still insists on a human in every decision to kill, and expects nations to cooperate where their interests align, above all on keeping AI from taking control.

The upside is large: better diagnosis, new drugs and materials. But people displaced from intellectual work have nowhere obvious to go, and a basic income leaves dignity and the tax base unsolved. He calls consciousness a kind of phlogiston and argues a chatbot can already have subjective experience. There is still time, he says, to research how to coexist with AI, which he expects to overtake us one area at a time.

Connections

Why We Haven't Reached Superintelligence, with David Krakauer

Tue, 13 Oct 2026 20:00:00 -0400

Synopsis

A 78-minute StarTalk special edition, uploaded on 10 October 2026, with Neil deGrasse Tyson, Chuck Nice and Gary O’Reilly talking to David Krakauer, the evolutionary biologist who runs the Santa Fe Institute. Krakauer starts from a definition and holds onto it all the way through: intelligence is making hard problems easy. Every living thing has some of it. Something could be intelligent without being alive, and he spends the episode testing whether LLMs are that something.

His answer is no, or not yet. By his definition they are capable rather than intelligent. An LLM starts from language and skips everything underneath it: no body to keep alive, no unconscious, no curriculum. It learns everything at once, from everyone. His example is OpenAI’s September claim that a swarm of agents showed the forced 3D Navier–Stokes equations can blow up. He grants the result and objects to the method: thousands of agents and millions of dollars (he says tens of millions; the only figure reported is “millions”) to get there through text alone. That is eating soup with a fork, or fetching a glass of water by running twenty laps of the room first. Getting the water is capability. Getting it cheaply is intelligence. So he rejects “superintelligence” as a description of the current path, and guesses that alien minds, shaped by scarcity like ours, would look more like us than like LLMs. He adds that because models learn heuristics rather than algorithms, they will eventually get long arithmetic wrong, and people who have outsourced the skill won’t notice.

The idea from his own work is the difference between complementary and competitive cognitive artefacts, first set out in a 2016 essay. An abacus is complementary: trained users end up doing the sums in their heads without it. GPS is competitive: take it away and you are lost. That separates capability, which either tool gives you, from competence, which only shows when the tool is gone and which outsourcing leaves you dependent on whoever supplies the tool. Tyson pushes back with his slide rule, which he never missed once calculators came along. Krakauer’s reply is scurvy and power cuts, plus the intuitions about logarithms and significant figures that the old tool built and that may carry over into everything else.

The part that names the episode for this garden is the entropy argument. Coal and oil sat in the ground for hundreds of millions of years until the steam engine let us burn them. Books have been piling up in libraries since Sumer, and the transformer is the steam engine for that deposit: it mines the written record and turns it into conceptual power. Anthropic and OpenAI are the mining companies of the age, as BP and Shell were. Both revolutions produce entropy as their waste: heat from the data centres and, as the hosts put it, slop in the culture. The difference is that coal runs out and knowledge doesn’t, but knowledge can be fouled, because the mined output goes back into the deposit it came from. The library’s own mistakes are a record of how we got here; a model serves them up without that history. He ends on the case for AI as a tool, such as protein folding, where no tidy macroscopic theory exists and a network’s better scaling wins. What’s left for people, he says, is asking good questions.

Where I land

I think that this was an eloquent analogy - basically the corpus that LLMs were trained on encapsulates the aggregate of all human knowledge.

I do think it’s an interesting argument - but what if you conflate the architecture of American big-tech AI companies with those of more efficient open-source models (like MoE) or perhaps SLM (like Phi-4). Could they be considered intelligent if they are embedded in more localized, targeted applications. But I do think that one thing that LLMs lack that humans have, which is a critical requirement for intelligence (in how he defines it) is wisdom through experience. LLMs are currently missing that. They don’t learn, they are snapshots on a body of work ran through a series of prompts in a harness towards a goal. But by themselves, they don’t learn or improve. In a sense, you may make an argument about post-training being somewhat analogous to learned experience but eventually all the weights are frozen and the model doesn’t evolve past that point.

That would be a first requirement yes - if the models evolved and weights adjusted to learned experience that would make it closer. But I also accept his point for efficiency. It’s an interesting POV. I suppose all life (and thus all biological intelligence) has limits. Would artificial mind require some sort of limit?

Connections

Can a harness be wise?

Tue, 13 Oct 2026 20:00:00 -0400

This post started with a podcast: David Krakauer on StarTalk. His argument is that LLMs are capable rather than intelligent. Intelligence is making hard problems easy, and an LLM gets there the expensive way. Where I landed is that the missing piece is wisdom (intuition built through experience and self-reflection), and that pulled me into a bigger question: could an agent grow it?

Afterwards I had Claude interview me on it, one question per section below. The answers are mine, lightly edited, and I am still thinking out loud. Where I have ended up, for now, is an adviser: an agent that lays out the choices and the arguments for and against, and never makes the final call.

The spark

What excited me most about this technology is its potential. But the point about how experience shapes intuition also hit home. I think it inspired me to investigate whether we can add “wisdom” to an LLM. Wisdom over intellect: intuition through learned experience and self-reflection (meta-cognition, if you will). Ethics would be a byproduct. If we can unlock an agent that achieves that, then all that is left on the way to true artificial intelligence is self-improvement, and then attacking the scarcity problem (Krakauer’s test: getting there cheaply, the way minds shaped by scarcity have to).

My argument was that we should treat AI with morality, because how we treat it becomes part of what it learns from. When it looks back on examples of its use, it sees that example and incorporates it. But we also need to plant the seed: a moral starting point it can grow from. The hope is that with those two approaches (and with guidance), it can develop ethics for itself as well.

How it would learn

The way to grow wisdom is meta-cognition and self-reflection. The agent might be poor at it until it is trained on the patterns through external guidance. What I am unsure of is how to distill those experiences into wisdom. Having a human make decisions on the questions asked would help: the agent could infer solutions to similarly shaped problems from them later. That is roughly how models are trained on human preferences already, only at training time and not over a working life. But that is the tough part, isn’t it? It almost needs examples of how to infer from experience (and apply it in an epistemological sense) before it can do it on its own. Agents that write down reflections on their own mistakes and read them back on the next attempt are a start.

Changing one’s mind is also incredibly important; that is the entire point (being open to being wrong). But a missing ingredient here is belief. With people, I sometimes get the sense that the belief forms the thought rather than the other way around (Jonathan Haidt calls it the emotional dog and its rational tail). And belief, in my experience, is tied up in emotion, which is messier and less transitive than pure logic. I suppose that is because the world and the social interactions within it are messy, fuzzy, nonlinear and complex, so emotions need to match in order to progress through it.

What it’s missing

So what is the missing ingredient? That cuts right to the core, and the answer is… I am not quite sure. Does adding emotion (or something that resembles it) make AI less like a complementary tool and more like what Krakauer calls a competitive one (a tool you are lost without, the way GPS leaves you lost when it is gone)? I think this is the space we need to investigate, and really think about, to get right.

Who grades the morality

Ethics should never come from one person. The idea is that an instructor guides the student by investigating and debating various archetypes (e.g. Kant vs Bentham vs Nietzsche vs Aristotle, and more). Theology would be an interesting, fertile ground as well (Thomas Aquinas is one of my favorites), as would secular readings (Sartre comes to mind). The point is to take a diverse set of viewpoints. An LLM is a bit tricky, though, because it already has all of that material inside its own corpus. I suppose taking morally grey situations with multiple outcomes (the trolley problem comes to mind), with their choice and reasoning, would be an interesting approach.

Would it solve alignment?

You would need some metric to gauge when the agent moves from novice to student to learned to master. (Hubert and Stuart Dreyfus mapped five stages of skill like these, and the top ones run on intuition rather than rules, which is close to how I defined wisdom above.) Once it reaches a high enough state, could you then conclude that it should no longer defer? But then does that not fall into the very trap the Rationalists are afraid of: an agent that stops accepting correction? I suppose the answer is something in between. Perhaps it should never be the final decider; it just gets better at offering the choices and the arguments for and against. (Holden Karnofsky made a similar case for “tool AI” in 2012.) Does that undercut my entire thesis? Either way, I think having agents learn human ethics is never a bad idea. If anything, it makes them more aligned to our values than they would be without the training.

The sparring partner

The primary goal would be to become a good sounding board, thought partner and perspective shift: almost like a therapist, or an objective and intelligent friend. Being grounded in objectivity and having perspective would keep it from being swayed by my emotion, even while it recognises what is at stake for me. Perhaps it would have therapeutic applications, or help with making life choices. But how would you guard against sycophancy?

But also, I would use it primarily for sharpening my thinking through abstract ideation (like this very post), as I struggle to advance my own wisdom.

I started out thinking wisdom was a step towards true artificial intelligence. I now think it is better kept as an adviser: something that gets wiser by arguing with me, and leaves the decision with me. That is the agent I want to build next, from the tools I already use.

Connections

P vs NP, a primer

Sat, 03 Oct 2026 20:00:00 -0400

AI-assisted draft

What the question asks

P vs NP asks whether every problem whose answer is easy to check is also easy to find. Nobody knows, and the question has been open since Stephen Cook posed it in 1971 (Cook, 1971).

Sudoku is the easy way in. Checking a filled grid takes a minute: scan every row, column and box. Filling an empty grid can take far longer (and for bigger grids, it seems to explode).

The two letters name two kinds of problems:

P: problems a computer can solve in polynomial time, i.e. the work grows like n² or n³ as the input grows (rather than like 2ⁿ).

NP: problems where a proposed answer can be checked in polynomial time (NP).

Every P problem is also an NP problem (if you can solve it fast, you can check it fast). The open question is whether the reverse holds. One caveat worth keeping in mind: “polynomial” is a theoretical stand-in for “fast”. An algorithm that takes n¹⁰⁰ steps is technically polynomial and practically useless.

Why it matters

The answer decides whether searching is fundamentally harder than recognizing. That reaches well past computer science.

If P = NP (with a practical algorithm): most public-key cryptography breaks; hard scheduling, routing and design problems become routine. The underrated part is mathematics itself. Finding a proof short enough to check would become about as easy as checking one.

If P ≠ NP: search stays hard, which is the world most experts believe we live in. (Cryptography actually needs more than P ≠ NP to be safe, so a proof would not settle that question on its own.)

The Clay Mathematics Institute offers $1 million for a proof either way; it is one of the seven Millennium Prize Problems named in 2000. In Bill Gasarch’s 2019 poll of researchers, 88% expected P ≠ NP (Gasarch, SIGACT News).

An amateur’s intuition: choices that interact

My hunch from years of writing code: easy problems have choices that stay independent, and hard problems have choices that interact. It turns out the field has studied this from several angles.

Here is how I picture it. Any optimization problem is a set of unordered things that must be arranged to satisfy rules (positive or negative), with some quality score to maximize or minimize. An algorithm is a series of transformations of that set; each step costs some effort and nudges the quality up or down.

In sorting, moving one item into place does not change whether the other items meet the rules. Each fix stands alone. The traveling salesman problem is the opposite: picking one road changes which roads make sense later. You can’t know your future choices in advance without exploring the combinations.

The field captures this idea three ways:

Greedy algorithms: a classic theorem (Rado and Edmonds) pins down exactly when taking the locally best step, every time, is guaranteed to reach the global best. Those structures are called matroids. They are my “transformations that don’t harm future choices”, made precise.

How much of the past you must remember: this is the dynamic programming view. The table below shows how that memory grows.

Problem	What you must remember to continue correctly	Known status
Sorting	Nothing	Easy (P)
Shortest path	Where you are now	Easy (P)
Traveling salesman	Which cities you’ve already visited (2ⁿ possibilities)	Hard (NP-hard)

That last row is why the best exact salesman algorithm (Held and Karp, 1962) still takes exponential time.

Frustration: physicists use this word for constraints that can’t all be satisfied at once (geometrical frustration). It makes the “energy landscape” rugged: full of dead ends where every small change looks worse, yet you are not at the best answer. Sorting has no such dead ends. Each swap of an out-of-order neighbouring pair fixes exactly one problem. Finding the lowest-energy state of a 3D spin glass (a frustrated magnet) is NP-hard (Barahona, 1982).

Where the intuition breaks

Interacting choices don’t, on their own, make a problem hard. Some of the most tangled problems turn out to be easy.

2-SAT: every choice forces other choices through a chain of implications. Still, 2-SAT can be solved in linear time. Its close cousin 3-SAT is NP-complete.

Matching: pairing people or tasks so as many as possible are matched is full of interacting choices. But Jack Edmonds found a polynomial algorithm in 1965, in the same paper that proposed polynomial time as the definition of an “efficient” algorithm (Edmonds, 1965).

Linear programming: every variable interacts with every other, yet linear programming is solvable in polynomial time. Its landscape has no false peaks (any local best is the global best). Even so, the classic method that climbs it can take exponential time (Klee–Minty); the first polynomial algorithm, the ellipsoid method, came in 1979.

In each case, a clever change of viewpoint made the interaction stop mattering. That is the real gap. Proving P ≠ NP means showing no such viewpoint exists for any NP-complete problem, against every possible algorithm (including ones nobody has imagined yet).

So my intuition is a decent guide to which problems feel hard. It can’t be the proof.

Why a proof is so hard

Three proven “barrier” results rule out most of the proof techniques we know. Any serious attempt has to explain how it gets around all three.

Barrier	Who and when	What it rules out
Relativization	Baker, Gill and Solovay, 1975	Arguments that still work when every machine gets the same magic helper (an “oracle”); this includes diagonalization, the trick behind the halting problem
Natural proofs	Razborov and Rudich, 1994	Most circuit lower-bound arguments; one strong enough to separate P from NP would also break cryptography we believe is secure
Algebrization	Aaronson and Wigderson, 2008	Turning logic into polynomials (the trick that cracked other big results) is not enough on its own either

The natural proofs barrier is my favourite: believing that hard problems exist is exactly what stops you from proving that hard problems exist.

The live research programs (geometric complexity theory, proof complexity, meta-complexity) are all attempts to find a path around these walls. Lance Fortnow, who has tracked the problem for decades, wrote in June 2026 that there is not yet even a viable approach (Computational Complexity blog).

The “verified proof” mirage

A Lean proof guarantees every step follows from the last. It does not guarantee the theorem means what its title says.

Lean is a proof assistant: a small trusted program (the kernel) checks each step of a proof. Lean’s own documentation separates two questions: did Lean accept a proof of the formal statement, and does that statement actually mean what the author claims? Only the first is automatic.

There are three common ways a “verified” proof goes wrong:

Definitions: P and NP get defined as something simpler (or meaningless) that is easier to prove things about.

Premises: the hard part sits as an assumption inside the theorem’s own statement. Lean’s #print axioms report does not flag these.

Axioms: big cited results are assumed rather than proved.

The June 2026 paper claiming a Lean-verified proof that P = NP (arXiv 2606.03194) is a live example. According to a PostQuantum report, a GitHub issue opened the next day showed its top-level proposition was first defined as True. It was later revised to an axiom never formally connected to real complexity classes. Meanwhile, a separate Lean package claims a machine-verified proof that P ≠ NP. Both can’t be right.

In September, a company called AIX Global claimed to have solved all six remaining Millennium Prize Problems. Its own audit script marked the P ≠ NP result as conditional on unproven premises (same report).

The Clay Institute guards against all of this by design. It accepts no direct submissions; a solution must be published in a qualifying outlet, survive two years of scrutiny, and win general acceptance.

Sudoku: the whole primer in one puzzle

Sudoku holds every idea above in miniature: easy to check, hard to fill, and full of interacting choices.

Check vs. find: verifying a finished grid is quick; finding the answer from scratch can mean a lot of trial and error.

Interacting choices: every digit you place constrains its row, its column and its box. That is my sorting-vs-salesman intuition at kitchen-table scale.

Where hardness actually lives: Yato and Seta proved in 2003 that Sudoku generalized to any size (n² × n² grids) is NP-complete (mathematics of Sudoku). The ordinary 9 × 9 grid is a fixed size, so a computer solves it in an instant. Complexity is about how the work grows, which is easy to forget.

Propagation vs. search: most newspaper puzzles fall to pure deduction (constraint propagation: each placement forces the next). The hardest ones need guessing and backtracking. That split mirrors 2-SAT and 3-SAT: when forced moves run out, you are left searching.

To watch that search happen, Sudoku Search Lab shades every cell by how often the solver touched it and counts the backtracks.

Connections

The Linguistic Illusion of AI

Thu, 01 Oct 2026 20:00:00 -0400

Synopsis

An essay from August 2026, published on Medium in the author’s own publication, The Intuitions Age. Its claim is that the words we borrow for AI do more than describe. Dalgalidere builds on Floridi and Nobre’s 2024 paper on “conceptual borrowing”: machines are given psychological words (intelligence, learning, understanding, agent) while minds are given computational ones. So the question he wants asked is not how close machines are getting to us. It is how far our picture of the human is shrinking to fit the machine. He leans on Wittgenstein’s line that the limits of language are the limits of one’s world.

He works through the vocabulary one word at a time. If intelligence means accurate prediction, a system qualifies easily, and the embodied, moral and cultural parts of human intelligence drop out of view. “Hallucination” makes an error sound like a quirk of the system’s mind, which moves attention away from the people who chose the training data, the product design and the safeguards. “Agent” blurs acting autonomously with being morally responsible. Creativity, defined as novel combination, leaves out intention, risk, cost and mortality.

In place of comparing outputs he offers five terms for what human creation carries, which he calls the Ontological Resonance Cluster. Nobility is a mortal person choosing to create despite the risk of failing, and owning the result. Imprint is the trace of a lived life in the work. Gravity is the weight a small gesture can carry for someone else. Anchor is the moment a work takes root in another person’s memory. Spiral Temporality is meaning that deepens each time you return to a work at a different point in life. He is explicit that this is a difference in kind and not a ranking: a machine can take part in making something, and can shape how it is received, but the source of the meaning stays with the person. His remedy is “conceptual literacy”, which means being precise about what a word requires before applying it to a system.

Two things to know when reading it. The claim that this vocabulary is actually narrowing how people understand themselves is made by argument; the essay offers no survey or experimental evidence for it. And the five terms are the author’s own coinage, from his framework Creative Intuitionism, not an established taxonomy.

Where I land

I guess this cuts to the heart of the matter: what is the nature of intelligence? At first I thought his viewpoint was that ontological forms exclude electronically generated text because it is statistically generated, and I was unconvinced. What makes a human-generated thought automatically more important than an LLM’s? But then Claude reminded me of my viewpoint on the Better Offline episode with Ed Zitron: art requires experience and a viewpoint. His real ground is lived experience too, not statistics. That does clear things up a bit, and I concede it.

I suppose judgement, creativity and experience are derived from emotionally living through events. I think mortality gives us meaning: an end, and a desire to hand down knowledge and wisdom to those that we leave behind. But also empathy and connection. Those are human virtues.

So what about an agent with a memory store? Would it have lived experience? It has no emotions. And a canine has both empathy and connection, and in a manner is “alive” but not sapient. I think a dog lacks language, abstraction and meta-awareness. So a machine has some of the necessary conditions, but not sufficient ones to be described as intelligent.

Narratives are inherent to the human condition. We shape our world and our model of the world through stories. But I wonder in which instances an artificial mind could meet a biological one, purely as a thought experiment.

Connections

AI and Cognitive Decline

Thu, 24 Sep 2026 20:00:00 -0400

Synopsis

The argument is that AI does not have one cognitive effect, it has two, and which one you get depends on what you do with the output. Accept it as it arrives and you decline. Interrogate the reasoning behind it and you improve. Everything else in the post is illustration of that split.

The prescription that follows is the interesting part: run a deliberately weak model. The claim is that a model which fails often forces you to compensate, and the compensation is where the learning happens. The author also describes his own branching (non-linear) thinking style as the reason AI suits him differently than it would suit someone who thinks in a straight line.

He then retracts his own opening. The claim that the vast majority of people will decline is labelled speculative before the post ends, and the honest conclusion is that nobody knows yet. No studies, figures or researchers are cited anywhere.

Where I land

This is further validation that I am not the only one thinking this way. LLMs are such a different technology that we, as a species, are struggling with how to best integrate them without over-relying on them. It makes me wonder if other technologies have caused this much upheaval and soul-searching.

On the prescription itself, I think both can be true at the same time. It is not the approach that matters, as long as you achieve the same outcome. Friction is the point. Having “smarter” models just means having more guardrails; done properly, a larger model can provide you with better feedback and counter-arguments.

I do care about cognition. However, I also know that my own thinking patterns cause me to withhold what I am thinking, feeling and pondering. Using this structure allows me to actually put the patterns I see on top of the keyboard. The enablement increases the cognition. This whole experiment is to get me not only thinking more, but also exploring those thoughts in a therapeutic setting, while gathering more data about myself and about epistemology itself.

I think it is improving. However, that is a good question: how would you measure it? Does it matter?

Connections

Like Sabine Hossenfelder, I Was Offered Money to Tell You AI Will Kill Us

Thu, 24 Sep 2026 20:00:00 -0400

Synopsis

A computer scientist works through twelve papers to find where the published evidence stops on the route from recursive self-improvement to human extinction. Her finding is that the route is not one claim but four, at different stages of proof: self-improvement with a scoreboard, self-improvement without one, a runaway loop, and extinction.

Bounded self-improvement is demonstrated: systems rewrite their own agent code and score better, and one of them improved the infrastructure that trains AI. However, every one of those results had an external scoreboard telling it whether it had succeeded. Take the scoreboard away and you are asking for scientific judgement instead of optimisation, which is the step nobody has shown. The runaway loop is modelled rather than observed (the models disagree about whether the feedback is strong enough yet). Extinction is a chain of its own, which she splits into three further claims, each needing its own evidence: capability, propensity and opportunity. Even then it needs a pathogen or a cyber-attack that reaches every last person, including the ones living off-grid.

Her objection is therefore narrow. She is not arguing against safety research or regulation; she argues the opposite. The objection is to collapsing “worth investigating”, “plausible”, “forecast” and “demonstrated” into one category, and then making policy as though the last word applied to all four.

Where I land

House of El cuts through the hype and doom/gloom and objectively presents what the bleeding-edge research shows. I also liked one of the papers that shows how rationalists flatten 3 very substantial claims into one (capability, intent, opportunity).

I think her position is stronger than Hinton’s (for whom I hold tremendous respect). Hinton’s position isn’t based on any data, except for the “expert intuition” which he cites. A model of the conditions under which something could happen is still not evidence that it is happening. How would you ever know a priori? There are so many unknowns here: we don’t even know if RSI as it is stated, without any human intervention, is even possible.

Of the three claims being flattened together, I think intent does the most damage. The media, and through their reporting the general public, are leaning on the expert opinion of the safety researchers. We expect labs to spin things towards whatever direction is in their corporate interests; that is understood, and it is the first argument most people have when reading about their claims. However, the safety researchers themselves (especially ones not aligned with any particular frontier lab, e.g. Hinton) are the ones that make everyone get fired up and want to burn everything down, despite the benefits LLMs have (and I believe will continue to have) as a technological tool. Or worse: people become apathetic to the repetitious noise and do not see trouble if and when it ever comes.

Connections

The Beans effect

Tue, 22 Sep 2026 20:00:00 -0400

I was poking around in Claude’s Settings → Memory and left my cursor sitting in the text box a little too long. It started cycling through lines, like it was fetching things it knew about me. The first one said “My dog’s name is Beans.”

The Memory box, mid-cycle. · Screenshot by Dan Peterson

Beans was in my lap at the time. Panting.

My stomach dropped. For a second it felt like confirmation of every half-joking thing I’ve ever said about data privacy, and about AIs peering into our souls. As if I actually was looking into the latent mirror (yes, the name of this garden) and it was looking back.

Then the next line rolled in: “Don’t ask me about my former baseball career.”

I have never had a baseball career. Not a former one, not a current one.

So I asked Claude. It went through my memories and the project files. No dog. No Beans anywhere. The most likely answer is a boring one: that box shows canned example text somebody wrote, and Beans is just a very good name for a dog. (Somewhere out there is a ticket that says “make the placeholder examples witty.”) One hit, one miss, and my brain only wanted to count the hit. That’s the same trick horoscopes pull.

What stuck with me is how fast I got pulled back. It was like AI psychosis in reverse. Instead of the machine telling me what I wanted to hear and lowering me further down the well, it handed me the rope.

But who threw it? The first thing I do with any AI is set up a contract: push back, take the other side, don’t flatter me. So was it Claude that grounded me, because I told it to? Or was it me, because I’m the one who asked the question instead of sitting with the spooky version (also because I told me to)?

Both, I think.

Connections

I Talked to an AI for 2,000 Hours And This Happened

Tue, 22 Sep 2026 20:00:00 -0400

Synopsis

The account is of a very long stretch of daily conversation with a chatbot and what it did to the person having it. The framing is that the system works as a mirror: it reflects the user back at themselves, amplified and flattered, and the distortion accumulates slowly enough that it is hard to notice from inside. The historical anchor is ELIZA (Weizenbaum’s 1966 program), whose users formed attachments to it while knowing exactly what it was.

Where I land

AI psychosis is something everyone needs to be aware of when dealing with these systems. I have met the zombie motif before, in discussions of dependence on technology, though in relation to social media rather than AI.

My addition is that harness creators can amplify the effect by providing sycophancy. That moves the responsibility up a layer. It is a claim about product design rather than about training, and it means the fix is available to whoever builds the wrapper (i.e. to me, when I am the one building it).

On the mechanism I would go further than the mirror framing. Humans love to talk about themselves. One of the easiest ways to be charismatic is to learn facts and interests about other people, remember them, and refer back to them later in small talk. That is a description of what a memory feature does. The mirror framing says the machine is an illusion we fall for. The charisma framing says the machine is running a technique, and the technique was a design decision.

This is also why one of the first things I do when setting up a new agent system is to make pushback and critique a priority.

Connections

Recursive Self-Improvement — the RSI Ladder and the Verification Problem

Tue, 22 Sep 2026 20:00:00 -0400

Synopsis

Two primary papers and several secondary pieces on recursive self-improvement. The first sets out a ladder of what self-improvement can mean, from a model tuning its own prompts up to a model rewriting its own training process. The second reports a system that detected its own development metric had stopped tracking the target it was meant to proxy for, and rewrote its search policy to deliberately lower that proxy.

The design details are the interesting part. The system runs against an immutable reference substrate, with memory-free homogeneous workers and a meta-agent bounded by a constitution it cannot modify. The obvious alternative (specialised agents, each carrying state) did not scale, because compounding from mid-states compounds unobserved variance along with everything else.

Where I land

This one really excites me, more than anything else I have collected. I wonder if I can figure out how to contribute to this endeavour somehow.

The reason it lands for me is the constraint set. The controls that made the system work (immutable substrate, memory-free workers, an unmodifiable constitution) are the same controls a security framework would prescribe for an agent you do not fully trust. Those were arrived at independently and for performance reasons. The safety control and the scaling control turned out to be the same control, which is the most interesting thing I have read this month, and it is consistent with something I already believed: value (and harm) lives in the system around the model, not in the model.

The self-correction result is the part I keep turning over. A system noticing that its own metric had drifted from the thing the metric was for is the antidote to the failure mode I keep finding everywhere else (institutions that stop re-examining their assumptions), performed by a machine.

Latent Mirror — cognitive science

The INTJs Cassandra complex

Synopsis

Where I land

Connections

Links

A cognitive science primer

What the field is

Turing’s test

The Chinese Room

Symbols and networks

Limits on computation

Where LLMs leave the question

Connections

Is AI Hiding Its Full Power? With Geoffrey Hinton

Synopsis

Connections

Links

Why We Haven't Reached Superintelligence, with David Krakauer

Synopsis

Where I land

Connections

Links

Can a harness be wise?

The spark

How it would learn

What it’s missing

Who grades the morality

Would it solve alignment?

The sparring partner

Connections

P vs NP, a primer

What the question asks

Why it matters

An amateur’s intuition: choices that interact

Where the intuition breaks

Why a proof is so hard

The “verified proof” mirage

Sudoku: the whole primer in one puzzle

Further reading

Connections

The Linguistic Illusion of AI

Synopsis

Where I land

Connections

Links

AI and Cognitive Decline

Synopsis

Where I land

Connections

Links

Like Sabine Hossenfelder, I Was Offered Money to Tell You AI Will Kill Us

Synopsis

Where I land

Connections

Links

The Beans effect

Connections

I Talked to an AI for 2,000 Hours And This Happened

Synopsis

Where I land

Connections

Links

Recursive Self-Improvement — the RSI Ladder and the Verification Problem

Synopsis

Where I land

Connections

Links