I read every major CS paper of the last 100 years...

usefulFireshipvideo2026-06-17read planted ai llmshistory

Synopsis — AI-drafted from Dan's notes

Fireship tells the history of computer science as a chain of ten papers that ends in the chatbot. Each one hands the next a missing piece: a definition of computing, a measure of information, a learning rule, a way to train deep networks, then data, compute and an architecture that scales. A sponsor segment sits in the middle.

  1. On Computable Numbers, Alan Turing, 1936. Answers Hilbert’s decision problem with a no, and to prove it defines the algorithm and the Turing machine; the halting problem shows some questions no program can settle. What it implies: computing has a formal definition, and hard limits, before any computer exists.
  2. A Mathematical Theory of Communication, Claude Shannon, 1948. Strips meaning out of messages and measures information as surprise, in bits, borrowing entropy to do it. What it implies: text is something you can predict, and prediction can be scored. The next-letter guessing game the video credits as the ancestor of next-token prediction is in his later paper, Prediction and Entropy of Printed English (1951). The video also repeats the story that Anthropic named Claude after him, which is widely reported but which Anthropic has never confirmed.
  3. The Perceptron, Frank Rosenblatt, 1958. A machine that learns by nudging its weights whenever it is wrong, to wild press. What it implies: behaviour can be learned from examples instead of programmed.
  4. Perceptrons, Marvin Minsky and Seymour Papert, 1969 (a book; the link is MIT Press’s reissue). Proves a single layer can’t learn exclusive-or. Stacked layers could, but nobody knew how to train them, and funding dried up. What it implies: the idea needed depth, and depth needed a training method.
  5. Time, Clocks, and the Ordering of Events in a Distributed System, Leslie Lamport, 1978. Orders events by cause rather than by wall-clock time. What it implies: many machines can act as one, which the video carries forward to today’s GPU training runs.
  6. Learning Representations by Back-propagating Errors, David Rumelhart, Geoffrey Hinton and Ronald Williams, 1986. Backpropagation trains hidden layers, which learn their own features. What it implies: the answer to Minsky and Papert; deep networks become trainable.
  7. The Anatomy of a Large-Scale Hypertextual Web Search Engine, Sergey Brin and Lawrence Page, 1998. Ranks pages by their links, read as weighted votes. What it implies: the organised web becomes the feedstock for later models.
  8. ImageNet Classification with Deep Convolutional Neural Networks (AlexNet), Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, 2012. Trained on two gaming GPUs in Krizhevsky’s bedroom, it cut the ImageNet error by about ten points. What it implies: deep learning works, and GPUs are its engine.
  9. Attention Is All You Need, Vaswani and others at Google, 2017. The transformer reads every token against every other at once and scales well. What it implies: one architecture every lab now builds on.
  10. Language Models are Few-Shot Learners (GPT-3), OpenAI, 2020. Bets that capability emerges from sheer scale, at 175 billion parameters, and leads to ChatGPT. What it implies: scale itself became the strategy.

The closing joke is that it all still predicts the next token, as Shannon’s subjects did.

Connections