AI Recently Learned to Think: Why Even Its Creators Can’t Explain The New Superintelligence

For years the put-down was easy: “It’s just autocomplete. A giant language database that predicts the next word.” Then the machines started solving maths problems they had never seen, catching their own mistakes and planning their answers before writing a word. Something changed, and the people who built these systems are still working out exactly what.

This is the story of how artificial intelligence outgrew the lookup-table label, why it now behaves less like a search engine and more like a thinker, and why many of the field’s leaders believe superintelligence – AI that outperforms the best humans at almost every intellectual task – is no longer a distant science-fiction idea but a question of years.

Why “it just predicts the next word” no longer tells the whole story

Early language models really were impressive pattern-matchers and little more. Ask one for a fact and it recalled something close; ask it a genuinely new logic puzzle and it fell apart. If that were still the whole picture, AI would have hit a hard ceiling: good at boilerplate text, hopeless at novel problems.

Today’s systems routinely debug unfamiliar code, work through multi-step analysis and reach correct answers to competition-level maths. Technically they still produce text one token at a time. But “predicting the next word” describes the output, not what happens inside – in the same way that “moving a pen” describes a mathematician’s hand, not their mind.

Compression, not memorisation: why AI is not a language database

A database stores exact records and hands them back on request. A large language model cannot work like that. It is trained on trillions of words, yet its parameters – its entire internal memory – are far too small to hold that text word for word. It can recall some popular passages, but it cannot store the internet.

So it is forced to compress. And the most efficient way to compress billions of pages is to learn the rules that generated them: grammar, cause and effect, how code executes, how arguments hang together, how physical objects behave.

Think of two students facing a physics exam. One memorises 500 worked answers and panics at question 501. The other learns Newton’s laws and can tackle anything. At scale, language models are pushed from the first student towards the second. Researchers call the result an internal world model: a compressed, working representation of how things relate to one another.

How AI learned to slow down and think

Psychologists describe two modes of human thought. System 1 is fast and instinctive – the answer that pops into your head. System 2 is slow and deliberate – the working-out you do on paper. Early chatbots were pure System 1: they blurted out an answer immediately, and on hard problems that answer was often wrong.

The breakthrough was giving models a scratchpad. With chain-of-thought reasoning, a model writes out intermediate steps, tests ideas, checks constraints and backtracks when something doesn’t add up – before it commits to a final answer. Text generation stops being a passive stream and becomes an active workspace.

The most striking evidence came from the Chinese lab DeepSeek. Its R1 research, published in January 2025, trained a model largely by reinforcement learning: the model was rewarded for correct answers to maths and coding problems, with no human-written examples of how to reason. Nobody taught it to reflect. Yet partway through training, the researchers saw what they called an “aha moment” – the model abruptly began stopping mid-answer, using the word “wait” far more often, and re-evaluating its own approach. It had discovered self-correction on its own. The work went on to appear on the cover of Nature in September 2025, widely reported as the first major language model to go through full peer review.

Test-time compute: giving AI time to think

For most of the last decade, making AI smarter meant one thing: build a bigger model with more data and more hardware. Then a second lever appeared. OpenAI’s o1, released in September 2024, popularised the idea of test-time compute – spending extra processing power at the moment you ask the question, not just during training.

A traditional model goes straight from prompt to reply. A reasoning model explores: it tries one line of attack, finds that it fails, tries another, verifies it, and only then answers. Give it more time to think and it generally performs better on hard problems. That simple finding changed the economics of AI. Intelligence is no longer frozen at the end of training; it can be dialled up on demand.

The great mystery: even AI’s creators don’t fully understand it

Here is the part that surprises most people. The engineers who build these systems design the architecture and the training process, but they do not write the rules the model ends up following. The model organises itself. What emerges is billions of numbers whose meaning nobody specified.

We do not understand how our own AI creations work.

In the same essay, Amodei likens building AI to growing a plant or a bacterial colony: you set the conditions, but the exact structure that grows is unpredictable. A whole field, mechanistic interpretability, now exists to look inside – effectively neuroscience for artificial minds. And what it finds keeps catching its own researchers off guard.

In 2025 Anthropic published research that traced the internal “circuits” of one of its Claude models. The team set out to show that the model did not plan ahead when writing rhyming poetry. They found the opposite: before writing a line, the model had already picked the rhyming word it was aiming for, then built the sentence to land on it. When researchers suppressed that planned word inside the model, it chose a different rhyme. They also found signs of a shared conceptual space across languages – a kind of universal “language of thought” beneath English, French or Chinese. Even so, the team was candid that its tools capture only a fraction of what the model is doing.

Other discoveries are just as strange:

• Grokking. In a 2022 study, OpenAI researchers trained small networks on modular arithmetic. The networks memorised their training examples quickly but failed on anything new – then, long after they appeared to have stopped learning, accuracy on unseen problems suddenly leapt from chance to near-perfect. The network had reorganised itself around the underlying rule.
• Emergent abilities. As models grow, some skills seem to switch on rather than improve gradually, such as multi-step arithmetic or decoding simple ciphers. Researchers still debate how sudden these jumps really are – some argue they partly reflect how we measure – but nobody explicitly programmed these abilities in.

Building modern AI is becoming less like writing software and more like cultivating an ecosystem: engineers create the conditions for intelligence to emerge, then study what grows.

Beyond human text: how AI keeps learning when the internet runs out

As models consume most of the high-quality text on the public web, an obvious question arises: what happens when the human writing runs out? Researchers are answering it in three ways.

Self-generated, verified practice. Models create their own problems, attempt them, and pass the answers through objective checkers – code compilers, maths engines, unit tests. Only verified answers are kept. Because a checker, not the model’s own opinion, decides what counts as correct, this sharply reduces the risk of “model collapse”, where AI trained on unfiltered AI output gradually degrades.

Learning from the physical world. Text is a thin slice of reality. Training on video, audio and 3D simulation lets models learn cause and effect, motion and space directly.

Learning how to think, not just what to know. Training increasingly uses high-quality reasoning traces – step-by-step deductions – rather than static facts, teaching models the process of problem-solving itself.

The road to superintelligence: four stages ahead

We are moving from the era of generative AI, which writes and answers, into the era of reasoning AI, which works problems through. The path beyond that is widely expected to run through four overlapping stages:

1. Agentic execution

   AI moves from producing text to taking action: browsing the web, operating software, running code and completing multi-step digital tasks on our behalf.

2. Multimodal world models

   Reasoning grounded in sight, sound and 3D simulation, giving robots and physical systems a working sense of space and motion.

3. Long-horizon autonomy

   Beyond short chat sessions to projects that run for days, with persistent memory and the ability to plan, check progress and adjust.

4. AI-driven scientific discovery

   From analysing what humans already know to proposing and testing genuinely new hypotheses in medicine, materials science and clean energy.

How soon? Opinions differ, but the boldest forecasts now come from the people closest to the work. Anthropic has said it expects AI systems with Nobel-level intellectual capabilities across many fields to emerge around late 2026 or early 2027. OpenAI’s Sam Altman has pointed to around 2028 for early forms of superintelligence. Google DeepMind’s Demis Hassabis is more measured, suggesting that general intelligence may be around five years away but still needs one or two further breakthroughs. Some forecasters have pushed their estimates back as real-world progress proves uneven. What has changed is that serious researchers now argue about the year, not the century.

A thinking partner that could change the world for the better

The early signs are already here. In 2024 the Nobel Prize in Chemistry recognised AlphaFold, an AI system that predicts the 3D structure of proteins – work that once took years per protein and now takes minutes, accelerating drug discovery and disease research worldwide. Reasoning models are already helping engineers write and review code, helping analysts work through complex data and helping researchers explore ideas far faster than before.

Extend that trajectory and the prospect is extraordinary: AI colleagues working alongside us in every field, new medicines discovered in months rather than decades, breakthroughs in clean energy and materials, and expert-level help available to every small business and every person, not just those who can afford it. The same leaders who make the bold predictions are also the loudest voices on the need to get safety and understanding right along the way – and that work, including interpretability research, is a big part of why the outlook is so hopeful.

Try a reasoning AI model on your own computer

You don’t need a data centre to see machine reasoning in action. Open-weight reasoning models, such as the smaller distilled versions of DeepSeek-R1, run on an ordinary modern laptop using free tools like Ollama. Once Ollama is installed, typing ollama run deepseek-r1:8b in a terminal downloads and starts a local reasoning model.

Ask it a tricky question and watch the text between its <think> tags: you can see it break the problem down, question itself and correct its mistakes before it answers. Because it runs entirely on your own machine, nothing you type leaves your computer – a useful option for anyone who cares about privacy. Developers can go further by wrapping a model in a verification loop, feeding test failures back into the prompt until the code passes, and so build their own self-correcting agents today.

From what machines know to how they think

AI is no longer a mirror reflecting the internet back at us. Through compressed world models, deliberate reasoning, test-time compute and automated verification, it has grown from a passive autocomplete engine into an active problem-solver – one whose inner workings still hold genuine mysteries for the very people who built it.

We are leaving the era of asking machines what they know, and entering one where we work alongside them because of how they think. For businesses, the practical question is no longer whether AI will matter, but how to adopt it securely, privately and with a clear purpose. If that’s on your mind, our cyber security team can help you do it without putting your data at risk.