Estimated Reading Time: 4 minutes

Imagine You’re sitting in a quiet room, sipping coffee. Outside, the world is silent, inside, billions of neurons fire, cells glow, memories strengthen, expectations form, and predictions whisper through your mind, all while consuming about the same energy as a dim light bulb. Every time you blink, this biological machinery updates itself effortlessly. Now imagine trying to replicate such elegance and power in computers. That contrast alone sets the stage for the deeper question that my write up explores. While the commercial angle is essential for companies, and we are seeing that out there, the labs can learn more from the principles of real brains. Should modern architectures like Transformers begin borrowing from neuroscience again?

For much of early AI, the connection to neuroscience was direct. Artificial neural networks were named and framed after biological neurons and synapses, even if the resemblance was mostly conceptual. But as deep learning matured, performance, scale and efficiency began to dominate over biological plausibility. Engineers discovered architectures and training procedures that scaled with compute, even if they diverged from how real brains operate. For today’s AI giants, Transformer models are achievements of mathematics, hardware, and enormous datasets. They are powerful, but not brain like, and maybe they should not be.

Meanwhile, neuroscience kept advancing. Work from institutions around the world, including Numenta, my personal favourite organisation, and the broader community around the Thousand Brains Project, began revealing principles about cortical structure, sensorimotor learning, and distributed memory systems that increasingly align with what future AI will need.

Intelligence in the brain isn’t about memorising patterns. It’s a dynamic dance of memory, prediction, correction, and continual refinement. The cortical theory underlying the Thousand Brains Project holds that each cortical column can learn complete models of objects through movement and sensation. In other words, the brain does not wait for labelled datasets. It learns through interaction, exploration, and experience. The Thousand Brains Project bring this theory to life through Monty, their first public implementation. Instead of treating perception as a passive, feedforward task, Monty approaches understanding as an active, sensorimotor driven process. When you explore the details, you see a system that learns not from frozen datasets, but from repeated interaction. Monty observes an object, moves around it, and refines its internal representation over time. As different viewpoints and sensory conditions accumulate, they build a stable, robust model of the object, just as a biological cortex would. This learning process doesn’t erase its past, it enriches it.

This kind of learning stands in stark contrast with Transformer architectures. Transformers operate under a different philosophy, ingest massive amounts of text or images, compress them into billions of parameters, freeze the model, and serve it. If you want to teach a Transformer something new, you must fine tune it or retrain it, risking catastrophic forgetting or computational cost that would make a neuron weep. There is nothing in the design that supports the everyday experience of learning. The memory is static, global, and entangled, an update to one concept can ripple unpredictably through many others.

Per my understanding, Neuroscience offers a much better blueprint for continuous learning. In the brain, memory is distributed across vast networks of neurons, and learning happens locally. If you learn a new concept, your brain updates only the relevant synapses, reshapes only the necessary circuits, and integrates the latest information without erasing old memories. Cortical columns maintain stable reference frames that help you recognise an object under rotation, noise, occlusion, or new contexts. The Thousand Brains theory provides direct demonstrations of this principle. When Monty learns using unsupervised continual learning, it responds to new experiences by strengthening and refining its existing object models rather than overwriting them.

Even at a broader architectural level, the Thousand Brains framework suggests a path forward for AI. Each learning module behaves like a mini cortex, which is independent, modular, and capable of forming its own understanding of objects. But these modules can collaborate. When several modules perceive the same scene through different sensors, they “vote” on their interpretations, echoing how real cortical columns converge on unified perceptions. This distributed, modular style of learning provides the structural foundation AI systems will need if they are ever to learn continuously without forgetting.

Transformers, for all their success, are missing this structural foundation. Their representations are global, their updates brittle, their memory shallow, and their learning phase strictly separated from their deployment phase. Neuroscience shows us a different way, learning should always be happening, even if only a little at a time. Brains learn through prediction and correction. They expect the world to move and change. They form stable concepts by anchoring them in sensorimotor coordinates rather than pixel statistics. They refine knowledge through experience, not retraining cycles. In short, they learn the way Monty learns incrementally, adaptively, interactively.

This vision points toward AI systems that can learn throughout their lifetimes. Imagine a model that becomes more attuned to you each day, absorbing your style, preferences, rhythms, constraints, and values through experience rather than expensive retraining. Imagine a robot that doesn’t need a million samples of the same object but can understand it after exploring it in real world conditions. Imagine a reasoning agent that doesn’t freeze after training but grows more competent the more it works with you. These scenarios are natural for biological systems but challenging for today’s machine learning models.

Continuous learning is not just desirable, it is essential for any system that hopes to approach general intelligence. The Thousand Brains framework provides a theoretical and practical path toward this capability. And its principles hint at how AI might ultimately merge with the strengths of Transformers rather than replace them. A hybrid system, one that uses Transformer like models for symbolic reasoning and communication, paired with cortical-inspired modules for perception, prediction, and learning, could represent the next significant milestone in AI’s evolution.

In the café scene, I like to imagine a Transformer, a Monty module, and a neuroscientist sitting together reviewing the same object. The neuroscientist explains how the brain maintains stable reference frames, Monty quietly builds a 3D model through simulated movement, and the Transformer provides contextual knowledge pulled from a vast corpus of text. Each has a role, but only one of them learns during the conversation. The future may belong to the systems that combine all three the speed and scale of Transformers, the grounding and adaptability of cortical models, and the biological elegance that neuroscience continues to reveal.

In the end, the most profound insight from neuroscience, the Thousand Brains Project, Monty, and even the limitations of our current AI systems is that intelligence is not static. It is cultivated. It evolves through experience.