Adam Marblestone – AI is missing something fundamental about the brain

Current AI systems are limited by their lack of understanding of the brain's complex functions and mechanisms, and that incorporating insights from brain function and structure could be crucial for developing more advanced and human-like artificial intelligence

 

Questions to inspire discussion

Understanding Brain's Learning Advantage

🧠 Q: What is the fundamental difference between how brains and current AI systems learn? A: The brain's superior learning from limited data comes from its complex reward functions encoded in the genome, not from its architecture or learning algorithms, suggesting AI needs better reward shaping rather than just architectural improvements.

🎯 Q: How does the brain implement curriculum learning during development? A: Evolution built different loss functions for different brain areas that activate at specific developmental stages, creating an automatic curriculum that guides learning from simple to complex tasks without external supervision.

🔄 Q: What makes the cortex a general prediction engine? A: The cortex learns to predict any subset of inputs given any other missing subset, enabling omnidirectional inference where the same neural substrate can perform diverse tasks by predicting different combinations of sensory and motor signals.

Brain Architecture Insights

🏗️ Q: What are the two main subsystems in the brain and their roles? A: The Learning Subsystem (cortex) builds world models through prediction with relatively fixed architecture, while the Steering Subsystem contains innate reward functions and has greater diversity of cell types requiring significant genomic information for pre-wiring.

👁️ Q: What innate capabilities does the Steering Subsystem provide? A: The superior colliculus and other Steering Subsystem components have innate heuristics for detecting faces, threats, social status, and friendliness using primitive sensory inputs rather than abstract learned representations.

🧬 Q: How much information does the genome actually encode about the brain? A: The genome conveys relatively little information compared to the brain's total complexity, with most functionality built through learning and plasticity rather than hardcoded instructions, focusing genomic resources on reward functions rather than architecture.

Reinforcement Learning Mechanisms

⚡ Q: What does dopamine actually signal in the brain? A: Dopamine signals reward prediction errors, not just rewards themselves, which is consistent with learning value functions in RL and supported by extensive neuroscience evidence.

🎮 Q: How does the brain combine model-free and model-based RL? A: The striatum and basal ganglia implement simple model-free RL with finite action spaces, while the cortex builds world models that predict actions and rewards, enabling complex RL with generalization and abstract concepts.

🔬 Q: What unique RL approach does the brain use that differs from current AI? A: The brain employs a unique form of reinforcement learning that integrates innate reward functions with world model predictions, creating a hybrid system that learns efficiently from limited data through biologically-inspired mechanisms.

Biological vs Digital Computation

💾 Q: What computational advantages does biological hardware provide? A: The brain performs unstructured sparsity and co-locates memory and compute at the cellular level, enabling cognitive dexterity that current computer hardware cannot replicate due to architectural constraints.

🔄 Q: How do cellular changes during learning compare to digital weight updates? A: Brain learning involves weight normalization and memory consolidation that are algorithmically similar to changing weights in digital computers but require more complex biological machinery to implement.

🧪 Q: Should biological hardware be viewed as a limitation for AI development? A: Biological hardware may be an advantage rather than limitation, as exploring biological constraints and mechanisms can inform novel AI architectures that leverage brain-like computational principles.

Human Brain Expansion

🗣️ Q: What brain expansions enabled human language and social learning? A: Both the cortex (enabling omnidirectional inference) and Steering Subsystem (developing social instincts) expanded in humans compared to other animals, with both systems being crucial for language and social learning.

👥 Q: How do social instincts relate to brain architecture? A: The Steering Subsystem contains innate heuristics for social status and friendliness that are pre-wired through genomic information, providing foundational social capabilities that learning builds upon.

Brain Mapping Technology

🔬 Q: What is the cost target for connectomic brain mapping? A: E11 Bio aims to reduce connectomic brain mapping costs from billions to tens of millions for a mouse brain connectome, making it feasible to map multiple mammal brains with different social instincts.

🧬 Q: What information do molecularly annotated connectomes provide? A: Molecularly annotated connectomes reveal cell types, synapse properties, and brain activity, enabling correlation of structure to function and prediction of brain activity patterns directly from connectome data.

📱 Q: How could portable brain scanners accelerate AI development? A: Portable brain scanners could generate brain activity patterns at scale, enabling AI training with brain signals similar to how GPUs enabled large-scale AI training, creating new training paradigms.

AI Training with Brain Data

🧠 Q: What are surrogate models in brain-AI research? A: Surrogate models distill brain activity data into neural networks, while brain-data-augmented AI adds auxiliary loss functions to predict brain activity patterns, sculpting networks to represent information like biological brains.

📊 Q: How can brain signals improve AI training? A: Adding brain activity prediction as an auxiliary loss function during training sculpts neural networks to process information using brain-like representations, potentially improving generalization and data efficiency.

Abstract Reasoning and Generalization

🎯 Q: How does the cortex achieve generalization? A: The cortex generalizes by predicting based on abstract variables and integrated information rather than raw sensory inputs, building hierarchical representations that capture underlying causal structure.

🔍 Q: What enables the brain's superior abstraction capabilities? A: The Learning Subsystem predicts using abstract variables in the world model while the Steering Subsystem uses primitive sensory inputs, creating a hierarchy from concrete to abstract representations.

Test-Time Computation

🤔 Q: How does test-time compute in AI relate to brain function? A: Test-time compute in AI models resembles sampling in the brain, where the model evaluates different approaches during inference, amortizing the need for extensive sampling during training.

Mathematics Automation

🔢 Q: What is RLVRing and its purpose? A: RLVRing (Reinforcement Learning with Verified Rewards) automates the mechanical parts of mathematics like validating lemmas and proofs, shifting the burden from humans to AI systems.

🎓 Q: What mathematical capabilities would AGI need? A: AGI would perform all human mathematical tasks including creative aspects like conjecturing new ideas and organizing concepts, not just mechanical proof verification.

AGI Development Principles

🤖 Q: What defines AGI according to this framework? A: AGI should have general ability to learn a world model but may need to focus on different salient factors relevant to post-singularity environments beyond just building comprehensive world models.

🌍 Q: How do evolution and culture relate to RL? A: Evolution and culture function as model-free RL at civilizational level, where learning happens through trial and error over generations, analogous to model-free RL algorithms.

Research Priorities

🔬 Q: Why is mapping the human brain essential for AI? A: Mapping the human brain is crucial to uncover fundamental principles of brain function and learning mechanisms that can inform AI development and lead to transformative insights.

🧩 Q: What is the relationship between architecture, loss functions, and learning rules? A: The brain's architecture, loss functions, and learning rules form a complex interplay rather than simple symbolic language, and studying this messy combination reveals key principles for intelligent systems.

🎯 Q: What should AI researchers focus on from neuroscience? A: Understanding the brain's reward functions encoded in the genome is fundamental for improving AI systems, as these guide learning more effectively than architectural innovations alone.

Investment and Resources

💰 Q: What level of investment is needed for brain mapping? A: Mapping the human brain requires significant investment to achieve cost reductions from billions to tens of millions per connectome, enabling systematic study of brain learning mechanisms.

🔬 Q: What makes connectomics feasible now? A: Advances in technology aim to make connectomic brain mapping several orders of magnitude cheaper, enabling mapping of multiple mammal brains to study variations in social instincts and learning.

 

Key Insights

Brain's Learning Architecture

🧠 The brain's reward functions, not its architecture, are the key to learning efficiency - the genome encodes innate behaviors and bootstrapping cost functions for building reward signals rather than the brain's architectural blueprint itself.

🎯 Evolution built complex loss functions into the brain with many different ones for different areas, turned on at different development stages, functioning like a Python code curriculum for brain learning.

🔄 The cortex operates as a universal prediction engine, learning to predict any subset of variables from any other missing subset, enabling omnidirectional inference unlike LLMs which compute only specific conditional probability for the next token.

⚙️ Understanding the brain's cortex requires describing how it embodies a learning algorithm - the architecture, training data, and loss function are key to understanding its capabilities, similar to how we understand neural networks.

Steering vs Learning Subsystems

🎛️ The brain's Steering Subsystem has a greater diversity of bespoke cell types compared to the Learning Subsystem, responsible for innately wired circuits and specific reward functions, while the Learning Subsystem's architecture is more uniform with changes through synaptic plasticity.

👁️ The Steering Subsystem has innate sensory systems like the superior colliculus for detecting faces and threats, and innate responses in the hypothalamus and brainstem, operating separately from the cortex.

🧩 The cortex generalizes by predicting based on abstract variables and integrated information, while the Steering Subsystem relies on primitive sensory input like the superior colliculus for its responses.

🤝 The Steering Subsystem models its own innate heuristics for social status and friendship, using them to wire up learned world model features to innate reward functions.

Reinforcement Learning in the Brain

🎮 The brain implements both model-free RL (striatum, basal ganglia with finite action space) and model-based approaches, with complexity exceeding current AI systems that rely on simplistic model-free RL.

💊 The basal ganglia and cortex use simple RL algorithms and world models to predict rewards from actions, with dopamine providing reward prediction error signal to learn value functions, supported by Peter Dayan's neuroscience work that influenced DeepMind's temporal difference learning.

🌍 Evolution and culture function as model-free RL at a civilizational level, where humans learn complex survival strategies through trial and error over generations, demonstrating that simple algorithms can achieve anything given enough time.

Social Learning and Evolution

🗣️ The brain's cortex and Steering Subsystem evolved to enable social learning and communication, increasing the incentive for a larger cortex capable of omnidirectional inference - this expansion can be achieved with a relatively small number of gene changes.

Biological Hardware Trade-offs

💻 The brain's biological hardware has advantages like cognitive dexterity through unstructured sparsity and co-located memory and compute, but disadvantages in copying and random-accessing individual neurons and synapses, potentially limiting co-design of algorithms and hardware compared to digital systems.

🔬 The brain's cellular changes and molecular machinery are necessary for learning and memory, implementing algorithms easily done in digital computers by changing weights, but with added complexity of modulating synapses according to gradient signals.

Brain Mapping Technology

🔭 E11 Bio aims to make connectomic brain mapping several orders of magnitude cheaper - from a mouse brain connectome costing billions to tens of millions, with the goal of mapping human brains at a fraction of the current multi-billion dollar cost.

🔬 Molecularly annotated connectomes from E11's optical microscopy reveal not just who is connected to who by synapse type, but also molecules present at each synapse and cell types involved, enabling structure-function correlation and activity mapping.

AI Training Innovations

🧠 Gwern's proposal suggests training models on the brain's hidden states, enabled by E11's molecularly annotated connectomes that map brain activity and structure at a level beyond just synaptic connections.

🎯 Behavior cloning involves training AI not just on labels like "cat" or "dog", but also on predicted human neural activity patterns when perceiving those stimuli, potentially enabling richer generalization and representation in AI models.

Mathematical Proof Systems

📐 RLVR (Reinforcement Learning with Verified Rewards) in Lean, a verifiable programming language for math proofs, enables searching for proofs and finding them analogous to AlphaGo's gameplay search, with a verifiable correctness signal.

🤖 AGI capable of conjecturing new ideas and conceptualizing math would do everything a human mathematician can do including creative aspects, but RLVR will still be a powerful tool for accelerating the mechanical parts of math.

🔒 LLMs can generate verifiable proofs of software correctness, enabling provably stable, secure, unhackable software with mathematical guarantees beyond unit tests, functioning as a powerful cybersecurity tool for various AI and real-world challenges.

⚠️ The specification problem in provable programming - where formal specs for complex systems like power grid code are difficult to derive - presents a user interface and AI challenge in determining what security properties to specify and verifying spec correctness.

Research Infrastructure

📊 The Gap Map, a list of fundamental capability gaps in science and technology, reveals that scalable infrastructure is needed across many fields including math (Lean for formal proofs), biology (-omics research), and even astronomy to enable significant progress.

Brain Mechanisms

🔄 The corticothalamic system involving both cortex and thalamus may perform functions analogous to attention mechanisms in transformers, such as matching or constraint satisfaction across different brain areas, but the exact nature of this attention remains unclear.

🧩 The brain's representations are likely a complex combination of architecture, loss functions, and learning rules rather than a clean symbolic language, with a messy and intricate organization that may not resemble traditional symbolic systems.

💾 The hippocampus plays a key role in memory consolidation and continual learning, with functions including storing and replaying sequences, organizing and representing memories, and potentially training the cortex, though exact mechanisms remain not fully understood.

AGI Capabilities

🎯 AGI or superhuman intelligence should have the ability to learn a general world model and pay attention to salient factors relevant for its environment, but may not require human-like social instincts and ethics, which are more about alignment than capabilities.

 

#SyntheticMinds #AI #Connectome

XMentions: @HabitatsDigital @dwarkesh_sp @AdamMarblestone

Clips

• 00:00 🤖 The brain's learning capabilities surpass current AI systems due to its complex, evolutionarily-encoded loss functions and ability to integrate information through "omnidirectional inference", allowing it to generalize and adapt more effectively.
  • The brain's learning capabilities far surpass those of current AI systems, likely due to complex, evolutionarily-encoded loss functions that guide learning, which are neglected in machine learning's focus on simple, mathematically-defined objectives.
  • The brain's cortex may be capable of "omnidirectional inference", where any area can predict any pattern in a subset of inputs given another subset, unlike current AI models like LLMs that are limited to specific tasks like next-token prediction.
  • The brain's higher-level areas can predict and associate sensory inputs, such as vision or hearing, with innate lizard brain responses, like reflexes or instinctive behaviors.
  • Evolution must encode high-level desires and intentions, such as avoiding social embarrassment, in a way that allows the brain to robustly wire learned features to innate reward functions.
  • The brain's Learning Subsystem, including the cortex, learns to predict responses from the innate Steering Subsystem, which has pre-programmed heuristics and reactions, allowing it to generalize and adapt to new situations.
  • The brain's Steering Subsystem, which handles innate responses, can learn to generalize and predict potential threats through abstract concepts, but its ability to do so is limited compared to the cortex, which can integrate complex information and generalize more effectively.
• 14:43 🤖 Current AI models lack a fundamental aspect of the brain: a flexible, multimodal learning algorithm that integrates different sensory data and enables omnidirectional inference, linking desires to rewards.
  • The brain likely uses a flexible, multimodal learning algorithm, possibly a modified version of backpropagation or an energy-based model, to enable omnidirectional inference and link higher-level desires to primitive rewards, which current AI models lack.
  • The missing piece in artificial neural networks may not be the reward function, but rather how different sensory data, such as video, audio, and text, are represented and interconnected, potentially requiring better encoding or embedding to enable meaningful connections between ideas.
  • Neural networks may be able to efficiently approximate Bayesian inference, which is the process of updating internal models of the world based on new observations, by directly mapping inputs to likely causes rather than sampling and evaluating all possible causes.
  • Digital minds, unlike biological minds, may benefit from amortizing more computational processes, such as inference-time compute, into their models as they can be easily copied, raising questions about what computations will be amortized in future AI systems.
  • Evolution may have optimized innate behaviors and reward signals to aid learning, rather than pre-training the brain, allowing for intelligence to emerge with minimal genetic information.
  • AI can build complex reward functions by combining innate knowledge with a compact learning algorithm, allowing it to generalize and adapt to various situations without needing to anticipate every detail.
• 30:16 🤖 Current AI systems are missing fundamental aspects of brain function, including innate hardwired circuits, value functions, and complex cellular machinery, which may be crucial for achieving human-like intelligence.
  • The brain's "Steering Subsystem", which handles innate behaviors and reward functions, has a much greater diversity of cell types and likely genetic complexity than the "Learning Subsystem", suggesting that AI is missing fundamental aspects of brain function by not incorporating similar innate, hardwired circuits.
  • The difference between human and animal brains, including advanced cognitive abilities, may be enabled by a relatively small number of genetic changes that tweaked existing brain structures, rather than fundamentally rewriting them.
  • Current AI systems lack understanding of fundamental brain mechanisms, such as primitive reinforcement learning, associative learning, and model-based versus model-free learning, which are present in even simple brains and crucial for human intelligence.
  • Current AI systems, like large language models, lack value functions, a fundamental aspect of the brain's reinforcement learning mechanisms, which enable predicting long-term consequences of actions and learning from rewards.
  • The brain's unique hardware, despite being energetically efficient and capable of cognitive dexterity through unstructured sparsity and co-located memory and compute, may have limitations that could be overcome by co-designing algorithms with advanced hardware to achieve better performance.
  • The speaker believes that while AI has made impressive progress, it may be missing a fundamental aspect of the brain: the complex cellular machinery that underlies neural function, which may be doing more than just implementing synaptic connections.
• 57:57 🤖 Current AI models may be missing fundamental concepts about the brain, requiring a new approach and vocabulary to understand brain function and potentially align AI with human values.
  • Creating a powerful AI system, like a paperclip maximizer, may require a minimal set of drives, including curiosity and social interaction, which could potentially be aligned with human values but also poses concerns about its goals and behavior.
  • Current AI models, inspired by neuroscience, may still be missing fundamental concepts about the brain and may require a new vocabulary and approach, rather than simply applying existing AI ideas to brain function.
  • AI research should pursue two complementary approaches: a bottom-up simulation of brain systems and a reverse-engineering approach using AI-inspired vocabulary to understand the brain's workings.
  • Understanding the brain requires describing it in terms of architectures, learning rules, and initializations, rather than trying to precisely replicate specific neural patterns, and the connectome can provide valuable constraints to refine our understanding of the brain's learning algorithm.
  • Current AI research on interpretability may not be practical or relevant for the development of future AI systems, with potentially transformative advancements still more than five years away.
  • The speaker questions the timelines of other AI researchers, such as Karpathy and Demis, and wonders about their own timeline and motivations.
• 01:10:17 🤖 Current AI research may be misguided without incorporating fundamental brain aspects, but mapping the brain's connectome could accelerate AI development with focused funding.
  • Current AI research may be misguided if it doesn't incorporate fundamental aspects of the brain, making it uncertain if advancements like GPT-5 are truly effective.
  • Mapping the human brain's connectome could cost billions of dollars, but focused funding of hundreds of millions to low billions could enable significant progress in 10 years.
  • A connectome, particularly a "molecularly annotated" one, can describe neural connections and molecular properties, but does not directly imply synaptic weights or the ability to simulate neural function.
  • The Human Genome Project's high initial cost was reduced by millions-fold by shifting from macroscopic chemical techniques to parallelized methods analyzing individual DNA molecules, a paradigm shift that could similarly accelerate and cheapen future brain mapping technologies.
  • Mapping the brain through connectomics could accelerate AI development by providing fundamental insights into its workings, but funding and linking it to AI applications remains uncertain.
  • Investing in comprehensive research, such as mapping the connectome, through moonshot startups, FROs, or corporate labs, can be a viable approach to advancing neuroscience and AI, despite potential risks of such endeavors going awry.
• 01:19:43 🤖 Training AI with brain activity patterns and incorporating brain-data-augmented training could improve AI's ability to generalize and generate new ideas, leading to breakthroughs in math and cybersecurity.
  • Training AI with both labeled data and brain activity patterns could help the network represent information in a way consistent with human brain representations, potentially leading to better generalization and richer labeling.
  • Current AI models can be improved by incorporating brain-data-augmented training, where the model predicts not only output labels but also internal brain signal measurements, effectively adding an auxiliary loss function for consistency with brain representations.
  • AI can automate certain parts of math, particularly formalized math proofing using programming languages like Lean, which can mechanically verify proofs and enable machines to search for and find new proofs.
  • Current AI advancements in math, while able to automate mechanical parts like proof validation, still lack the fundamental ability to generate new, interesting ideas and conjectures, which may require a loss function for good explanations.
  • AI will significantly impact math and cybersecurity by enabling provably stable and secure software through mathematical proofs, potentially leading to formally verified software and breakthroughs in complex problems like the Riemann hypothesis.
  • Formal verification and specification of security properties in code is a complex problem, but advancements in large language models may soon make it easier to generate verifiable proofs for software and hardware verification.
• 01:31:27 🤖 Current AI systems lack a fundamental understanding of the brain's workings, particularly in areas like continual learning, symbolic operations, and world model representation, hindering the development of true AGI.
  • AI-assisted math tools may revolutionize mathematics by increasing accessibility, speed, and innovation, but it's unclear whether they will replace or enhance human intuition and creativity in developing new mathematical concepts.
  • Future AGI systems might rely on combining automated cleverness with provable symbolic representations, enabling intelligences to collaborate and build on each other's work through formally verifiable statements.
  • The brain's representation of the world model is likely a complex, non-symbolic system that may resemble a geometric or spatial map, but the specifics of how it organizes abstract concepts and enables symbolic operations, such as variable binding, remain unclear.
  • The speaker believes that continual learning in the brain may involve fundamental aspects of neural architecture, memory storage, and plasticity, but the exact mechanisms, including how the hippocampus and corticothalamic system contribute, remain unclear.
  • The Gap Map, a list of fundamental capabilities needed to advance research, reveals that filling gaps in scientific progress may require only a few hundred deep tech startup-size projects, totaling a relatively manageable amount of funding, rather than a trillion dollars.
  • Many areas of science, including math, AI, and genomics, require scalable infrastructure, such as specialized tools and programming languages, to make progress, in addition to low-scale creative work.
• 01:49:08 🤝 Adam Marblestone's online presence can be found at convergentresearch.org and his blog, Longitudinal Science on WordPress.

-------------------------------------

Duration: 1:49:54

Publication Date: 2025-12-30T18:57:02Z

WatchUrl:https://www.youtube.com/watch?v=_9V_Hbe-N1A

-------------------------------------