Richard Sutton's 'Experience Era': The RL Father's WAIC 2026 Keynote Deep Dive — AI's Paradigm Shift from Data to Experience

Richard Sutton’s ‘Experience Era’: The RL Father’s WAIC 2026 Keynote Deep Dive — AI’s Paradigm Shift from Data to Experience

Introduction: A Paradigm Shift in Progress

July 17, 2026, Shanghai World Expo Center. Richard Sutton, the 2024 Turing Award laureate and “Father of Reinforcement Learning,” stood on the WAIC 2026 main forum stage and delivered a statement that silenced the room: “The progress of AI is exaggerated.”

This is not a tech giant’s mockery of a competitor, but the most sobering assessment from the world’s most influential reinforcement learning researcher. Sutton’s core thesis is sending shockwaves through the industry: AI is transitioning from the “Human Data Era” to the “Experience Era” — the current paradigm of “predicting the next token” is approaching its limits, and true intelligence must emerge from agents continuously interacting with the real world, accumulating first-person experiential knowledge.

This article provides an in-depth technical analysis of Sutton’s WAIC 2026 keynote, his newly proposed OaK (Options and Knowledge) architecture, the vision of a Comprehensive Mental Science, and the strategic direction of his new venture Oak Lab, with Go and Python implementations of the core concepts.

1. The Core Distinction: Computation vs. Intelligence

1.1 Current AI: Large-Scale Pattern Recognition

Sutton opened his speech by identifying a systematic overestimation of AI capabilities. The most significant breakthroughs in recent years — language proficiency, image and video generation — are fundamentally large-scale pattern recognition capabilities, not genuine intelligence.

“We must not confuse computation with intelligence. Most current AI capabilities are still fundamentally built on large-scale computation and pattern recognition.”

This directly challenges the dominant narrative of the AI industry. Sutton argues that today’s large language models primarily repackage existing human knowledge rather than discovering new knowledge. They are high-performance “knowledge recombination engines,” not “knowledge discovery engines.”

// Knowledge Reconstructor vs Knowledge Discoverer
package main

import "fmt"

type KnowledgeReconstructor struct {
    KnowledgeBase map[string]string
}

func (kr *KnowledgeReconstructor) Reconstruct(query string) string {
    if answer, exists := kr.KnowledgeBase[query]; exists {
        return fmt.Sprintf("Recombined from existing knowledge: %s", answer)
    }
    for k, v := range kr.KnowledgeBase {
        if fuzzyMatch(k, query) {
            return fmt.Sprintf("Approximate recombination: %s", v)
        }
    }
    return "No matching knowledge"
}

func fuzzyMatch(a, b string) bool {
    return len(a) > 0 && len(b) > 0 && a[0] == b[0]
}

type Experience struct {
    State     string
    Action    string
    Reward    float64
    NextState string
}

type KnowledgeDiscoverer struct {
    ExperienceBuffer []Experience
}

func (kd *KnowledgeDiscoverer) Discover(initialState string) string {
    state := initialState
    for i := 0; i < 100; i++ {
        action := "explore"
        nextState := state + "_next"
        reward := 1.0
        kd.ExperienceBuffer = append(kd.ExperienceBuffer, Experience{
            State: state, Action: action, Reward: reward, NextState: nextState,
        })
        state = nextState
    }
    return fmt.Sprintf("Generated %d new experiences through interaction", len(kd.ExperienceBuffer))
}

func main() {
    reconstructor := &KnowledgeReconstructor{
        KnowledgeBase: map[string]string{
            "What is RL": "RL is a paradigm where agents learn through interaction with environments",
        },
    }
    fmt.Println(reconstructor.Reconstruct("What is RL"))

    discoverer := &KnowledgeDiscoverer{}
    fmt.Println(discoverer.Discover("initial"))
}

1.2 Seven Definitions of Intelligence

Sutton organized a spectrum of intelligence definitions across disciplines, forming a progressively refined framework:

Sutton’s definition — intelligence is the ability to achieve goals through behavioral adaptation — positions “adaptation” and “behavior” at the core, providing a theoretical foundation for reinforcement learning as a paradigm.

2. Comprehensive Mental Science: Beyond AI’s Disciplinary Boundaries

2.1 Coverage Blind Spots

Sutton proposed a visionary framework: establishing a Comprehensive Mental Science covering humans, animals, and machines. No existing discipline fully covers this domain:

• Psychology: Focuses on human minds, excludes machine intelligence
• AI: Focuses on machine intelligence, ignores biological intelligence commonalities
• Cognitive Science: Cross-disciplinary but lacks unified theoretical framework

The foundation stone of this comprehensive mental science is reinforcement learning — the only learning paradigm applicable to humans, animals, and machines.

2.2 RL as Universal Framework

"""
Comprehensive Mental Science Framework
— Unified paradigm for humans, animals, and machines
"""
import numpy as np
from typing import Tuple, List, Callable
from dataclasses import dataclass
from collections import defaultdict

@dataclass
class Agent:
    """Universal agent — representing human, animal, or machine"""
    policy: Callable
    value_function: Callable
    experience_memory: List
    learning_rate: float = 0.1
    discount_factor: float = 0.95

class ComprehensiveMentalScience:
    def __init__(self):
        self.agents = {}
        self.shared_metrics = {
            'adaptation_speed': [],
            'goal_achievement_rate': [],
            'exploration_efficiency': [],
        }
    
    def register_agent(self, name: str, agent: Agent):
        self.agents[name] = agent
    
    def run_episode(self, agent_name: str, environment: Callable, 
                    max_steps: int = 1000) -> Tuple[float, List]:
        # Core interaction loop: Observe → Act → Reward → Learn
        agent = self.agents[agent_name]
        state = environment(reset=True)
        total_reward = 0
        trajectory = []
        
        for step in range(max_steps):
            action = agent.policy(state)
            next_state, reward, done = environment(action=action)
            agent.experience_memory.append((state, action, reward, next_state))
            trajectory.append((state, action, reward))
            
            # Q-learning update
            current_q = agent.value_function(state, action)
            next_max_q = max(agent.value_function(next_state, a) for a in ['left', 'right'])
            new_q = current_q + agent.learning_rate * (
                reward + agent.discount_factor * next_max_q - current_q
            )
            
            total_reward += reward
            state = next_state
            if done:
                break
        
        return total_reward, trajectory

# Run cross-species experiment
cms = ComprehensiveMentalScience()
human_agent = Agent(
    policy=lambda s: 'explore' if np.random.random() < 0.3 else 'exploit',
    value_function=lambda s, a: 0.0,
    experience_memory=[],
)
machine_agent = Agent(
    policy=lambda s: np.random.choice(['left', 'right']),
    value_function=lambda s, a: 0.0,
    experience_memory=[],
)
cms.register_agent('human', human_agent)
cms.register_agent('machine', machine_agent)

def simple_env(action=None, reset=False):
    if reset: return 'start'
    return 'goal', 1.0 if action == 'exploit' else 0.1, True

for species in ['human', 'machine']:
    reward, traj = cms.run_episode(species, simple_env)
    print(f"{species} agent total reward: {reward:.2f}, steps: {len(traj)}")

3. The Limits of the Human Data Era and the Dawn of the Experience Era

3.1 Three Fundamental Limitations

Sutton identified three critical limitations of the current data-driven paradigm:

Limitation 1: High-quality human data is being exhausted. Epoch AI Research estimates that high-quality text data may be fully depleted by 2026-2028. Simply scaling data volume can no longer sustain model improvement.

Limitation 2: Static datasets cannot generate new knowledge. Pre-training corpora are snapshots of past human knowledge, unable to capture ongoing dynamics or generate knowledge beyond human cognition.

Limitation 3: Absence of reward signals. LLMs lack genuine reward mechanisms, making them unable to judge the correctness of their behavior or form goal-directed learning.

3.2 The Three Elements of Experiential Learning

1. First-person data: Observations, actions, and rewards generated through agent-environment interaction
2. Continuous interaction: Data is not static but a flowing stream generated through ongoing engagement
3. Reward mechanisms: Standards for evaluating behavior quality, enabling goal-directed learning

Sutton used the analogy of infant learning: “Babies constantly try different toys. Their behavior determines what information they access and what they learn. These data are not pre-prepared static datasets but emerge naturally through interaction.”

4. The OaK Architecture: Options and Knowledge

4.1 Design Principles

At the WAIC 2026 Thinkers Forum, Sutton fully presented the OaK (Options and Knowledge) architecture, which enables macro-level planning through two abstraction mechanisms:

Temporal Abstraction (Options): Encapsulating primitive action sequences into high-level skills. For example, “make coffee” is an option comprising: get cup → add coffee → add water → heat → serve.

State Abstraction (Knowledge): Mapping low-dimensional perceptual states to high-dimensional concept spaces. An agent doesn’t need to track every dust particle — it just needs to know “the room is clean.”

4.2 OaK Implementation

"""
OaK (Options and Knowledge) Architecture Core Implementation
"""
import numpy as np
from typing import Dict, List, Tuple, Callable, Optional
from dataclasses import dataclass
from enum import Enum

class PrimitiveAction(Enum):
    MOVE_LEFT = "left"
    MOVE_RIGHT = "right"
    GRAB = "grab"
    ACTIVATE = "activate"

@dataclass
class Option:
    """Temporal abstraction unit"""
    name: str
    initiation_set: Callable
    policy: Callable
    termination_condition: Callable
    expected_reward: float = 0.0
    learned_duration: float = 0.0
    
    def execute(self, state, max_steps: int = 100) -> Tuple[List, float]:
        trajectory = []
        total_reward = 0.0
        for step in range(max_steps):
            if self.termination_condition(state):
                break
            action = self.policy(state)
            trajectory.append((state, action))
            next_state = f"{state}_{action.value}"
            reward = 1.0 if "goal" in str(next_state) else -0.01
            total_reward += reward
            state = next_state
        self.learned_duration = len(trajectory)
        self.expected_reward = (self.expected_reward * len(trajectory) + total_reward) / (len(trajectory) + 1)
        return trajectory, total_reward

@dataclass
class Knowledge:
    """State abstraction unit"""
    name: str
    feature_extractor: Callable
    transition_model: Callable
    is_achievable: Callable

class OaKAgent:
    def __init__(self):
        self.options: Dict[str, Option] = {}
        self.knowledge_base: Dict[str, Knowledge] = {}
        self.option_usage = {}
    
    def add_option(self, option: Option):
        self.options[option.name] = option
    
    def add_knowledge(self, knowledge: Knowledge):
        self.knowledge_base[knowledge.name] = knowledge
    
    def decompose_goal(self, goal: str, current_state: str) -> List[str]:
        subgoals = []
        for name, knowledge in self.knowledge_base.items():
            if not knowledge.is_achievable(current_state):
                subgoals.append(name)
        return subgoals[:3]
    
    def plan(self, initial_state: str, goal: str) -> List[str]:
        subgoals = self.decompose_goal(goal, initial_state)
        plan = []
        current_state = initial_state
        
        for subgoal in subgoals:
            best_option = None
            best_value = float('-inf')
            for opt_name, option in self.options.items():
                if option.initiation_set(current_state):
                    value = option.expected_reward - 0.1 * option.learned_duration
                    if value > best_value:
                        best_value = value
                        best_option = opt_name
            if best_option:
                plan.append(best_option)
                current_state = f"{current_state}_{best_option}"
        return plan

# Example: robot planning with OaK
agent = OaKAgent()
agent.add_option(Option(
    name="navigate_to_kitchen",
    initiation_set=lambda s: "living_room" in str(s),
    policy=lambda s: PrimitiveAction.MOVE_LEFT,
    termination_condition=lambda s: "kitchen" in str(s),
))
agent.add_option(Option(
    name="make_coffee",
    initiation_set=lambda s: "kitchen" in str(s),
    policy=lambda s: PrimitiveAction.ACTIVATE,
    termination_condition=lambda s: "coffee_ready" in str(s),
))
agent.add_knowledge(Knowledge(
    name="have_coffee",
    feature_extractor=lambda s: hash(s) % 100,
    transition_model=lambda f, a: f"coffee_{a}",
    is_achievable=lambda s: "coffee" in str(s)
))

plan = agent.plan("living_room", "have_coffee")
print(f"Macro plan: {' → '.join(plan) if plan else 'No plan available'}")

4.3 World Model Critique

Sutton sharply criticized the industry’s reduction of “world models” to “low-level physics simulators,” calling this understanding “extremely narrow.” Human transition models “should not be as trivial as low-level physics but should carry high-level world knowledge.”

Key distinction:

• Low-level physics simulator: Predicting next pixels, next physical states (current mainstream approach)
• High-level transition model: Answering questions like “If I accept this job, will I be happy?” (Sutton’s ideal)

5. The Bitter Lesson and Oak Lab

5.1 The Bitter Lesson

Sutton reiterated his famous article “The Bitter Lesson”: AI researchers always try to hardcode human knowledge into AI, but in the long run, this hinders progress.

History repeatedly proves this:

• 1990s: Rule-based expert systems → replaced by statistical methods
• 2010s: Hand-crafted features → replaced by end-to-end deep learning
• 2020s: Human-labeled fine-tuning → being replaced by RL self-exploration?

5.2 Oak Lab

Just one week before WAIC 2026, Sutton announced the founding of Oak Lab with University of Alberta researcher Khurram Javed. The company’s mission: break the current deep learning path and build AGI using entirely new principles — developing AI agents that can independently and continuously learn through first-person experience.

Sutton’s ultimate judgment: “AGI may still need 5, 10, or even 20 years. But we are moving in the right direction.”

6. Impact on the AI Industry

6.1 From Scaling Law to Learning Law

The industry’s heavy reliance on Scaling Law (performance improving with parameter count and data volume) is approaching its limits. The Experience Era’s core law will be Learning Law — agent performance depends on the quality and scale of its interaction with the environment.

6.2 Three-Layer Impact Structure

┌─────────────────────────────────────────────────┐
│              Layer 3: Application               │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐        │
│  │ Robotics │ │Autonomous│ │Industrial│        │
│  │     OS   │ │ Driving  │ │ Control  │        │
│  └──────────┘ └──────────┘ └──────────┘        │
├─────────────────────────────────────────────────┤
│            Layer 2: Infrastructure              │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐        │
│  │ Simulation│ │  Reward │ │Experience│        │
│  │ Platform │ │Engineering││  Replay  │        │
│  └──────────┘ └──────────┘ └──────────┘        │
├─────────────────────────────────────────────────┤
│              Layer 1: Theory                    │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐        │
│  │    OaK   │ │Comprehen.│ │Experience│        │
│  │Architect.│ │Mental Sci│ │ Learning │        │
│  └──────────┘ └──────────┘ └──────────┘        │
└─────────────────────────────────────────────────┘

Conclusion

Richard Sutton’s WAIC 2026 keynote is important not because it provides a definitive answer, but because it asks a profound question: Are we really building intelligence, or are we just building computation?

When the LLM industry is consumed by parameter competitions, Sutton reminds us to return to the essence of intelligence — the ability to achieve goals through behavioral adaptation. The OaK architecture, Comprehensive Mental Science, and Oak Lab are not endpoints but exploratory paths toward true intelligence.

As Sutton concluded: “We are experiencing this generation of AI for the first time, and we will collectively witness the arrival of the ‘Experience Era.’”

References:

1. Sutton WAIC 2026 Main Forum Keynote (July 17, 2026, Shanghai)
2. Sutton “Cultivating Superintelligence from Experience” Thinkers Forum (July 19, 2026)
3. Sutton, R.S. (2019). “The Bitter Lesson”
4. Sutton, R.S. & Barto, A.G. (2018). “Reinforcement Learning: An Introduction” (2nd ed.)
5. Oak Lab official announcement (July 2026)

• ←Previous
• Next→