ExpertsFMDavid Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience
Training Data · October 13, 2026
5 Selected Lessons:
heuristic · 2 case · 1 priority · 1 reasoning_pattern · 1
Bet against human priors to maximize depth and breadth of discovery
Source
Training Data — David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience (October 13, 2026) · at 24:21 ▸ · at 27:28 ▸ · at 32:13 ▸
When this applies
when deciding whether to build on an existing large, well-trodden knowledge base versus a from-scratch, self-discovering approach
The takeaway
Rank long-run potential for novel discovery above near-term performance when the dominant existing approach is built on a specific legacy knowledge base.
Why it works
Leaning on an existing large human/legacy dataset caps the system's ceiling at (and biases it toward) that dataset's embedded assumptions and blind spots, and makes the system a poor fit for domains where that dataset doesn't apply; building without it costs short-term performance but removes the ceiling.
Where it stops holding
This stops working when near-term performance/competitiveness is the actual goal, or when the domain genuinely requires the legacy knowledge base to be useful at all.
Detail
hierarchy:
- Ability to discover knowledge far beyond human knowledge (depth)
- Ability to operate in domains where human data doesn't apply (breadth)
- Near-term performance/competitiveness using human data
rationale: Human data embeds era-specific biases (e.g., flat earth, four humours) that are hard to remove and cap the system's ultimate potential; dropping it sacrifices short-term capability for long-term ceiling.
In their words
the more we lean on human data, the less able we are to actually discover the algorithms that are really going to take us all the way to something that's going to, for example, discover a whole new field of knowledge.
Build/debug in simulation before committing to the real world
Source
Training Data — David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience (October 13, 2026) · at 38:14 ▸ · at 41:28 ▸ · at 51:03 ▸
When this applies
when deciding where to first develop and validate a risky, powerful, or expensive system before deploying it in a high-stakes real environment
The takeaway
When a target environment is complex, expensive, or risky, first build and validate the capability in a parallelizable, reproducible, contained proxy environment, then selectively transfer and re-validate against increasingly high-fidelity versions before full real-world deployment.
Why it works
Simulation allows massive parallelization, perfect reproducibility, and containment of failure modes, which accelerates iteration speed (10-100x per Silver) and reduces real-world risk, even though results may not transfer 1:1 to reality (per Sutton's point, which Silver partly accepts).
One level up
Separate 'discovering the underlying science/algorithm' from 'deploying it in the costly target environment' — do the former fast and cheap, the latter carefully and slowly.
Where it stops holding
This stops working when the target domain contains properties that cannot be represented in any feasible simulation (e.g., true quantum computation, per Silver) or when sim-to-real transfer gaps are too large to bridge even with science learned in simulation (Sutton's concern).
Detail
approach: Ask first whether the target capability is theoretically expressible in simulation (e.g., via Turing-completeness for computation, a medium of exchange for economics, a communication channel for language); if yes, build and iterate there because it's faster and safer; then use the discovered algorithm/science to guide careful real-world deployment, raising simulation fidelity incrementally.
signals:
- domain concept is formally expressible in simulation (computation, economics, language, math)
- some phenomena (e.g. quantum computation) may only be expressible in the real world
rationale: Faster, safer iteration in simulation plus gated, evidence-based progression to real world balances speed with risk management.
In their words
So if it is possible for it to be built in simulation, I believe it will happen 10 times or 100 times faster. Just because simulation can be infinitely paralyzed, you get perfect reproducibility, has a lot of really nice properties. It's also the safest place for a superintelligence to be built.
Timing determines whether an ambitious bet succeeds
Source
Training Data — David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience (October 13, 2026) · at 32:13 ▸ · at 36:36 ▸
When this applies
when evaluating whether now is the right moment to launch an ambitious venture or research push
The takeaway
Judge the viability of an ambitious bet primarily by whether the surrounding technology/infrastructure has just reached the threshold needed, not solely by the quality of the idea.
Why it works
A venture too early lacks supporting infrastructure (compute, tooling, scaling understanding) to succeed even with the right idea; a venture too late faces saturated competition with no differentiated upside — success requires the convergence of idea and infrastructure at the same moment.
One level up
Great scientific/business moments are won on timing, not just on correctness of vision.
Where it stops holding
This stops working when the bet's success doesn't actually depend on external infrastructure thresholds, or timing predictions are based on insufficient tracking of the relevant trend lines.
Detail
rule: Whether a hard research bet succeeds or fails is fundamentally a timing question — too early means insufficient infrastructure, too late means no differentiation.
threshold: Evidence cited: correctly predicted Computer Go's solve date 10 years ahead 'to the year', and correctly predicted StarCraft and IMO (AlphaProof) milestones.
rationale: Enormous recent progress in compute, scaling-law understanding, and AI-accelerated research makes 'now' the right time for the super-learner bet, per Silver.
In their words
whether this succeeds or fails, I think is about timing. Like when you create a startup, you want to be at the right time. You don't want to be too early or too late.
Match compute architecture to actual workload type
Source
Training Data — David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience (October 13, 2026) · at 53:04 ▸ · at 57:57 ▸
When this applies
when provisioning infrastructure/resources, recognizing that different workload types (fast-iteration vs. heavy-compute) need different tooling, not a one-size-fits-all solution
The takeaway
When one resource (e.g., GPUs) doesn't natively support all your workload types equally well, use algorithmic/architectural tricks to adapt the workload to the resource rather than reverting to a non-scalable legacy resource mix.
Why it works
GPUs have been optimized over years mainly for training (and lately inference), not for simulation; using the wrong-shaped resource for a workload (e.g., relying on bundled CPUs for simulation as done years ago at Google) doesn't scale, so workload-specific solutions (model-based RL, replay buffers, data reuse) are needed to keep the dominant resource (GPUs) well-utilized across all workload types.
Where it stops holding
This stops working when no algorithmic trick can close the gap between workload needs and available hardware, forcing genuine reliance on a different, less scalable resource.
Detail
rule: Avoid falling back to non-scalable legacy infrastructure (e.g., bundling millions of CPUs) for workloads hardware isn't optimized for; instead adapt algorithms (model-based RL, replay buffers, data reuse) to make the dominant scalable resource (GPUs) work for all workload types.
rationale: LLM companies now face the CPU-sandbox scaling problem Ineffable's team hit years ago at Google; avoiding it by design keeps the whole stack GPU-native and scalable.
threshold: All training, inference, and simulation kept on GPU rather than requiring millions of separate CPUs.
In their words
I think like some of the things we did like eight, 10 years ago at Google, where we basically had like a bundle CPUs at Google that doesn't scale anymore. So we really need to make sure that everything we do is on the GPUs and we don't need like millions of CPUs beyond these GPUs.
Scale-then-fine-tune surprise
Source
Training Data — David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience (October 13, 2026) · at 0:00 ▸
When this applies
when you assume behavior change requires large amounts of new data or time
The takeaway
Once a system (or team, or market) has absorbed enough foundational experience, subsequent adaptation to new specifics can be surprisingly fast and cheap.
Why it works
A system trained at sufficient scale builds general representations that can be redirected with very little additional signal, because the heavy lifting of representation-building is already done.
One level up
Invest in scale/foundation first; the speed of later pivots is a function of prior depth, not proportional effort.
Where it stops holding
This stops working when the initial scale/foundation hasn't actually been reached — insufficient base training won't yield fast fine-tuning, it just yields a weak system.
Detail
situation: Silver expected that changing a large trained system's behavior would require large amounts of new data or be slow.
decision: He observed that after sufficient scale was reached, very small amounts of new data could rapidly and completely change the system's behavior.
outcome: This reshaped his mindset: 'at the right scale' hard/slow problems become fast once enough has been learned.
lesson: Foundational scale converts future adaptation from slow to fast — plan investment accordingly.
In their words
it was definitely a surprise to me how a system that was trained on a massive amount of data could then be very quickly fine-tuned on a very small amount of data to kind of completely change its behavior