ExpertsFMBeyond Bigger Models: Recursion As The Next Scaling Law In AI
Y Combinator Startup Podcast · May 1, 2026
5 Selected Lessons:
heuristic · 3 reasoning_pattern · 1 framework · 1
Three-level recursion with stop-gradient (fixed-point) training to sidestep backprop-through-time
Source
Y Combinator Startup Podcast — Beyond Bigger Models: Recursion As The Next Scaling Law In AI (May 1, 2026) · at 8:31 ▸ · at 10:32 ▸ · at 27:53 ▸
When this applies
when any iterative process needs to be trained/optimized end-to-end, but full backpropagation through every iteration is computationally or numerically infeasible
The takeaway
When full differentiation through a long process is intractable, approximate optimization by truncating the backward pass to a short window while still iterating forward across the full process and varying the state at which you measure the signal.
Why it works
By running many recursive steps with gradients detached (no_grad), then updating weights using only a frozen glimpse of the final state before continuing on an un-reset carry/hidden state, the model gets exposure to many different points in latent memory space without ever needing to store or backprop through the full computational graph.
One level up
You don't need to optimize the full causal chain if you can get a representative, varying sample of its intermediate states and optimize locally against them repeatedly.
Where it stops holding
This stops working when the truncated gradient signal doesn't actually correlate well with long-horizon performance, or (per S6) when the theoretical justification (fixed-point convergence) doesn't actually hold but the trick still happens to work for unexplained reasons.
Detail
procedure:
- Initialize latent carry states ZL and ZH at zero.
- Run TL steps of the low-level module without gradient tracking.
- Run TH steps alternating low/high-level modules without gradient tracking (two nested loops).
- Detach/stop-grad before the final step.
- Run one more pass through both modules with gradients enabled and backprop only through that final step.
- Do NOT reset ZL/ZH between outer refinement iterations — carry them forward so each 'batch' samples a different point in latent memory space.
- Repeat the whole cycle NSUP times during training (outer refinement loop).
benchmarks: {"model_size":"27 million parameters","result":"state of the art on ArcPrize 1 and 2, ~70% on ArcPrize 1 vs 0% for o3 at the time","training_data":"only ~1000 puzzle tasks, no pretraining"}
rationale: Avoids the memory/compute explosion of full backprop-through-time while still letting the model learn useful recursive computation, achieving strong results from a tiny model with no pretraining.
In their words
what they do instead is they actually do that 16 times. And so, and as you do that, you actually can see the change in your residuals get less and less and less.
Diagnose the structural tradeoff behind a capability gap
Source
Y Combinator Startup Podcast — Beyond Bigger Models: Recursion As The Next Scaling Law In AI (May 1, 2026) · at 0:00 ▸ · at 2:29 ▸ · at 5:29 ▸
When this applies
when a system that is fast and scalable systematically underperforms on a class of problems — look for what structural capability it traded away to gain that speed
The takeaway
When a design choice optimizes for one dimension (speed, scale, stability), identify the capability it sacrificed, because that sacrifice becomes the bottleneck for a specific class of problems.
Why it works
Transformers gained massively parallel, stable training by giving up a compressed iterative latent state; that missing 'memory tape' is exactly what's needed for problems with known lower bounds like n log n sorting.
One level up
Every architectural or process shortcut has a hidden tax; find out what it is before declaring the shortcut sufficient.
Where it stops holding
This stops working when the problem doesn't actually require the sacrificed capability, so the tradeoff turns out to be irrelevant.
Detail
approach: Identify what a design optimizes for, then ask what was given up, then map that loss onto specific problem classes with known theoretical requirements (e.g., comparison sort lower bounds).
signals:
- a system hits a hard wall on a narrow task class despite general competence
- there's a known theoretical lower bound (e.g., n log n) that the system's architecture can't satisfy structurally
rationale: Mapping failure modes to structural causes (lack of memory/iteration) rather than treating them as mysterious lets you predict which future tasks will also fail.
In their words
And what you actually paid for that you have to give up is this latent reasoning thing and this compression in the time direction.
Use bioplausibility as inspiration, not as a design constraint
Source
Y Combinator Startup Podcast — Beyond Bigger Models: Recursion As The Next Scaling Law In AI (May 1, 2026) · at 13:53 ▸
When this applies
when borrowing a design pattern from nature or an analogous real-world system to inspire a solution
The takeaway
Treat analogies from nature/other domains as a generator of ideas, not as a binding requirement; be willing to discard the fidelity to the source once you find what actually performs on your real constraints.
Why it works
Bio-inspired ideas spark novel hypotheses, but the biological constraint itself (e.g., refractory periods in AlexNet) often isn't what makes the solution work on the actual substrate (GPUs); stripping the constraint and going 'bio-implausible' frequently outperforms it.
One level up
The origin story of an idea and its justification for continuing to use it are often different things — don't confuse inspiration with validation.
Where it stops holding
This stops working when the bio-inspired mechanism is itself the actual causal driver of performance, not just a launching metaphor.
Detail
rule: Use biological/natural analogies to generate hypotheses, but validate and optimize for what actually works on your engineering substrate, even if it diverges from the biological inspiration.
threshold: If removing the bio-plausible component and replacing it with something 'implausible' but GPU/substrate-friendly improves results (as with AlexNet's inhibition removed in VGG), drop the bio constraint.
rationale: Bioplausibility is a fertile source of ideas but an unreliable guide to what will actually perform best in a different substrate (silicon vs biology).
In their words
The thing about them is that often we use bioplausibility to inspire us to come up with ideas, but we end up veering away from the bio-plausible to something adjacent to them that is likely bio-implausible, but that seems to work better.
Strip the first paper down to its core mechanism (ablate aggressively)
Source
Y Combinator Startup Podcast — Beyond Bigger Models: Recursion As The Next Scaling Law In AI (May 1, 2026) · at 19:01 ▸ · at 22:01 ▸ · at 27:53 ▸ · at 31:25 ▸
When this applies
when a successful complex system bundles many changes at once — run systematic ablations to find the 20% that drives 80% of the result, then discard the rest
The takeaway
After a breakthrough combines many novel components, systematically ablate each one to find which single mechanism is actually doing the work, then rebuild a simpler system around just that mechanism.
Why it works
Konstantin's scaling ablations isolated the outer refinement loop as the dominant driver of HRM's performance; once identified, everything else (dual networks, extra transformer depth, high test-time recursion) could be stripped away without losing — and even gaining — performance.
One level up
Complexity accrued during initial discovery is often scaffolding, not substance — the real lesson is usually smaller and more generalizable than the first working prototype.
Where it stops holding
This stops working when the components are genuinely interdependent (removing one breaks the others non-additively), so isolating a single 'main' mechanism via ablation gives a misleading picture.
Detail
rule: Run per-component scaling ablations on a complex working system to find the dominant mechanism, then build the simplest possible version around it.
threshold: If removing a component (e.g., test-time recursion depth above 1) only costs ~1/8 of performance per Konstantin's findings, it's a candidate for deletion.
rationale: Simpler systems built around the true driver of performance tend to be both cheaper and, paradoxically, sometimes better (TRM beat HRM with 4x fewer parameters).
In their words
Okay, so like a lot of machine learning, the follow-on paper is basically delete 75% of the first paper, as we've often done in videos here, and keep the magic, basically.
Don't assume a more powerful component (e.g., attention/transformer) is always better — test simpler alternatives per task
Source
Y Combinator Startup Podcast — Beyond Bigger Models: Recursion As The Next Scaling Law In AI (May 1, 2026) · at 31:25 ▸
When this applies
when selecting a 'more sophisticated' component for a subsystem, check whether a simpler, cheaper component performs as well or better on the specific task at hand before defaulting to the fancier option
The takeaway
The most capable general-purpose tool is not always the best choice for a specific sub-task; test cheaper/simpler alternatives against the task's actual structure.
Why it works
On Sudoku, a plain MLP outperformed the attention-based transformer layer, while on the maze task the MLP scored zero and the transformer was necessary — showing the 'better' component is task-dependent, not universal.
One level up
Generality and task-fit are different axes — optimize for fit, not just capability.
Where it stops holding
This stops working when tasks are heterogeneous or unknown in advance, making per-task component selection impractical, and a general-purpose component becomes the safer default despite being suboptimal on any single task.
Detail
rule: Test simple vs complex components per-task rather than assuming the more sophisticated one (e.g., attention) always wins.
threshold: MLP outperformed attention on Sudoku but scored zero on maze; transformer was needed for maze.
rationale: Task structure (e.g., Sudoku's local constraint-satisfaction nature vs maze's spatial/sequential nature) determines which inductive bias wins, so it's not obvious a priori.
In their words
Yeah, Sudoku, MLP actually outperformed the attention. It scored zero on the maze. The MLP scored zero on the maze. And so it's not clear. It's not obvious that the transformer is always better.