ExpertsFMWhat happens now that AI is good at math? - Episode 17
OpenAI Podcast · April 28, 2026
5 Selected Lessons:
reasoning_pattern · 2 case · 1 heuristic · 1 framework · 1
AI-agent-assisted verification with human accountability
Source
OpenAI Podcast — What happens now that AI is good at math? - Episode 17 (April 28, 2026) · at 38:14 ▸ · at 40:15 ▸
When this applies
when a surge of AI-generated submissions (code, documents, claims) threatens to overwhelm a human review/verification bottleneck
The takeaway
Use AI to triage/flag likely errors so humans can focus limited verification effort, but keep a human accountable for final acceptance to preserve incentive-aligned quality control.
Why it works
A second AI agent can pre-screen large volumes of content for likely errors, narrowing what a human reviewer must deeply check, while keeping the human as the accountable decision-maker who stakes their reputation on acceptance.
One level up
Scale verification capacity with automated triage, but keep accountability anchored to a human stakeholder to preserve trust.
Where it stops holding
This stops working when the triage AI itself is unreliable at flagging the right issues, or when there is no clear human stakeholder willing to stake reputation on the final call.
Detail
procedure:
- Deploy AI agents to scan submissions (proofs, code commits) and flag specific potential issues
- Present flagged issues to human reviewers to narrow their verification workload
- Require the human committing/publishing to take explicit reputational responsibility for the final result
- Strengthen cultural norms (as already exist in math) that incorrect published work damages the publisher's reputation
rationale: Full trust in AI-only verification is premature, but AI triage plus human-anchored accountability scales review without abandoning quality control.
In their words
we can have the AI agent flagging specific potential issues. So kind of bringing to the front, okay, hey, maybe this part, I'm not totally sure about it. So that will accelerate, will help the human to have less to verify, basically.
Test the claim yourself rather than trusting debate or social consensus
Source
OpenAI Podcast — What happens now that AI is good at math? - Episode 17 (April 28, 2026) · at 2:51 ▸ · at 5:46 ▸
When this applies
when there's noisy public disagreement about whether a new tool/technology can do X in your field
The takeaway
When capability claims are contested, run your own controlled test on a problem you can personally verify rather than deferring to crowd opinion.
Why it works
Direct personal experimentation on a problem you deeply understand gives higher-fidelity evidence than secondhand debate, because you can both generate and verify the result.
One level up
Prefer direct, verifiable experience over aggregated opinion when evaluating uncertain new capabilities.
Where it stops holding
This stops working when you lack the expertise to properly verify the result yourself, in which case your personal test is no more reliable than the crowd's guess.
Detail
approach: Pick a problem you deeply understand and personally verify, then test the tool on it directly instead of relying on others' claims.
signals:
- public debate with no consensus
- you have deep domain expertise to verify results
- you can design your own test case
rationale: Removes reliance on secondhand reporting and social media hype/skepticism.
In their words
So then I got curious and a lot of people got curious, can ChatGPT do research level mathematics? And there was a lot of debate online, and then I thought to myself, should try it on my own problems.
Debate over recombination vs. 'sparks of insight' as the mechanism of discovery
Source
OpenAI Podcast — What happens now that AI is good at math? - Episode 17 (April 28, 2026) · at 18:33 ▸
When this applies
when evaluating whether an automated or systematic process can substitute for what looks like genius-level insight in any creative/intellectual field
The takeaway
Before concluding a domain requires unreplicable human genius, consider whether the apparent 'spark' might itself be recombination at a scale/speed humans can't match.
Why it works
If most 'breakthroughs' are actually recombination of existing pieces plus modest additional reasoning, then a system capable of vast, fast recombination could match a large fraction of human progress, with the open question being whether some irreducible 'spark' remains beyond reach.
One level up
Question whether perceived 'genius' moments are qualitatively different from systematic recombination, or just rare, high-quality instances of it.
Where it stops holding
This stops working if it turns out some discoveries genuinely require a non-recombinatory leap that no amount of literature search and reasoning can reach — a question the speaker says is still open.
Detail
approach: Examine whether apparent breakthroughs (even famous ones like relativity) are decomposable into recombination + reasoning, to judge if AI's recombination-based approach can scale to real discovery.
signals:
- a canonical 'genius' example is invoked (e.g., Einstein)
- uncertainty about whether that example truly required non-recombinatory insight
rationale: The jury is still out on whether this process of just recombination and a little bit of thinking can increase human knowledge with no limit, or whether sparks of genius are needed.
In their words
there is this question of whether the progress that we're seeing in science, is it just putting together different pieces and, you know, doing a little bit of reasoning on top of it? Or are there those brilliant sparks of insight?
Expertise becomes more valuable, not less, as AI gets powerful
Source
OpenAI Podcast — What happens now that AI is good at math? - Episode 17 (April 28, 2026) · at 35:59 ▸
When this applies
when deciding whether domain mastery still matters once an AI tool that 'does the work' is available
The takeaway
The leverage a tool provides scales with the user's underlying expertise; powerful general tools amplify expert judgment but do not substitute for its absence.
Why it works
Extracting genuinely correct, valuable output from a powerful but fallible AI requires the human to catch errors and steer toward productive directions, which is only possible with deep prior training; without that, errors compound undetected.
One level up
Tools that remove execution friction raise, rather than lower, the ceiling value of expertise, because expertise becomes the limiting factor on correct use.
Where it stops holding
This stops working for tasks with trivial, low-stakes correctness requirements where even an unverified AI answer causes no harm.
Detail
rule: The ability to get real value out of a capable AI tool is gated by the user's own domain expertise.
threshold: Contrast: experts (Sebastian, Ernest) produced verified new results; non-mathematicians produced tens of pages of proof that 'turn out to be just wrong.'
rationale: The AI's output quality is not self-certifying; only an expert can reliably catch its mistakes and productively redirect it.
In their words
I think it's really important for the audience, for everyone listening to us, to understand that expertise is even more valuable than it ever was.
Solving a 42-year-old optimization open problem via iterative ChatGPT verification
Source
OpenAI Podcast — What happens now that AI is good at math? - Episode 17 (April 28, 2026) · at 2:51 ▸ · at 20:38 ▸
When this applies
when an expert wants to use an AI tool to crack a hard, previously-unsolved problem in their domain
The takeaway
An expert who acts as the verifier/director of an AI's output, rather than a passive consumer of it, can reach results far faster than either could alone.
Why it works
The expert supplied domain judgment (spotting errors, redirecting toward promising approaches) while the model supplied rapid generation and iteration, compressing a process that would otherwise take weeks into hours by combining human verification with machine speed.
One level up
Augment, don't replace: pair deep domain expertise with the AI's speed, with the human retaining the verification role.
Where it stops holding
This stops working when the person interacting with the AI lacks enough domain expertise to catch the model's mistakes, since a single uncaught error can invalidate the whole chain of reasoning.
Detail
situation: A 42-year-old open question in optimization theory (does Nesterov's method ever diverge) had resisted resolution.
decision: Ernest spent ~12 hours over 3 evenings interacting with ChatGPT, correcting its mistakes and steering it toward novel approaches rather than just prompting for an answer.
outcome: A correct proof was produced and double-checked by both Ernest and ChatGPT, resolving the open problem — a task Ernest estimates would have taken ~40+ hours over a month without AI.
lesson: Expert-guided iterative use of AI, with the human as verifier/director, can compress research timelines dramatically.
In their words
I played the role of the verifier. I told whenever the model made a mistake, I corrected it. I also tried to point the conversation into approaches that I felt were novel.