ExpertsFM🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics)
Latent Space: The AI Engineer Podcast · September 23, 2026
5 Selected Lessons:
case · 2 framework · 1 heuristic · 1 reasoning_pattern · 1
Dual mandate for dual-use technology
Source
Latent Space: The AI Engineer Podcast — 🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics) (September 23, 2026) · at 0:00 ▸ · at 8:26 ▸ · at 1:11:07 ▸
When this applies
when your organization builds a capability that could be weaponized or misused by others
The takeaway
If you create a capability with dual-use risk, build the defensive/detection counterpart yourself because your technical expertise transfers directly and few others will have it.
Why it works
The same generative model that is good at creating something is structurally also the best-positioned model to discriminate/detect it, so the team building offense has a comparative advantage in building defense.
One level up
Responsibility for a capability's downside should sit with whoever has the technical leverage to address it fastest, not be outsourced to parties lacking that leverage.
Where it stops holding
This stops working when the defensive task requires fundamentally different data, skills, or incentives than the offensive capability, so the 'same model' advantage doesn't actually transfer.
Detail
procedure:
- Identify that your generative capability (design) and discriminative capability (detection) share the same underlying model class
- Make it a company mandate to build both sides, not just the exciting design side
- Direct the discriminative capability toward surveillance, attribution, and countermeasure tooling for the broader biodefense community
benchmarks: {}
rationale: A model good at generating sequences is also very good at discriminating whether a sequence is pathogenic, so the same team/model stack can serve both missions efficiently.
In their words
We felt it was important as a lab that a team that was both building the design capabilities is actually also best suited for building the defense capabilities because they're basically the same models.
Weigh benefits against harms rather than halting development out of fear
Source
Latent Space: The AI Engineer Podcast — 🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics) (September 23, 2026) · at 1:24:37 ▸
When this applies
when deciding whether to continue developing a powerful dual-use technology despite real risk
The takeaway
Decide whether to pursue a risky but high-upside technology by explicitly weighing benefit magnitude against harm probability and severity, rather than defaulting to caution or fear alone.
Why it works
Because unintentional misuse risk (accidental leakage, scientific curiosity gone wrong) is judged more probable near-term than deliberate state-actor weapon use, and because the technology's benefits (disease understanding, treatment, antimicrobial discovery) are large and tangible, a net cost-benefit calculation favors continued development paired with defensive investment.
One level up
Rational technology governance requires explicit cost-benefit reasoning under uncertainty, not just risk aversion or risk-seeking defaults.
Where it stops holding
This stops working when the harm side of the ledger includes low-probability but catastrophic, irreversible outcomes that dominate expected-value reasoning regardless of benefit size.
Detail
approach: Explicitly enumerate intentional (state actor) vs unintentional (accidental leakage) risk scenarios, assess relative likelihood, and weigh against concrete benefits (disease understanding, antimicrobial discovery, human health).
signals:
- state bioweapon programs exist but are more concerning in wartime
- unintentional leakage from legitimate research seems more likely near-term
- benefits to human health are large and already demonstrated
rationale: I think building this type of technology is ultimately a game that you weigh out the pros and cons and the costs and benefits. I think the benefits far outweigh the potential harm.
In their words
I think building this type of technology is ultimately a game that you weigh out the pros and cons and the costs and benefits. I think the benefits far outweigh the potential harm.
Recognize that AI agents acting autonomously need built-in safety checks before deployment, not after
Source
Latent Space: The AI Engineer Podcast — 🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics) (September 23, 2026) · at 1:16:37 ▸
When this applies
when a pipeline that used to be gated by slow, accountable human requesters becomes accessible to fast, high-volume autonomous agents
The takeaway
When autonomous agents can start requesting outputs from a previously human-gated pipeline at scale, build automated safety filters into that pipeline proactively, anticipating the volume increase, rather than relying on existing manual/rare-case assumptions.
Why it works
As the requester population shifts from a small number of accountable humans to a potentially unlimited number of autonomous agents, any manual or rare-use safety assumption embedded in the old pipeline breaks down, because volume and speed overwhelm it.
One level up
Any system designed around human-scale usage patterns must be re-audited for safety once autonomous agents can operate it at machine scale.
Where it stops holding
This stops working when the pipeline already has robust automated, AI-based safety filtering suited to the new scale, making this a non-issue.
Detail
rule: Anticipate scale/speed shifts (e.g., proliferation of autonomous agents) and proactively build safety filters for pipelines that currently assume slower, human-scale usage.
threshold: Presence of 'billions and trillions of agents' as the scale trigger that makes manual/rare-event safety assumptions untenable
rationale: It's wild that they don't have any tools to basically tell it if it's making anything dangerous or not.
In their words
It's wild that they don't have any tools to basically tell it if it's making anything dangerous or not. And so that was literally our first motivation of like, we should probably make something that can detect if something is dangerous or not.
Six months of rejected pitches before founding Evo
Source
Latent Space: The AI Engineer Podcast — 🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics) (September 23, 2026) · at 56:21 ▸ · at 59:38 ▸
When this applies
when pitching a capability-first innovation to domain experts who demand a use case upfront
The takeaway
When domain experts reject an idea for lacking an obvious use case, treat that as evidence of the idea's novelty rather than its invalidity, and test it empirically instead of only pitching it conceptually.
Why it works
Deep domain experts often judge new capabilities by whether a use case is already obvious, but transformative capabilities frequently precede their own use cases; persistence past near-universal rejection let the team discover utility empirically.
One level up
Breakthrough capabilities often can't be justified in advance — they must be built first to reveal their own applications.
Where it stops holding
This stops working when the underlying technical bet is actually unfounded and persistence simply wastes resources chasing a dead end.
Detail
situation: Eric spent six months pitching the idea of a generative DNA model to Stanford scientists, who called it a stupid idea, saying rules weren't understood and no use case existed.
decision: He proceeded anyway, training the model as an open-ended experiment with no clear use case defined in advance.
outcome: The first checkpoint was unexpectedly competitive with protein-specific benchmarks despite never being trained on protein labels, convincing ARC, NVIDIA, and OpenAI's Greg Brockman to back and accelerate the work, eventually leading to founding the company.
lesson: Persisting through near-universal expert skepticism and testing empirically can reveal capabilities and use cases that couldn't be argued into existence beforehand.
In their words
I heard all the reasons and I was just so stubborn about it. There's got to be a use case from being able to generate DNA. I just, it just feels right. And I didn't know what it was.
Evo generating a functional bacteriophage genome from scratch
Source
Latent Space: The AI Engineer Podcast — 🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics) (September 23, 2026) · at 8:26 ▸ · at 59:38 ▸
When this applies
when your generative AI capability crosses a threshold from assisting humans to fully autonomous creation of something previously only humans (partially) could do
The takeaway
A capability milestone (first fully novel generation of a whole complex system) is a credible signal of both the technology's potential value and its potential harm, and should trigger a parallel investment in safeguards, not just celebration.
Why it works
Because the model learned shared structure across DNA/RNA/protein from a single substrate, it could generate a complete, functional genome end-to-end rather than just copy-pasting known motifs as humans had done.
One level up
Treat capability inflection points as simultaneously opportunity signals and risk signals requiring immediate dual response.
Where it stops holding
This stops working when the capability milestone is overstated or trivial (e.g., merely recombining near-identical natural sequences), so the implied risk/opportunity signal is weaker than it appears.
Detail
situation: Evo was used to generate a complete functional bacteriophage genome from scratch, something humans had never done (only copy-paste/edit of existing genomes was previously possible).
decision: The team treated this as both a landmark achievement and a wake-up call, publicly highlighting the dual-use implications and increasing investment in defensive/biosecurity capabilities.
outcome: Got significant feedback split between excitement and concern; some critics argued the output was close to recapitulating nature rather than a truly novel design, but it catalyzed the company's dual mandate strategy.
lesson: A capability crossing a new threshold (design vs. edit) is the moment to invest in defense, even amid valid critiques about how novel the achievement really is.
In their words
This was a key turning point for, I think, the scientific community and for us as a company at Radical Numerics, because we felt this was such an indicative manifestation of the potential to create a whole organism, not existing in nature, but also the potential harm that that means as well.