ExpertsFMEpisode 8 - OpenAI x Broadcom and the future of compute
OpenAI Podcast · October 13, 2025
5 Selected Lessons:
heuristic · 3 framework · 1 priority · 1
Infrastructure build value lies in what it enables, not the raw resource itself
Source
OpenAI Podcast — Episode 8 - OpenAI x Broadcom and the future of compute (October 13, 2025) · at 9:41 ▸
When this applies
when evaluating any large infrastructure investment (roads, rail, fiber, chips)
The takeaway
The value of infrastructure should be measured by the downstream capabilities and economic activity it enables, not by the physical/technical commodity it is built from.
Why it works
Historical infrastructure projects (highways, railroads) are retrospectively misjudged if evaluated by their literal material (asphalt, steel) rather than by the economic activity and new possibilities they unlocked.
One level up
Always evaluate an investment's strategic significance one level removed from its literal substrate — in terms of what becomes possible, not what it's made of.
Where it stops holding
This stops working when the infrastructure fails to catalyze genuinely new downstream use cases, in which case it really is just a commodity investment.
Detail
hierarchy:
- What becomes possible on top of the infrastructure
- The infrastructure/network itself
- The raw material/commodity (chips, asphalt, steel)
rationale: People get fixated on the chips themselves, but like highways weren't about asphalt, this buildout isn't really about chips — it's about what new capabilities and economic activity it enables.
In their words
It's kind of like thinking the National Highway Project was about selling asphalt or railroads are about steel. In reality, it's the things become possible on top of that.
Full-stack vertical integration for compute efficiency
Source
OpenAI Podcast — Episode 8 - OpenAI x Broadcom and the future of compute (October 13, 2025) · at 3:28 ▸ · at 8:10 ▸ · at 13:13 ▸ · at 15:22 ▸
When this applies
when a company's product depends on a performance-critical resource that generic suppliers can't fully optimize for its specific use case
The takeaway
When a resource becomes mission-critical and generic, cross-layer optimization of the full stack unlocks gains that optimizing individual layers in isolation cannot.
Why it works
Owning the design from the transistor to the end-user output lets you optimize interactions between layers (chip, rack, networking, algorithms) that siloed vendors can't jointly tune, producing efficiency gains unavailable through piecemeal sourcing.
One level up
Control the parts of your value chain that determine your strategic destiny, even if it requires entering domains outside your original core competency.
Where it stops holding
This stops working when the workload is too generic or too small-scale to justify the fixed cost of custom full-stack design, or when you lack a partner with deep process expertise.
Detail
procedure:
- Identify a workload that existing generic hardware serves poorly or inefficiently.
- Partner with a company that has deep semiconductor/systems expertise you lack.
- Co-design across every layer: transistor, chip, rack, networking, algorithms, end product.
- Measure gains in efficiency (performance per watt, cost per token) rather than just raw speed.
benchmarks: {}
rationale: Optimizing across the entire stack yields efficiency gains that lead to faster, cheaper, smarter models.
In their words
We are able to think from like etching the transistors all the way up to the token that comes out when you ask Chachi PDT a question and design the whole system.
Efficiency gains create more demand than they satisfy
Source
OpenAI Podcast — Episode 8 - OpenAI x Broadcom and the future of compute (October 13, 2025) · at 3:28 ▸ · at 18:49 ▸ · at 21:48 ▸
When this applies
when you make any product/service cheaper or more capable
The takeaway
A 10x efficiency improvement in a bottlenecked resource routinely attracts more than 10x in new demand, because cheaper/better capability unlocks previously unviable use cases.
Why it works
Lowering cost and improving quality reveals latent demand that was previously suppressed by price or capability constraints; users find new use cases once the barrier drops.
One level up
Never assume that solving a scarcity problem will create slack — growth in capability tends to be met by proportionally larger growth in appetite.
Where it stops holding
This stops working when the market for the use case is already saturated or when efficiency gains don't translate to accessible pricing or usability improvements for end users.
Detail
rule: Expect demand to grow faster than the efficiency gain that enabled it.
threshold: optimize by 10x -> expect ~20x more demand (as stated)
rationale: Better, cheaper, faster intelligence causes people to find more new things to do with it, consistently outpacing supply.
In their words
So we used to think like, oh, we'll optimize things by 10x and we'll solve all of our problems, but you optimize by 10x and there's 20x more demand.
Match chip architecture to workload type (training vs inference)
Source
OpenAI Podcast — Episode 8 - OpenAI x Broadcom and the future of compute (October 13, 2025) · at 13:13 ▸
When this applies
when allocating specialized resources, match the resource's design to the specific phase or task it serves rather than using a one-size-fits-all solution
The takeaway
Different phases of a process often have fundamentally different resource bottlenecks, so the optimal tool for one phase is suboptimal for another.
Why it works
Training workloads are compute- and network-bound (need high TFLOPs and interconnect), while inference workloads are memory-bound (need high memory and memory bandwidth relative to compute); designing a chip that conflates both wastes resources on the dimension not needed.
One level up
Decompose a pipeline into its distinct phases before optimizing resource allocation, rather than applying a single uniform solution across the whole pipeline.
Where it stops holding
This stops working when workload phases are too intertwined or too small in scale to justify separate specialized designs, making a general-purpose solution more cost-effective.
Detail
rule: Design training chips for high compute (TFLOPs) and strong networking; design inference chips for high memory and memory access relative to compute.
threshold: n/a - qualitative distinction stated directly
rationale: Over time this creates a platform of workload-optimized chips rather than one generic chip, maximizing effectiveness of the overall system.
In their words
For instance, if you want to train, you design chips that are much stronger in computing capacity measured TFLOPs as well as network. Because it's not just one chip that makes it happen. It's a cluster, as Charlie put it. But if you want to do inference, you put in more memory and memory access relative to compute.
Compute allocation scarcity directly throttles team output
Source
OpenAI Podcast — Episode 8 - OpenAI x Broadcom and the future of compute (October 13, 2025) · at 26:13 ▸
When this applies
when any team's output is gated by allocation of a scarce shared resource (budget, headcount, compute, shelf space)
The takeaway
Observe the intensity of internal competition for a resource as a diagnostic for how binding a constraint it is on organizational output.
Why it works
When a resource is the binding constraint on output, the intensity of internal competition for its allocation becomes a direct signal of how scarce and valuable that resource is, and output scales almost linearly with the allocation received.
One level up
Resource scarcity inside an organization is itself diagnostic data about where the true bottleneck to growth lies.
Where it stops holding
This stops working when internal competition for a resource reflects political dynamics rather than genuine output constraints, making the signal noisy.
Detail
rule: Treat intense internal competition over a resource as evidence that the resource, not something else, is the binding constraint on growth.
threshold: n/a - qualitative
rationale: Teams' output being a direct function of compute allocation, with extreme intensity around who gets allocation, confirms compute is OpenAI's binding constraint, motivating the drive toward 'compute abundance.'
In their words
Like teams within OpenAI, that their output is just like a direct function of how much compute they get. And so that the amount of intensity on the like who gets the compute allocation is so extreme.