ExpertsFMGoogle's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI
Training Data · October 6, 2026
5 Selected Lessons:
heuristic · 3 priority · 1 framework · 1
Model-side gains outweigh hardware gains for intelligence-per-watt
Source
Training Data — Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI (October 6, 2026) · at 14:13 ▸
When this applies
when deciding where to invest to improve a system's efficiency per unit of scarce resource
The takeaway
Treat the hardware improvement curve as a baseline multiplier, but prioritize software/model-level optimization as the primary lever for efficiency gains.
Why it works
Hardware delivers a reliable ~2x/year multiplier that lifts everyone, but most realized intelligence-per-watt gains come from model-side optimizations and software/system work that ensures hardware is used effectively.
One level up
The layer that controls how a resource is used often contributes more value than the layer that merely supplies the resource.
Where it stops holding
This stops working when hardware itself is the binding constraint (e.g., physical limits reached) or when software/model optimization has been exhausted.
Detail
hierarchy:
- Model-side improvements (largest share)
- Software/system optimizations (ensuring hardware used effectively)
- Hardware improvements (~2x/year, a 'free multiplier' for everyone above)
rationale: Hardware is necessary but model and software layers determine how much of that capability actually gets converted into delivered intelligence.
In their words
I don't have the exact breakdown, but I would say that in our experience, most of the benefits come from model side improvements in terms of intelligence per watt.
Co-design partnership across organizational and hardware-lifecycle stages
Source
Training Data — Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI (October 6, 2026) · at 26:12 ▸ · at 30:36 ▸
When this applies
when two interdependent teams (e.g. hardware and software, or any upstream/downstream functions) must coordinate on a long-lead asset while one side iterates much faster
The takeaway
Match the depth of cross-team intervention to the asset's stage in its lifecycle, enabling tight coupling where changes are still cheap and looser coupling where they're expensive.
Why it works
Chips exist in production, bring-up, pre-tape-out, design, and concept stages simultaneously; the stage determines how much a late-breaking model insight can still change the hardware — late stages allow full disruption, pre-taped-out stages allow negotiated partial changes, and in-production chips allow only software-level adaptation.
One level up
Co-location and daily interaction between fast-moving and slow-moving functions let the organization capture more of the mechanism's value than contractual/arms-length collaboration can.
Where it stops holding
This stops working when teams are organizationally or physically separated (e.g., across company boundaries), making it 'much harder' to intercept and adjust assets in flight.
Detail
procedure:
- Classify every hardware project by lifecycle stage: in production, in bring-up/debug, pre-tape-out, in design, in concept.
- For in-production/near-tape-out chips, allow negotiated compromises (e.g., 90% of desired benefit) rather than full redesign.
- For in-design/concept-stage chips, run deep joint simulation of future model architectures against hardware Pareto curves.
- Co-locate researchers and hardware engineers physically and have leadership interact multiple times per week.
rationale: Enables intelligence delivered/good put per watt to be maximized across the full stack because both sides understand what's happening end-to-end.
In their words
We also get to collaborate deeply on though the chips that are actually just about to tape out. Why? Because we can intercept and we can make changes to the chip, literally the chip architecture in flight, which would be somewhere between hard and impossible to do if we were working across company boundaries.
Failure rate scales predictably with accelerator count
Source
Training Data — Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI (October 6, 2026) · at 5:31 ▸ · at 8:46 ▸
When this applies
when operating any system composed of tens of thousands of tightly-coupled, synchronous components
The takeaway
Design for continuous failure detection and near-real-time recovery as a baseline operating assumption once component count crosses tens of thousands in a synchronous system.
Why it works
Each component, however reliable individually, has nonzero failure probability; at massive scale the aggregate failure rate becomes near-continuous, especially when the system requires microsecond/millisecond synchronous coordination so one failure can halt everything.
One level up
At sufficient scale, rare events become routine operating conditions, not exceptions.
Where it stops holding
This stops working when components operate asynchronously/independently, where individual failures don't cascade or require coordinated recovery.
Detail
rule: At 100,000-accelerator scale, expect failures multiple times a day to multiple times an hour, with no single dominant root cause (long tail of hardware/software/network issues).
threshold: 100,000 accelerators -> multiple failures/day or /hour
rationale: Many interdependent, bleeding-edge components (chiplets, HBM, optics, network) each have failure modes; at scale, something is always failing.
In their words
I think that it is definitely going to be at the scale multiple times a day. And perhaps, depending on the exact configuration, multiple times an hour, something is going to fail.
Calibrate ask size to partner's cycle time
Source
Training Data — Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI (October 6, 2026) · at 30:36 ▸
When this applies
when a fast-iterating team wants to request changes from a slow-iterating partner (engineering, manufacturing, regulatory, etc.)
The takeaway
Only escalate a change request to a slower-moving partner when the expected payoff clearly exceeds that partner's cost of change; let that asymmetry govern what gets proposed.
Why it works
Researchers who understand that hardware changes carry much higher cost/friction than software changes self-select which requests are worth escalating, preventing wasted disruption on marginal gains.
One level up
Shared understanding of a partner's cost structure, cultivated over time, substitutes for formal gatekeeping and improves overall throughput.
Where it stops holding
This stops working when the requesting team lacks visibility into the partner's true cost of change, leading to either over-asking or under-asking.
Detail
rule: Researchers know that a 0.5-1% gain isn't worth asking to stop a tape-out, but a big gain is worth the disruption.
threshold: roughly 'small tweak ~0.5-1%' = don't ask; 'really good opportunity' = escalate
rationale: This shared calibration emerged from years of researchers and hardware engineers working in overlapping rooms.
In their words
they also know that, look, if they have a small tweak that's going to deliver like 1% or 0.5% or something like that for a chip that's about to tape out, they're probably not going to come to us because they know enough to know that actually it's not like software where you can just do a change list and it's going to ship to production in two weeks' time.
Training clusters favor size/concentration; serving clusters favor distribution
Source
Training Data — Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI (October 6, 2026) · at 47:49 ▸
When this applies
when a system has two distinct usage modes — one benefiting from concentration/density, the other from proximity to distributed end users
The takeaway
Design infrastructure topology around each usage mode's actual constraint — concentration for throughput-heavy, latency-insensitive batch work; distribution for latency-sensitive, geographically dispersed demand.
Why it works
Training benefits from minimizing network distance between nodes (favoring large, concentrated clusters), while serving must be close to end users worldwide and needs tighter co-location of compute, storage, and networking rather than pure accelerator density.
One level up
Don't assume one infrastructure shape serves all phases of a product's lifecycle — different operating modes need structurally different deployments.
Where it stops holding
This stops working when training and serving converge in resource profile (e.g., if latency requirements for serving become as relaxed as training, or vice versa).
Detail
rule: Training clusters are large, concentrated, and vertically integrated (homogeneous); serving clusters are smaller, distributed globally, and less vertically integrated because they must co-locate compute, storage, and networking near users.
threshold: n/a - qualitative pattern
rationale: Old training clusters alone can't satisfy inference demand because training clusters are geographically concentrated by design, leaving gaps elsewhere that require dedicated serving infrastructure.
In their words
it really does have to be, okay, here are the training clusters. Yes, probably they're going to be used for serving in some number of years. But then we're also having to build the serving clusters as well.