Source
Y Combinator Startup Podcast — Inside Claude Code With Its Creator Boris Cherny (February 17, 2026) ·
at 40:02 ▸
When this applies
when trying to quantify the real impact of an internal tool rollout rather than relying on anecdote or hype
The takeaway
When evaluating a tool's impact, normalize for scale (e.g., per-engineer, not total) and cross-check with multiple metrics before trusting a dramatic number.
Why it works
Measuring productivity via a simple, cross-checked metric (pull requests, cross-checked against commits and commit lifetime) before and after a tool's adoption isolates the tool's effect from team growth, since team size alone could confound raw output.
One level up
Extraordinary productivity claims warrant triangulating multiple measurable proxies rather than resting on a single headline statistic.
Where it stops holding
This stops working when the proxy metrics themselves are gameable (e.g., more, smaller PRs) or don't capture quality/value, only volume.
Detail
situation: Anthropic's engineering team doubled in size over a year.
decision: They measured productivity per engineer via pull requests, cross-checked against commits and commit lifetimes.
outcome: Productivity per engineer grew ~70% year-over-year, and 150% since Claude Code's launch — compared to a 2% gain Boris previously considered a major achievement at Meta after a year of work by hundreds of people.
lesson: AI coding tools can produce productivity gains an order of magnitude beyond what was historically considered a major org-wide win.
In their words
I think the team doubled in size last year, but productivity per engineer grew something like 70%