ExpertsFMHow Cursor Trained Composer on Fireworks: Distributed Infrastructure for High-Performance RL
Training Data · May 26, 2026
5 Selected Lessons:
framework · 2 reasoning_pattern · 1 priority · 1 heuristic · 1
Reward verifiability ranking for scaling RL
Source
Training Data — How Cursor Trained Composer on Fireworks: Distributed Infrastructure for High-Performance RL (May 26, 2026) · at 37:34 ▸
When this applies
when choosing how to measure success for any optimization process across a spectrum of task verifiability
The takeaway
Prefer the most automatable/verifiable proxy for success available, since verifiability is what allows a reward signal to scale with compute; fall back to judge-based approximation only when true verification isn't possible, and decompose complex judgments into separate simpler rubrics.
Why it works
More verifiable (automatable, deterministic) rewards let you scale compute arbitrarily without bottlenecking on human judgment, since you can just 'turn on more compute and see the graph go up'; less verifiable tasks require LLM-as-judge approximations, which still enable scale but at lower fidelity than ground-truth verification.
One level up
The scalability of any optimization process is gated by the scalability of its feedback signal, not by the scalability of compute alone.
Where it stops holding
This stops working when even decomposed rubric-based LLM judgment is unreliable or when the task's success criteria genuinely can't be captured by any approximate proxy.
Detail
hierarchy:
- Deterministic/verifiable rewards (e.g., does code compile, is math answer correct) — the best option
- LLM-as-judge with decomposed, precise rubrics (style, factuality, etc. judged separately) — next best, scalable approximation
- Experts crafting the task/evaluation criteria — still essential upstream, even if not directly judging each rollout
rationale: Judging is easier than generating, so LLM judges can reliably rank outputs against crafted criteria even when they couldn't generate the ideal output themselves; decomposing multi-aspect judgments into separate rubrics avoids confusing a single judge model.
In their words
I mean, generally, the more verifiable your reward is, the better, because it allows you to scale the compute and just get better outcome.
Two-axis training recipe: continual pre-training then large-scale RL
Source
Training Data — How Cursor Trained Composer on Fireworks: Distributed Infrastructure for High-Performance RL (May 26, 2026) · at 6:16 ▸
When this applies
when trying to improve a capability-rich but behaviorally unrefined base model for a specific application
The takeaway
Separate 'teaching the system what exists' from 'teaching the system how to act correctly in its real environment' as two distinct training stages, each with its own optimization target.
Why it works
Mid-training on domain data widens the model's knowledge distribution (what it knows), while RL in the actual target environment sharpens that distribution toward correct, in-context behavior (how it acts) — the two stages address different failure modes (lack of knowledge vs. lack of correctness/calibration).
One level up
Complex capability transfer often requires sequential stages that first broaden a distribution, then sharpen it against a real task signal.
Where it stops holding
This stops working when there isn't a cheap way to generate near-pretraining-scale domain data for the mid-training stage, or when the RL environment can't closely approximate the real deployment context.
Detail
procedure:
- Start from a strong sparse MOE base model (e.g., Kimi 2.5, 1T params, 30B active)
- Run large-scale mid-training ('almost pre-training scale') on code tokens, libraries, patterns, and web data to widen the knowledge distribution
- Take checkpoints from mid-training and run large-scale RL on many tasks inside the actual product environment
- RL teaches correct code, proper tool calling, and navigation of the real environment the model will live in
rationale: Mid-training teaches the model how to write code generally; it doesn't teach the model to differentiate correct from incorrect code. RL tunes that distinction by rewarding correctness directly.
In their words
So, we started off the training run by doing lots of mid-training on code tokens, almost sort of pre-training scale, actually. And then coming out of that mid-training run, we took the checkpoints and we did the very large scale RL on lots of lots of tasks.
Disaggregating RL infrastructure across heterogeneous, globally distributed clusters
Source
Training Data — How Cursor Trained Composer on Fireworks: Distributed Infrastructure for High-Performance RL (May 26, 2026) · at 14:38 ▸
When this applies
when a workload has components with very different hardware/network requirements, and scaling one monolithic resource is difficult
The takeaway
When a workload's sub-components have different resource profiles, disaggregate them onto hardware matched to each component's actual requirements rather than forcing everything onto one uniform, maximally-provisioned resource.
Why it works
RL training is heterogeneous — the trainer needs tightly interconnected, high-bandwidth clusters, while inference/rollout generation needs looser interconnects and can tolerate smaller, cheaper, even mixed-generation hardware spread across locations; splitting these lets you avoid needing one giant expensive cluster, exploit idle production capacity, and scale each piece independently.
One level up
Exploiting heterogeneity in resource requirements is a general lever for efficiency that larger, monolithic designs often leave on the table.
Where it stops holding
This stops working when the components genuinely require tight synchronous coupling (e.g., workloads that cannot tolerate network latency or version staleness at all), or when operational complexity of managing distributed clusters outweighs the cost savings.
Detail
procedure:
- Keep one contiguous, high-interconnect cluster for the trainer
- Distribute the inference/rollout component across smaller clusters globally, even using idle production inference capacity (e.g., spare Composer 1.5 serving GPUs)
- Leverage different GPU generations/types for inference since it doesn't need wide interconnect
- Treat the whole inference pool as fungible between serving real users and serving simulated RL environments
rationale: Large contiguous clusters are hard to find and expensive; at Composer's scale, finding a 2x larger cluster is significantly harder than finding the current size. Disaggregating avoids this bottleneck and unlocks cost and flexibility advantages.
In their words
but then the inference component of reinforcement learning, we can globally distribute that across small clusters all over the world.
Specialization doesn't contradict the bitter lesson because frontier labs already specialize on code at scale
Source
Training Data — How Cursor Trained Composer on Fireworks: Distributed Infrastructure for High-Performance RL (May 26, 2026) · at 3:01 ▸
When this applies
when deciding whether narrow specialization conflicts with a broader trend favoring scale and generality
The takeaway
Pursuing scale (more relevant data, more capacity used efficiently) and pursuing specialization are not opposites — specialization can be a way of applying the scaling principle to a narrower domain.
Why it works
Large general models already allocate significant training data to the specific skill (code) because labs value it; specializing further just pushes harder on the data dimension within a finite-capacity model, which is consistent with scaling principles rather than opposed to them.
One level up
Apparent contradictions between two strategic principles often dissolve once you examine the actual mechanism each relies on.
Where it stops holding
This stops working when the specialized domain truly has far less usable data than general domains, making the 'push harder on data' strategy infeasible.
Detail
approach: Reconcile apparent tension between 'bigger is better/more general' and 'specialize narrowly' by identifying that both rely on the same underlying lever: maximizing useful data ingested relative to model capacity.
signals:
- Frontier labs already train heavily on code, so they're not purely 'general'
- Models have finite capacity that gets diluted by irrelevant tasks
- Removing distractions frees capacity to ingest more relevant data
rationale: If you believe in the bitter lesson, you push hard on data; specializing is one way to free capacity to ingest more of the data that matters for your specific task.
In their words
I think for our actually, you know, if we believe about the bitter lesson, we are just pushing very hard on the data dimension. And we know that the models inherently have finite capacity. And so if we want to saturate all that capacity, we need to scale data.
Keep RL environments on the customer's own infrastructure, not a hosted abstraction
Source
Training Data — How Cursor Trained Composer on Fireworks: Distributed Infrastructure for High-Performance RL (May 26, 2026) · at 40:13 ▸
When this applies
when deciding where to physically/organizationally locate a system component that depends heavily on implementation-specific context owned by one party
The takeaway
Locate each component of a distributed system where its required context/fidelity naturally lives, rather than forcing uniform centralization for operational convenience.
Why it works
Environments are tightly coupled to the customer's real implementation (their actual product, services, data); trying to wrap and componentize that on a generic hosted platform introduces differences from the true production setup, which (per the earlier lesson) can degrade reward quality — so running environments where the real implementation already lives avoids that fidelity loss, even while other components (like the trainer) can be centrally hosted.
One level up
Architectural decisions about 'where something runs' should follow from fidelity/context requirements, not from a default preference for centralizing everything on one platform.
Where it stops holding
This stops working when the customer lacks the infrastructure/expertise to host and isolate their own environment safely, making a hosted/wrapped version from the vendor preferable despite fidelity loss.
Detail
rule: Default to running RL environments on the customer's own side (close to or within their real implementation), while the trainer can run on a central training platform.
rationale: This avoids introducing differences from the true production setup that would otherwise result from trying to wrap/componentize a customer's environment on a generic hosted platform.
In their words
We actually default to running them on the customer side, because that's where the actual implementation is. And you effectively have the same setup of trainer, even if it's part of a works platform or on the customer side, call in the actual production environment, not trying to kind of wrap it and componentize it on the hosted platform, because that's really hard and that introduces differences.