ExpertsFMSimulating Humans at Scale: Simile's Joon Sung Park
Training Data · June 16, 2026
5 Selected Lessons:
reasoning_pattern · 2 framework · 2 heuristic · 1
Grounding simulated agents in real panel data before scaling questions
Source
Training Data — Simulating Humans at Scale: Simile's Joon Sung Park (June 16, 2026) · at 12:44 ▸
When this applies
when you want a model to answer an unbounded number of future questions about a population, but can only afford limited direct contact with real members of that population
The takeaway
Invest limited real-world data collection into building a persistent, reusable representation (agent/model) rather than a one-off answer set, so it amortizes across many future queries.
Why it works
By collecting efficient, generalizable data from a smaller real sample (via panel partners) and using it to build persistent agent-models of those people, a platform can answer many future questions beyond the original survey's scope without recontacting people.
One level up
Convert a scarce, expensive resource (real human attention) into a durable, reusable asset (a calibrated model) rather than consuming it per-question.
Where it stops holding
This stops working when the initial real-data collection fails to capture generalizable signal, or when the questions asked later are too far outside the domain the initial data could support.
Detail
procedure:
- Customer specifies the population of interest.
- Simile works with panel/polling vendor partners to reach real humans.
- Collect data during a limited time window (e.g., 15 minutes) designed to be maximally efficient and generalizable.
- Use that data to construct persistent agents/simulations of those people.
- Load agents onto a SaaS platform where customers can ask arbitrary follow-up questions beyond the original survey domain.
benchmarks: {}
rationale: Grounding in real data first, then generalizing via agent models, lets a single data-collection effort serve unlimited future queries.
In their words
So imagine you have 15 minutes. What are the magical questions you can answer or you can ask these people during that time? We collect that data, use that data to create agents or simulations of these people that can basically be used to answer a large number of questions that goes way beyond the original domain.
Dual data pipeline: RCT-trained behavioral foundation model plus efficient interviews
Source
Training Data — Simulating Humans at Scale: Simile's Joon Sung Park (June 16, 2026) · at 17:51 ▸
When this applies
when you need both deep individual context and broad factual coverage to predict outcomes of structured experiments
The takeaway
Combine a model trained directly on the outcome type you want to predict (e.g., experimental results) with an efficient, objective-optimized data-collection process for individual-level context, using the cheapest modality (survey vs. interview) suited to each data type.
Why it works
Training on a large repository of real randomized controlled trials lets the model learn to predict RCT outcomes directly (a 'foundation model of human behavior'); meanwhile a trained interviewer model optimized to extract maximum relevant information per unit time efficiently captures each individual's distinguishing context, and discrete surveys efficiently capture broad factual/attitudinal breadth.
One level up
Separate your data pipeline into outcome-grounding data (trains the prediction target directly) and context-grounding data (personalizes predictions), and optimize collection efficiency separately for each.
Where it stops holding
This stops working when there isn't a large enough repository of real RCTs in the domain, or when customers' proprietary data can't be ethically/responsibly incorporated.
Detail
procedure:
- Build a foundation model trained on a large repository of real RCTs (e.g., pricing studies) to directly predict experimental outcomes.
- Train a reinforcement-learning-optimized interviewer model whose objective is minimum time for maximum visibility into a person's long-tail life story.
- Use discrete surveys for factual/attitudinal breadth questions efficiently.
- Explore responsibly incorporating customers' own large proprietary datasets to create customer-specific fine-tuned model variants.
benchmarks: {}
rationale: Different data types (experimental outcomes, deep personal context, broad factual breadth) are each captured by the modality best suited to extracting them efficiently.
In their words
So one of the models that we are training is basically the foundation model of human behavior in quite a literal sense. We have all the behavioral signals from RCTs. Can we actually encode that into the model so that the end outcome is a model that can basically predict the results of any RCTs.
Simulation as substitute for costly field-testing of social platforms
Source
Training Data — Simulating Humans at Scale: Simile's Joon Sung Park (June 16, 2026) · at 3:33 ▸
When this applies
when a design decision's real risk only manifests at large scale/interaction with many other agents, and testing at that scale in reality is expensive or dangerous
The takeaway
When the real risk of a decision only emerges from aggregate interaction effects at scale, build a pre-deployment simulation of that scale rather than relying solely on live field tests.
Why it works
Field-testing at scale (releasing to real users) is the only current way to observe large-scale emergent effects, but it incurs high cost (time, harm, reputational risk) if the design is flawed; a validated simulation can surface the same emergent effects cheaply and before harm occurs.
One level up
Move irreversible, costly discovery processes into a cheap, repeatable simulated environment whenever fidelity allows.
Where it stops holding
This stops working when the simulation's fidelity to real human behavior is too low to predict the actual emergent effect, making the simulation misleading rather than protective.
Detail
approach: Identify that the hardest part of platform design is not UI/UX but how millions/billions of users jointly produce emergent phenomena.
signals:
- high human/time cost of field testing
- risk of negative emergent effects like harmful content propagation
- lack of existing tools to test for this pre-launch
rationale: A simulation tool lets designers test for bad emergent outcomes before they occur in the real world, avoiding the cost of live experimentation.
In their words
The only way we test it today is you basically field test it. You release your prototype, see what happens, and sometimes it actually comes at a real cost.
Why simulation beats live experiments: scale, representativeness, and second-order effects
Source
Training Data — Simulating Humans at Scale: Simile's Joon Sung Park (June 16, 2026) · at 23:48 ▸
When this applies
when deciding whether to test a decision via a real-world pilot versus a model-based simulation
The takeaway
Favor simulation over live real-world pilots when (a) the real-world channel's population is smaller or less representative than needed, and (b) you need to trace downstream/second-order consequences that a single-shot real experiment cannot reveal.
Why it works
Live experiments (e.g., Facebook ads) are bounded by the platform's available population and biased toward whoever self-selects to respond, while a validated simulation can pull any number of agents matched to a true representative sample; simulations can also chain forward to model downstream, second-order effects (e.g., how launching an EV changes perception of the rest of a product line) that a single live test cannot observe at all.
One level up
Choose your evaluation method based on what dimension of truth you most need — direct realism, representativeness, or causal chain visibility — since no single method maximizes all three.
Where it stops holding
This stops working when the simulation's underlying fidelity to real behavior is not validated, or when the real-world channel already has excellent representativeness and reach for the question being asked.
Detail
approach: Compare the achievable population size/representativeness and causal chain visibility of a live test versus a simulation.
signals:
- live test population is bounded and self-selected
- customers' core pain point is reaching the right population, not designing the right questions
- customers want multi-year downstream implications, not single-point answers
rationale: Simulation solves the representativeness/reach bottleneck and uniquely enables modeling of second-order, long-term implications that no live pilot can test today.
In their words
So you can basically pull in any number of users, doesn't even have to be bounded by the number of population that's available on Facebook. And it's also much more representative because only certain groups of people will actually respond to the online experiments.
Closing the say-do gap with behavioral and long-tail life-story data
Source
Training Data — Simulating Humans at Scale: Simile's Joon Sung Park (June 16, 2026) · at 15:27 ▸ · at 17:51 ▸
When this applies
when a model trained mostly on stated opinions/attitudes needs to predict actual behavior, which often diverges from stated attitudes
The takeaway
When a model's training data is biased toward one data type (stated attitudes) but the goal is a different type (actual behavior), deliberately collect the underdersampled data type to bridge the gap.
Why it works
LLMs are trained predominantly on attitudinal text (what people say online), which differs systematically from what people actually do; collecting behavioral data and deep personal narrative ('story of your life') data creates a translational layer that bridges stated attitudes to predicted behavior.
One level up
Identify the systematic bias in your data source relative to your prediction target, and target data collection specifically to close that bias.
Where it stops holding
This stops working when the say-do gap is small for the domain in question, making extra behavioral/narrative data collection unnecessary overhead.
Detail
rule: Supplement attitudinal LLM training data with collected behavioral data and deep personal narrative data to close the say-do gap.
threshold: Narrative data like 'tell me the story of your life' is valued because it surfaces 'long tail' information (life decisions, origins) rather than momentary opinions.
rationale: This long-tail narrative data acts as a translational layer between attitudes and behavior, improving behavioral prediction fidelity.
In their words
There are things that people say, and then there are people, there are things that people actually do. And the gap, there is real.