ExpertsFMEpisode 14 - Building AI for better healthcare
OpenAI Podcast · March 16, 2026
5 Selected Lessons:
framework · 2 heuristic · 2 case · 1
Integrate domain expertise across every stage of model training
Source
OpenAI Podcast — Episode 14 - Building AI for better healthcare (March 16, 2026) · at 10:24 ▸
When this applies
when building any AI system for a specialized domain where generic training data won't capture necessary expert judgment
The takeaway
Integrate specialized expertise throughout the entire pipeline of a system's development, not just at one checkpoint.
Why it works
Embedding domain expert feedback at pre-training, mid-training, and post-training stages (not just final fine-tuning) compounds improvements and closes gaps that a single late-stage patch can't fix.
One level up
Depth and consistency of integration beats a single point of expert review when stakes are high.
Where it stops holding
This stops working when the domain's needs are simple enough that general-purpose training already suffices, or when the cost of full-pipeline expert integration outweighs the marginal safety/accuracy gains for a low-stakes use case.
Detail
procedure:
- Integrate health-relevant data/feedback into pre-training.
- Continue integration into mid-training.
- Continue integration into post-training (including rubric-based scoring from physicians).
- Monitor production traffic in a privacy-preserving way, feeding learnings back into further training.
rationale: Results in leading performance not just on internal benchmarks but external ones too.
In their words
to my knowledge, OpenAI's models are the only major models where every phase of model training from pre-training to mid-training to post-training and every step in between really integrates health into every major stage.
Grounding answers in current, local, and institutional guidance (not just training data)
Source
OpenAI Podcast — Episode 14 - Building AI for better healthcare (March 16, 2026) · at 16:58 ▸
When this applies
when an AI's static training knowledge must be supplemented with live, local, and institution-specific information for trust and correctness
The takeaway
For professional-grade trust, ground AI answers in live, external, and context-specific sources (latest literature, regional guidelines, institutional protocols) in addition to the model's trained knowledge.
Why it works
Static pretraining knowledge can go stale or ignore regional/institutional variation (e.g., different standard-of-care by region or resource level); connecting the model to live data sources keeps answers current and locally appropriate, which is the basis professionals need to trust the tool.
One level up
Trust from domain experts requires visible grounding in authoritative, current, and locally relevant sources, not just an averaged/generic trained response.
Where it stops holding
This stops working when integrations with external/live sources aren't available, reliable, or current, in which case grounding claims are only as good as the weakest connected source.
Detail
procedure:
- Ground answers in latest medical literature and guidelines, not just training data.
- Incorporate institution-specific or regional guidance where conditions are treated differently.
- Account for different care settings' resource/specialist availability to produce personalized care plans.
- Build secure connectivity (HIPAA-aligned) to combine sensitive data with current medical knowledge.
rationale: Keeps trust as the top priority for professional engagement with AI.
In their words
a lot of our recent work has been making sure that answers that the AI is providing are not just grounded in what the model was training on, but are grounded in the latest medical literature, the latest guidelines.
Train models to express calibrated uncertainty, not confident guesses
Source
OpenAI Podcast — Episode 14 - Building AI for better healthcare (March 16, 2026) · at 10:24 ▸
When this applies
when any high-stakes decision-support system faces incomplete information
The takeaway
In high-stakes systems, reward and train for explicit uncertainty acknowledgment and escalation suggestions over false confidence.
Why it works
Overconfident wrong answers in high-stakes domains cause real harm; training the model to recognize and voice its own uncertainty, and suggest follow-up/escalation paths, converts a potential failure into a safe handoff.
One level up
Knowing what you don't know is itself a core competency to optimize for, not a side effect.
Where it stops holding
This stops working when excessive hedging erodes user trust/usability to the point the tool is abandoned, or when the domain demands a decisive single answer regardless of confidence.
Detail
rule: Model should say when it doesn't know and suggest a follow-up (referral, additional test) instead of guessing confidently.
threshold: Applied specifically to high-stakes fields like healthcare.
rationale: One of the most important properties for safety in high-stakes domains.
In their words
I think in such a high stakes field like healthcare, one of the most important things is that the model can be trained to better know when it doesn't know and say that.
Security plus empowerment as the foundation for sensitive-data products
Source
OpenAI Podcast — Episode 14 - Building AI for better healthcare (March 16, 2026) · at 3:05 ▸
When this applies
when building any product that must handle users' sensitive personal data while still being genuinely useful
The takeaway
Pair strict data protection guarantees with mechanisms for users to voluntarily bring in personal context, rather than treating privacy and personalization as in tension.
Why it works
Strong technical/policy protections (encryption, no training on user data) remove the trust barrier to engagement, while features letting users bring in personal context remove the generic, 'one-size-fits-all, amnesiac' limitation of prior tools (e.g., search engines), together enabling deeper, safer usage.
One level up
Trust and usefulness are not trade-offs when the product is designed so that privacy protection is the precondition for deeper personalization, not its opposite.
Where it stops holding
This stops working when users don't have enough control/visibility to meaningfully consent, or when the protections (e.g., 'never train on this data') can't be credibly enforced or verified by users.
Detail
rule: Combine encryption/non-training guarantees ('one-way valve') with user-chosen context features.
threshold: Applies specifically to conversations/data flagged as health-related.
rationale: Search engines of the past were one-size-fits-all with amnesia; context matters in health, so security must be paired with empowerment to differentiate.
In their words
So these extra security layers, these protections to make sure that we will never train on users' health care data, combined with empowerment, really, you know, search engines that people have used before to navigate health have amnesia. They're one-size-fits-all.
Active change management builds indispensability, not just adoption
Source
OpenAI Podcast — Episode 14 - Building AI for better healthcare (March 16, 2026) · at 26:50 ▸
When this applies
when rolling out any new tool into an established expert workflow and hoping for genuine, sustained adoption rather than token compliance
The takeaway
Deep, repeated, on-site collaboration with end users during rollout converts a tool from 'available' to 'indispensable.'
Why it works
Repeated in-person engagement (traveling to the site multiple times) to co-design how the tool fits into existing routines builds the kind of deep workflow integration that produces felt indispensability, as opposed to a tool merely being available.
One level up
Sustained hands-on change management is often the differentiator between adoption and true behavior change.
Where it stops holding
This stops working when the resource/time investment for repeated on-site engagement isn't available at scale, limiting this approach to pilot-sized deployments.
Detail
situation: Clinicians in Nairobi were being introduced to a new AI copilot tool.
decision: The team flew to Kenya multiple times to work closely with clinicians, deepening how the tool fit into their workflows.
outcome: The tool became something clinicians felt was indispensable, to the point that running a future study with a non-AI control group felt dangerous/unethical to the partner team.
lesson: Close, repeated, in-person collaboration during deployment can turn a tool into something users feel they can't do without.
In their words
the team was actually thinking about the team at Panda Health was thinking about potentially running another study. And they actually had a lot of hesitance around running another study, because that would have involved having some group of clinicians using AI and some group of clinicians not using AI. They actually felt that it was dangerous to have a group of clinicians not using the AI.