---
title: What 250 validation runs reveal about agentic AI in real-world evidence
description: Discover how HealthVerity eXOs outperformed traditional methods in RWE generation, achieving transparency and reproducibility in real-world evidence analysis.
image: https://blog.healthverity.com/hubfs/what-250-validation-runs-featured.png
---

Announcing HealthVerity Precision Event Alerts, a next-generation physician alerting solution. [Learn more](https://healthverity.com/news/healthverity-precision-event-alerts-launch/)

- [Contact](https://healthverity.com/contact/)
- [Sign In](https://marketplace.healthverity.com/login/?next=/marketplace/)

[![HealthVerity-Logo-Dark-3](https://blog.healthverity.com/hs-fs/hubfs/HealthVerity-Logo-Dark-3.png?width=1662&height=320&name=HealthVerity-Logo-Dark-3.png)](https://healthverity.com/)

- How We Help 
    - Featured Content 
          - [HealthVerity MarketplaceTM](https://healthverity.com/marketplace/healthverity-marketplace/)
          - [HealthVerity TaxonomySM](https://healthverity.com/taxonomy-portal/)
          - [Life Sciences](https://healthverity.com/life-sciences/)
          - [Identity Manager (HVID)](https://healthverity.com/identity-manager/)
    - Data Ecosystem 
          - [HealthVerity MarketplaceTM](https://healthverity.com/marketplace/healthverity-marketplace/)
          - [HealthVerity TaxonomySM](https://healthverity.com/taxonomy-portal/)
          - [Precision Event Alerts](https://healthverity.com/precision-event-alerts/)
          - [HealthVerity Mastersets](https://healthverity.com/mastersets/portal/)
          - [Media Measurement](https://healthverity.com/media-measurement/)
          - [Insurance Underwriting](https://healthverity.com/insurance-underwriting/)
    - Technology Products 
          - [Identity Manager (HVID)](https://healthverity.com/identity-manager/)
          - [HealthVerity FLOW](https://healthverity.com/flow/portal/)
          - [Governance Manager](https://healthverity.com/governance-manager/)
          - [Smart Contracts](https://healthverity.com/smart-contracts-manager/)
- Symphony 
    - Symphony Health Platform 
          - [Symphony Health Overview](https://healthverity.com/symphony-health/overview/)
          - [Integrated Dataverse](https://healthverity.com/symphony-health/integrated-dataverse-and-source-solutions/)
          - [Symphony HealthCloud](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/)
    - Symphony Solutions 
          - [Health technology and business intelligence solutions](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/)
          - [Integrated Dataverse and Source Solutions](https://healthverity.com/symphony-health/integrated-dataverse-and-source-solutions/)
          - [Brand strategy and sales](https://healthverity.com/symphony-health/optimize-brand-strategy-and-sales-effectiveness/)
          - [HealthRankings](https://healthverity.com/symphony-health/health-rankings/)
          - [Market access](https://healthverity.com/symphony-health/solve-market-access-challenges/)
          - [Patient Journey](https://healthverity.com/symphony-health/understand-the-patient-journey/)
          - [Medicines affordability](https://healthverity.com/symphony-health/making-medicines-more-affordable/)
    - Symphony Products 
          - [PayerSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PayerSource.pdf)
          - [PrescriberSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PrescriberSource.pdf)
          - [PatientSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PatientSource.pdf)
          - [PatientSourceCompleteTM](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PatientSourceComplete.pdf)
          - [PatientSourceDUOTM](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PatientSourceDUO.pdf)
          - [Symphony FLEXTM](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-Symphony-FLEX.pdf)
          - [CustomerSource®](https://healthverity.com/symphony-health/integrated-dataverse-and-source-solutions/)
          - [DiagnosticSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-DiagnosticSource.pdf)
          - [NonRetailSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-NonRetailSource.pdf)
    - Symphony Technology 
          - [Symphony HealthCloud](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/)
          - [Synoma Data Integration](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/)
          - [Vantage Applications](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-Vantage-Applications.pdf)
          - [Metys](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/metys/)
- Who We Serve 
    - [Life Sciences](https://healthverity.com/life-sciences/)
    - [Government](https://healthverity.com/government/)
    - [Insurance](https://healthverity.com/insurance/)
    - [Channel Partners](https://healthverity.com/channel-partners/)
    - [Data Ecosystem Partners](https://healthverity.com/data-ecosystem-partners/)
- Resources 
    - [HealthVerity Blog](https://blog.healthverity.com/)
    - [News & Events](https://healthverity.com/resources/news-and-events/)
    - [Resource Library](https://healthverity.com/resources/resource-library/)
    - [Publications](https://healthverity.com/resources/publications/)
    - [The IPGE Approach](https://healthverity.com/ipge-approach/)
    - [Zombie Data Explained](https://healthverity.com/get-verified/)
- [Careers](https://healthverity.com/careers/)
- [About Us](https://healthverity.com/about-us/)

[Request Demo](https://healthverity.com/request-demo/)

[AI](https://blog.healthverity.com/tag/ai), [eXOs](https://blog.healthverity.com/tag/exos)

# What 250 validation runs reveal about agentic AI in real-world evidence

[By \- HealthVerity,](https://blog.healthverity.com/author/healthverity)  September 23, 2026

![](https://blog.healthverity.com/hubfs/what-250-validation-runs-featured.png)

## Stress-testing a research question

Every technology vendor claims their product works. The question that matters for life sciences is: **how do you know, and can someone else verify it?**

When Medeloop’s Chief Epidemiologist John Ayers set out to evaluate [HealthVerity eXOs](https://healthverity.com/exos/), he brought the skepticism of a scientist, not the enthusiasm of a vendor. And what he was most skeptical of was not the AI, but the status quo.

> "We evaluated studies from the New England Journal of Medicine, JAMA, the Lancet, the British Medical Journal. It turned out about 90% of those studies misinterpreted their primary finding." **— John Ayers, Chief Epidemiologist, Medeloop**

 

That finding reframes the question. Compared with how RWE is done today, how reliable, transparent and verifiable is this platform? And by that measure, the relevant standard is not perfection, but whether agentic AI can make real-world evidence generation more transparent, reproducible and verifiable than the status quo.

## How HealthVerity eXOs handles the hallucination problem

One of the most frequently cited concerns about AI systems in high-stakes settings is hallucination: the generation of confident-sounding outputs that have no basis in the underlying data. In a real-world evidence context, hallucinated numbers could find their way into regulatory submissions, payer dossiers, or clinical guidelines.

Across 250 independent runs, HealthVerity eXOs recorded zero hallucinations.1 The platform did not invent numbers not present in the data or generate results from language model inference. The platform wrote and executed code against real data, then reported what the code returned. The outputs were grounded in the data by construction.

That distinction between a system that generates text about data and a system that analyzes data and then explains the results, is fundamental to understanding why agentic AI can be more trustworthy than generative AI for this class of problems. [Read our blog](https://blog.healthverity.com/genai-vs.-agentic-ai-the-difference-for-real-world-evidence) to understand the differences between agentic and generative AI.

## Reproducibility: What the score actually means

The HealthVerity eXOs validation study reported a reproducibility consistency score of 0.758 across repeated runs of the same research questions. For non-technical audiences, that number requires some context to interpret correctly.1

First, the benchmark: in published epidemiological literature, fewer than 2% of studies are independently judged reproducible meaning a reader could understand the methods well enough to replicate the study and expect to arrive at a comparable result. eXOs's score is not measured against a theoretical ideal; it is measured against the noisy, variable reality of how real-world research is actually conducted.

Second, the gap between the score and perfect reproducibility reflects that different methodological choices can produce different results, which is why visibility into study design, cohort logic, code sets and analysis steps is essential.

## eXOs and accuracy against published literature

The gold standard for accuracy in observational research is concordance with previously published studies. If an agentic AI system produces results that are systematically inconsistent with the existing evidence base, that is a signal worth taking seriously.

Across the 250 eXOs runs, more than 90% achieved high or medium accuracy alignment with published literature. That figure is not derived from cherry-picked questions, rather it reflects performance across 50 diverse research questions designed to represent the range of analyses RWE teams actually conduct.

The code prompt adherence score, 99.2 out of 100, captures something related but distinct: whether the system executed the analysis it said it would execute. The protocol, the code, and the results are consistent. What the system documented doing is what it actually did.

## Democratizing quality

One consequence of these validation results that deserves more attention is the equity dimension. Not every organization has the same access to experienced biostatisticians, seasoned epidemiologists and rigorous QA processes. Smaller teams, earlier-stage companies, and functions like Medical Affairs that have historically received a "lesser version" of RWE — they can now access the same analytical rigor as the best-resourced teams in the industry.

What the validation evidence demonstrates is that HealthVerity eXOs works consistently, transparently and in a way that can be independently verified. For the field, that is the proof of concept the industry has been waiting for.

Curious how eXOs performs when tested against published literature? Read the full validation study: [Validating HealthVerity eXOs](https://info.healthverity.com/validating-healthverity-exos-an-agentic-ai-platform-for-real-world-evidence)

[![Get to know HealthVerity eXOs](https://no-cache.hubspot.com/cta/default/5412465/interactive-222477532334.png)](https://blog.healthverity.com/hs/cta/wi/redirect?encryptedPayload=AVxigLJYiuoEPA583zaTWQ0i7D2Ukly7UdsbYLtFUuZESlJqf66yck0bg%2B2XVR%2FdFpD7z74bMqJAjIiE%2BRPC0elck%2BjmXbHycNS6Aox%2BIzqDJesgks5hn6r9K7VsJ8BClB4Vjv79VZGAv7qfRR6UgQK04AThk816fXC3P1QlVLiQGMMicrdjkSlnaWNSOiswN7lDvrU3knvgvfpkF%2BwyDJNuoEENRFhAfAbH%2BzmIzpsLBt5mxfgkptkt8gfqKzPF3%2BFnZUIQI%2BxD8pqwUVVP3qU%3D&webInteractiveContentId=222477532334&portalId=5412465)

## References

HealthVerity. *Validating HealthVerity eXOs: An Agentic AI Platform for Real-World Evidence*. Published May 4, 2026. [https://info.healthverity.com/validating-healthverity-exos-an-agentic-ai-platform-for-real-world-evidence](https://info.healthverity.com/validating-healthverity-exos-an-agentic-ai-platform-for-real-world-evidence)

 

[![Explore the eXOs validation study](https://no-cache.hubspot.com/cta/default/5412465/interactive-210403853571.png)](https://blog.healthverity.com/hs/cta/wi/redirect?encryptedPayload=AVxigLLbRQM7bPrseyUwLC2klmEGjaTj85Lz45eUlR%2Bf2MpKJgQlh0X0pyDavnu3VXUpaV79xrGXZyhUWPAAfepLVffcys%2Ft1KMPk8DBilHswzTUvA8bZu2%2FSgpZmgmczGxIP1mXZHfg7NCWZouufTD1Uou4DT6%2FizJAEsIgjVTuy7W6mEm0tDD%2FPZCT1dQ%2FNWGXfP2TNRkDzp5g%2F6IMqg7uGdNIOco3tLG4rDFSmIsphhWjmklGISQYxm2IbU0ykTO2m0cKu7PCQVd6SDunSCKOYi8iNb%2BjRLTr92cqulUq2Do%3D&webInteractiveContentId=210403853571&portalId=5412465)

<https://info.healthverity.com/healthverity-live-march-2025>

###### Category

- [Solutions & Best Practices](https://blog.healthverity.com/tag/solutions-best-practices)
- [Trends & Insights](https://blog.healthverity.com/tag/trends-insights)
- [HEOR](https://blog.healthverity.com/tag/heor)
- [Marketplace (RWD)](https://blog.healthverity.com/tag/marketplace-rwd)
- [Thought Leadership](https://blog.healthverity.com/tag/thought-leadership)
- [Industry Updates](https://blog.healthverity.com/tag/industry-updates)
- [Open & Closed Claims](https://blog.healthverity.com/tag/open-closed-claims)
- [Case Studies](https://blog.healthverity.com/tag/case-studies)
- [Commercial Pharma](https://blog.healthverity.com/tag/commercial-pharma)
- [AI](https://blog.healthverity.com/tag/ai)
- [Government](https://blog.healthverity.com/tag/government)
- [eXOs](https://blog.healthverity.com/tag/exos)
- [Audiences](https://blog.healthverity.com/tag/audiences)
- [taXonomy Pathways](https://blog.healthverity.com/tag/taxonomy-pathways)
- [Clinical Notes](https://blog.healthverity.com/tag/clinical-notes)
- [Rare Diseases](https://blog.healthverity.com/tag/rare-diseases)

###### Popular Post

- [The GLP-1 trend: What the real-world data reveals](https://blog.healthverity.com/the-glp-1-surge-what-the-real-world-data-reveals)
- [GLP-1 trends 2025: real-world data, patient outcomes & future therapies](https://blog.healthverity.com/glp-1-trends-2025-real-world-data-patient-outcomes-future-therapies)
- [Open claims vs. closed payer claims: What’s the difference?](https://blog.healthverity.com/open-claims-vs-closed-payer-claims)
- [An integrated approach for deeper insight with both open and closed claims data](https://blog.healthverity.com/an-integrated-approach-for-deeper-insight-with-both-open-and-closed-claims-data)
- [GLP‑1 real‑world data for obesity: A new standard for metabolic care](https://blog.healthverity.com/glp-1-real-world-data-for-obesity-a-new-standard-for-metabolic-care)

###### Subscribe

### Similar Blogs

[![How AI is changing epidemiology and real-world evidence](https://blog.healthverity.com/hs-fs/hubfs/ask-an-expert-blog-featured-alt-5.png?width=352&name=ask-an-expert-blog-featured-alt-5.png)](https://blog.healthverity.com/how-ai-is-changing-epidemiology-and-real-world-evidence)

#### [How AI is changing epidemiology and real-world evidence](https://blog.healthverity.com/how-ai-is-changing-epidemiology-and-real-world-evidence)

By [Alexa Woodward, Ph.D.](https://blog.healthverity.com/author/alexa-woodward-ph-d) 23 October 2025

 AI changed how we discover medicines. What's next? In just a few years, artificial intelligence...

[READ MORE](https://blog.healthverity.com/what-250-validation-runs-reveal-about-agentic-ai-in-real-world-evidence)

[![Can you trust AI for RWE? HealthVerity eXOs validation study says yes](https://blog.healthverity.com/hs-fs/hubfs/can-you-trust-ai-for-rwe-healverity-exos-validation-study-says-yes-featured.png?width=352&name=can-you-trust-ai-for-rwe-healverity-exos-validation-study-says-yes-featured.png)](https://blog.healthverity.com/can-you-trust-ai-for-rwe-healthverity-exos-validation-study-says-yes)

#### [Can you trust AI for RWE? HealthVerity eXOs validation study says yes](https://blog.healthverity.com/can-you-trust-ai-for-rwe-healthverity-exos-validation-study-says-yes)

By [HealthVerity](https://blog.healthverity.com/author/healthverity) 6 April 2026

 Artificial intelligence is moving quickly into healthcare research, but one question still comes...

[READ MORE](https://blog.healthverity.com/what-250-validation-runs-reveal-about-agentic-ai-in-real-world-evidence)

[![Building guardrails for AI-driven real-world data programs](https://blog.healthverity.com/hs-fs/hubfs/building-guardrails-for-ai-featured%20(1).png?width=352&name=building-guardrails-for-ai-featured%20(1).png)](https://blog.healthverity.com/building-guardrails-for-ai-driven-real-world-data-programs)

#### [Building guardrails for AI-driven real-world data programs](https://blog.healthverity.com/building-guardrails-for-ai-driven-real-world-data-programs)

By [HealthVerity](https://blog.healthverity.com/author/healthverity) 12 February 2026

 AI is rapidly reshaping how real-world data (RWD) is generated and analyzed across healthcare...

[READ MORE](https://blog.healthverity.com/what-250-validation-runs-reveal-about-agentic-ai-in-real-world-evidence)

#### Who we serve

- [Life Sciences](https://healthverity.com/life-sciences/)
- [Government](https://healthverity.com/government/)
- [Insurance](https://healthverity.com/insurance/)
- [Channel Partners](https://healthverity.com/channel-partners/)
- [Data Ecosystem Partners](https://healthverity.com/data-ecosystem-partners/)

#### How we help

- [HealthVerity Marketplace](https://healthverity.com/marketplace/)
- [HealthVerity Mastersets](https://healthverity.com/mastersets/)
- [HealthVerity FLOW](https://healthverity.com/flow/)
- [The IPGE Approach](https://healthverity.com/ipge-approach/)

#### Resources

- [HealthVerity Blog](https://blog.healthverity.com/)
- [News & Events](https://healthverity.com/resources/news-and-events/)
- [Resource Library](https://healthverity.com/resources/resource-library/)
- [Publications](https://healthverity.com/resources/publications/)

#### Our company

- [About Us](https://healthverity.com/about-us/)
- [Leadership](https://healthverity.com/about-us/#leadership)
- [Board of Directors](https://healthverity.com/about-us/#board)
- [Careers](https://healthverity.com/careers/)
- [Contact](https://healthverity.com/contact/)

©2026 HealthVerity, Inc.    |    [Terms & Conditions](https://healthverity.com/terms-and-conditions/)    |    [Privacy Policy](https://healthverity.com/privacy-policy/)

<https://twitter.com/healthverity> <https://www.linkedin.com/company/healthverity>