---
title: The risks of false negatives in healthcare data linkage
description: False negatives in record matching lead to low match rates, short patient histories, and fragmented insights. Here’s how to spot and fix them.
image: https://blog.healthverity.com/hubfs/Blog%20Thumbnail%20-%20Green.png
---
Announcing HealthVerity Precision Event Alerts, a next-generation physician alerting solution. [Learn more](https://healthverity.com/news/healthverity-precision-event-alerts-launch/)
- [Contact](https://healthverity.com/contact/)
- [Sign In](https://marketplace.healthverity.com/login/?next=/marketplace/)
[](https://healthverity.com/)
- How We Help
- Featured Content
- [HealthVerity MarketplaceTM](https://healthverity.com/marketplace/healthverity-marketplace/)
- [HealthVerity TaxonomySM](https://healthverity.com/taxonomy-portal/)
- [Life Sciences](https://healthverity.com/life-sciences/)
- [Identity Manager (HVID)](https://healthverity.com/identity-manager/)
- Data Ecosystem
- [HealthVerity MarketplaceTM](https://healthverity.com/marketplace/healthverity-marketplace/)
- [HealthVerity TaxonomySM](https://healthverity.com/taxonomy-portal/)
- [Precision Event Alerts](https://healthverity.com/precision-event-alerts/)
- [HealthVerity Mastersets](https://healthverity.com/mastersets/portal/)
- [Media Measurement](https://healthverity.com/media-measurement/)
- [Insurance Underwriting](https://healthverity.com/insurance-underwriting/)
- Technology Products
- [Identity Manager (HVID)](https://healthverity.com/identity-manager/)
- [HealthVerity FLOW](https://healthverity.com/flow/portal/)
- [Governance Manager](https://healthverity.com/governance-manager/)
- [Smart Contracts](https://healthverity.com/smart-contracts-manager/)
- Symphony
- Symphony Health Platform
- [Symphony Health Overview](https://healthverity.com/symphony-health/overview/)
- [Integrated Dataverse](https://healthverity.com/symphony-health/integrated-dataverse-and-source-solutions/)
- [Symphony HealthCloud](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/)
- Symphony Solutions
- [Health technology and business intelligence solutions](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/)
- [Integrated Dataverse and Source Solutions](https://healthverity.com/symphony-health/integrated-dataverse-and-source-solutions/)
- [Brand strategy and sales](https://healthverity.com/symphony-health/optimize-brand-strategy-and-sales-effectiveness/)
- [HealthRankings](https://healthverity.com/symphony-health/health-rankings/)
- [Market access](https://healthverity.com/symphony-health/solve-market-access-challenges/)
- [Patient Journey](https://healthverity.com/symphony-health/understand-the-patient-journey/)
- [Medicines affordability](https://healthverity.com/symphony-health/making-medicines-more-affordable/)
- Symphony Products
- [PayerSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PayerSource.pdf)
- [PrescriberSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PrescriberSource.pdf)
- [PatientSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PatientSource.pdf)
- [PatientSourceCompleteTM](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PatientSourceComplete.pdf)
- [PatientSourceDUOTM](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-PatientSourceDUO.pdf)
- [Symphony FLEXTM](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-Symphony-FLEX.pdf)
- [CustomerSource®](https://healthverity.com/symphony-health/integrated-dataverse-and-source-solutions/)
- [DiagnosticSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-DiagnosticSource.pdf)
- [NonRetailSource®](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-NonRetailSource.pdf)
- Symphony Technology
- [Symphony HealthCloud](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/)
- [Synoma Data Integration](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/)
- [Vantage Applications](https://blog.healthverity.com/hubfs/HealthVerity-Symphony-PDFs/HealthVerity-Symphony-Health-Factsheet-Vantage-Applications.pdf)
- [Metys](https://healthverity.com/symphony-health/health-technology-and-business-intelligence-solutions/metys/)
- Who We Serve
- [Life Sciences](https://healthverity.com/life-sciences/)
- [Government](https://healthverity.com/government/)
- [Insurance](https://healthverity.com/insurance/)
- [Channel Partners](https://healthverity.com/channel-partners/)
- [Data Ecosystem Partners](https://healthverity.com/data-ecosystem-partners/)
- Resources
- [HealthVerity Blog](https://blog.healthverity.com/)
- [News & Events](https://healthverity.com/resources/news-and-events/)
- [Resource Library](https://healthverity.com/resources/resource-library/)
- [Publications](https://healthverity.com/resources/publications/)
- [The IPGE Approach](https://healthverity.com/ipge-approach/)
- [Zombie Data Explained](https://healthverity.com/get-verified/)
- [Careers](https://healthverity.com/careers/)
- [About Us](https://healthverity.com/about-us/)
[Request Demo](https://healthverity.com/request-demo/)
[Contact](https://healthverity.com/contact/) [Sign In](https://marketplace.healthverity.com/)
[](https://healthverity.com/)
- How We Help
- Symphony
- Who We Serve
- Resources
- [Careers](https://healthverity.com/careers/)
- [About Us](https://healthverity.com/about-us/)
[Request Demo](https://healthverity.com/request-demo/)
Featured Content
###### HealthVerity MarketplaceTM
The nation’s largest healthcare and consumer data ecosystem
###### taXonomy Pathways
Unique features per therapeutic area with claims, labs and EHR including notes
###### HealthVerity eXOs
AI-native real-world evidence generation
###### Identity Manager (HVID)
Standalone de-identification as a service
Data Ecosystem
HealthVerity MarketplaceTM
HealthVerity TaxonomySM
HealthVerity Notes
Precision Event Alerts
HealthVerity Mastersets
Media Measurement
Syndicated Audiences
Insurance Underwriting
Technology Products
Identity Manager (HVID)
HealthVerity for Clinical Trial Linkage
HealthVerity for Commercial
Smart Contracts
HealthVerity for Registries
Symphony Health Platform
###### Symphony Health Overview
Seamless Data. Stronger Insights. Smarter Decisions.
###### Integrated Dataverse
Integrated Dataverse (IDV®) is de-identified claims data platform that combines medical, hospital, prescription, and demographic data into a single unifed view.
###### Symphony HealthCloud
Securely store, integrate, access, and analyze virtually unlimited volumes of data with Symphony HealthCloud.
Symphony Solutions
Health technology and business intelligence solutions
Integrated Dataverse and Source Solutions
Brand strategy and sales
HealthRankings
Market access
Patient Journey
Medicines affordability
Symphony Products
PayerSource®
PrescriberSource®
PatientSource®
PatientSourceCompleteTM
PatientSourceDUOTM
Symphony FLEXTM
CustomerSource®
DiagnosticSource®
NonRetailSource®
Symphony Technology
Symphony HealthCloud
Synoma Data Integration
Vantage Applications
Metys
###### Life Sciences
Technology solutions from development through commercialization
###### Channel Partners
HealthVerity enables partners to unlock unprecedented value and insights for clients
###### Government
Comprehensive and compliant solutions to manage the nation’s care
###### Data Ecosystem Partners
Join the nation’s largest data partner ecosystem
###### Insurance
Insurance-grade solutions that excel in managing risk and rebates
###### HealthVerity Blog
Insights and updates from industry experts
###### Publications
Peer-reviewed research and discoveries
###### News & Events
Stay informed about our latest announcements and happenings
###### The IPGE Approach
Fundamentals that power everything that we do and every solution we offer
###### Resource Library
Explore guides, webinars, and tools to enhance your understanding
###### Zombie Data Explained
Learn how data quality drives better research outcomes
# The risks of false negatives in healthcare data linkage
[By - Austin Eliazar, PhD, Chief Data Scientist,](https://blog.healthverity.com/author/austin-eliazar-chief-data-scientist-healthverity) June 7, 2019
#### Part 3: Measuring false negatives in anonymized healthcare data
\[Part 3\] of *Matching Accuracy Metrics |*
Catch up on [Part 1](https://blog.healthverity.com/matching-accuracy-metrics) and [Part 2,](https://blog.healthverity.com/indirect-measures-false-positives)or read [Part 4](https://blog.healthverity.com/matching-accuracy-metrics-indicator-measures)By: Austin Eliazar, Chief Data Scientist, [HealthVerity](https://medium.com/u/3603b8998f7b)
---
False negatives, the missed links between records, are the most obvious errors in a matching system. They can be easily seen in low match rates, short histories, and isolated records. However, it can be difficult to tease apart how many of these problems are coming from missed matches and how many are from honest gaps in the data coverage. In this section, we focus on the challenges introduced by false negatives and present solid ways to measure the false negatives without resorting to artificial scenarios like simulation or well-controlled ground truth.
**Read the full** [Matching Accuracy Metrics White Paper](https://info.healthverity.com/matching-accuracy-metrics-white-paper)
# What false negatives are and why they matter
False negatives in matching represent a job half done. Correlations between data sets are weak and only partially complete, leaving the data fragmented and isolated. This problem is compounded by each new data source, as each additional source provides a new set of broken links, leaving the data fragmented and siloed all over again. False negatives dilute the effectiveness of your data, negatively affecting your analysis in several ways:
**Short patient history:** Whether the false negatives happen across providers or within a single system, they can severely limit the duration of a patient history. Twenty years of historical data can be rendered moot if the false negative rate (FNR) is limiting the average patient history to three years. Short histories can significantly limit the effectiveness of the analysis, especially when looking for cause and effect, long term side effects and outcomes, or survivability.
[](https://info.healthverity.com/matching-accuracy-metrics-white-paper)
**Narrow patient history:** False negatives affect the width of the patient history even more significantly, making it difficult to get a full picture of the patient activity. Most analysis these days focus on the whole patient journey, with insights into medical, pharmacy, and lab data as a minimum set, with additional behavioral and purchase histories commonly being added. Adding more complexity, most patients will have history at multiple lab or pharmacy sites, further expanding the breadth of linking necessary. High FNR limits the number of additional data types that can be associated with the patient journey, giving a narrow, myopic view of the patients, leaving most of your valuable data behind.
**Failed indexes:** Weak linking affects the discovery process just as much as histories. Using a matching system for the initial discovery phase means FNR has a major impact on discovering re-enrollment, prior conditions, or data purchasing. In many of these cases a false negative is not just diluting the results, but it is providing false conclusions. This can have as much danger in skewing the results as a false positive, as the presence or absence of an event can be just as important as the result.
#### Using match rates to estimate false negatives
One approach to measuring the FNR is to look to match rates, leveraging the failed indexes as a measure of false negatives. By looking at the first time a new data source is matched against an existing base dataset, we can use the cold start match rate as an approximation of the false negatives. This provides an easy, obvious measurement, but is often noisy and easily biased. Extreme care must be taken to control for common sources of inaccuracy.
This measurement is computed using an existing base dataset — comprised of one or many different data sources — and a new incoming data source, which has never been included in the base dataset before. The incoming dataset is matched against the base dataset, then the number of shared identities between the two data sets is counted. Any records in the incoming data that don’t match to the base set are matched against each other to deduplicate these new identities. We calculate the total negative rate as
Total negative rate = (number of shared IDs) / (total number of incoming IDs)
Keep in mind that this rate includes both false negatives and true negatives. The key to this method is to reduce and remove as many sources of true negatives as possible, until all that is left is the FNR. Here are a few pointers to help with that:
**Regional coverage.** This measurement method only works well if there is good overlap between the data sets. Otherwise, the false negatives and the true negatives are intermingled. You can boost the overlap if you only use identifiable subsets — such as specific zip codes and/or age brackets — where the base data set has high coverage. Creating these subsets and comparing the identity counts against known census figures can help in finding sections with sufficiently high coverage.
**Baseline noise**. Certain amounts of noise are going to be present, even in a perfect matching system. Input data is not perfect and may have invalid records or dummy values included. Also, the population itself is in flux, with people moving in and out, plus new births and deaths. To get a good estimate, you need to subtract this baseline from the observed negatives. The baseline noise can be computed from outside sources, but it requires good external statistics combined with an accurate sense of the average update rate of individual records. Depending on your data sources, you could also estimate this from the incoming data feed itself, by matching the data set against itself. Although this baseline estimate will also be affected by false negative and false positives, data sources with a high record frequency and good internal consistency will have far fewer errors than matching across data sources.
**Repeated tests.** Each data source has its own eccentricities that can cause stark differences in initial matching. Issues of coverage, noisy input, invalid data, and dummy data can cause wide variation. That is why it is critical that this test is repeated over many different sources. The goal is to get an average sense of the FNR across all types of data sources. Using three different sources is a bare minimum to be aware of outliers, while eight or more are good to get a reliable estimate.
#### A probabilistic approach to measuring missed matches
In any random set of people, there is a chance that some pair of them will share the same birthday. This is not surprising since there are only 366 possible birthdays (including February 29). What may be surprising to you is how often this happens in small groups. In a group of 100 people, on average 24 of them will share a birthday with someone else. Even in a group as small as 23, there is a >50% chance that there will be at least one shared birthday. For just about every identifier in a matching system, from age to zip code, these duplications happen regularly and at a reliable rate. That’s part of why identity matching is so hard.
Fortunately, we can use these random duplicates to measure the false negative rate. The basic reasoning is simple — missed links result in extra data. We know how often duplicates should occur, so we can check and see how close the duplicate rates are in our matched identities. The number of missing duplicates is a measure of false positives.
The key to making this idea effective as a metric lies in choosing small groups, where each group would have held the identities of both of the people in a false positive if they hadn’t been merged together. Choosing groups along some of the more reliable identifiers is effective. For our example, we will consider a group of matched individuals who all share the same tokenized name. Of course, there are plenty of name tokens to choose from, so we can choose one with a reasonable size, say 100 individuals. Within this group, we use birthdays as our test field, and count the number of duplicate birthdays. We compare the observed number to the expected number of duplicates — in this case 24 — and figure out the difference. In some instances we will see more or less, just based on sampling errors. However, if we take the average over a large number of similarly sized groups, the noise dies down and any overcount is a sure sign of false negatives.
It is important to keep in mind that the number of duplicates is affected by false positives as well as false negatives. Incorrect data links often are triggered by these coincidental matches. These false positives then drive down the number of observed duplicates. When using this method to measure FNR, it is important to have a reliable estimate of FPR from an independent test (such as the post mortem activity, [as described in our previous blog](https://blog.healthverity.com/indirect-measures-false-positives)).
FNR = (average observed duplicates) — (expected duplicates) + FPR
**Duplicate rates:** The rate of duplicates in a group depends on the size of the group, the range of the test values, and the natural distribution of those values. You can find the expected value for the number of duplicates using mathematical equations, but these often assume the values are uniformly distributed. If you have enough individuals in the matching system (which most matching systems do), you can measure the value distribution empirically from the data and simulate appropriately sized sample groups. The FNR is assumed to be low enough and largely independent of the test field so as to not significantly skew the results.
**Sample errors: **For testing, more is better.The number of groups that you use in your average affects the precision of this test, and using more groups is more reliable. Sample error calculators are very helpful to quantify your margin of error.
**Selection bias: **Keep in mind that the choice of fields to use — both for group selection and for duplicate measurement — can affect your results. Really, the example above is only measuring the number of false positives where the name and birthday matched. If the source data for one of these values is often noisy, you will miss any false negatives that are made outside of your selected group. Repeating this test across different pairs of fields can give different views of the FNR, and even give insights into where the system may be stronger or weaker.
*Continue reading [Part 4](https://blog.healthverity.com/matching-accuracy-metrics-indicator-measures).*
**Interested in learning more about matching accuracy****?**
[Get the full white paper here](https://info.healthverity.com/matching-accuracy-metrics-white-paper).
###### Category
- [Solutions & Best Practices](https://blog.healthverity.com/tag/solutions-best-practices)
- [Trends & Insights](https://blog.healthverity.com/tag/trends-insights)
- [HEOR](https://blog.healthverity.com/tag/heor)
- [Thought Leadership](https://blog.healthverity.com/tag/thought-leadership)
- [Marketplace](https://blog.healthverity.com/tag/marketplace)
- [taXonomy](https://blog.healthverity.com/tag/taxonomy)
- [Industry Updates](https://blog.healthverity.com/tag/industry-updates)
- [Case Studies](https://blog.healthverity.com/tag/case-studies)
- [Commercial Pharmaceutical](https://blog.healthverity.com/tag/commercial-pharmaceutical)
- [Government](https://blog.healthverity.com/tag/government)
- [AI](https://blog.healthverity.com/tag/ai)
- [eXOs](https://blog.healthverity.com/tag/exos)
- [Audiences](https://blog.healthverity.com/tag/audiences)
- [taXonomy Pathways](https://blog.healthverity.com/tag/taxonomy-pathways)
- [Notes](https://blog.healthverity.com/tag/notes)
- [Rare Diseases](https://blog.healthverity.com/tag/rare-diseases)
###### Popular Post
- [The GLP-1 trend: What the real-world data reveals](https://blog.healthverity.com/the-glp-1-surge-what-the-real-world-data-reveals)
- [GLP-1 trends 2025: real-world data, patient outcomes & future therapies](https://blog.healthverity.com/glp-1-trends-2025-real-world-data-patient-outcomes-future-therapies)
- [Open claims vs. closed payer claims: What’s the difference?](https://blog.healthverity.com/open-claims-vs-closed-payer-claims)
- [What is Privacy Preserving Record Linkage (PPRL)?](https://blog.healthverity.com/what-is-privacy-preserving-record-linkage)
- [GLP‑1 real‑world data for obesity: A new standard for metabolic care](https://blog.healthverity.com/glp-1-real-world-data-for-obesity-a-new-standard-for-metabolic-care)
###### Subscribe
### Similar Blogs
[](https://blog.healthverity.com/indirect-measures-false-positives)
#### [The risks of false positives in healthcare data linkage](https://blog.healthverity.com/indirect-measures-false-positives)
By [Austin Eliazar, PhD, Chief Data Scientist](https://blog.healthverity.com/author/austin-eliazar-chief-data-scientist-healthverity) 23 May 2019
Part 2: Understanding false positives in patient data matching \[Part 2\] ofMatching Accuracy Metrics
[READ MORE](https://blog.healthverity.com/indirect-measures-false-negatives)
[](https://blog.healthverity.com/matching-accuracy-metrics-indicator-measures)
#### [Beyond the numbers: indirect ways to assess patient matching accuracy metrics](https://blog.healthverity.com/matching-accuracy-metrics-indicator-measures)
By [Austin Eliazar, PhD, Chief Data Scientist](https://blog.healthverity.com/author/austin-eliazar-chief-data-scientist-healthverity) 4 August 2024
Part 4: Indirect and qualitative metrics for matching accuracy \[Part 4\] of Matching Accuracy...
[READ MORE](https://blog.healthverity.com/indirect-measures-false-negatives)
[](https://blog.healthverity.com/matching-accuracy-metrics)
#### [How to evaluate the performance of a de-identification solution](https://blog.healthverity.com/matching-accuracy-metrics)
By [Austin Eliazar, PhD, Chief Data Scientist](https://blog.healthverity.com/author/austin-eliazar-chief-data-scientist-healthverity) 8 May 2019
Part 1: Matching accuracy metrics for patient tokenization in healthcare \[Part 1\] ofMatching...
[READ MORE](https://blog.healthverity.com/indirect-measures-false-negatives)
#### Who we serve
- [Life Sciences](https://healthverity.com/life-sciences/)
- [Government](https://healthverity.com/government/)
- [Insurance](https://healthverity.com/insurance/)
- [Channel Partners](https://healthverity.com/channel-partners/)
- [Data Ecosystem Partners](https://healthverity.com/data-ecosystem-partners/)
#### How we help
- [HealthVerity Marketplace](https://healthverity.com/marketplace/)
- [HealthVerity Mastersets](https://healthverity.com/mastersets/)
- [HealthVerity FLOW](https://healthverity.com/flow/)
- [The IPGE Approach](https://healthverity.com/ipge-approach/)
#### Resources
- [HealthVerity Blog](https://blog.healthverity.com/)
- [News & Events](https://healthverity.com/resources/news-and-events/)
- [Resource Library](https://healthverity.com/resources/resource-library/)
- [Publications](https://healthverity.com/resources/publications/)
#### Our company
- [About Us](https://healthverity.com/about-us/)
- [Leadership](https://healthverity.com/about-us/#leadership)
- [Board of Directors](https://healthverity.com/about-us/#board)
- [Careers](https://healthverity.com/careers/)
- [Contact](https://healthverity.com/contact/)
©2026 HealthVerity, Inc. | [Terms & Conditions](https://healthverity.com/terms-and-conditions/) | [Privacy Policy](https://healthverity.com/privacy-policy/)
