title: "Presidio False Positives: What They Cost in Legal and Healthcare" description: "A 2024 benchmark found Presidio generated 13,536 false positive name detections across 4,434 samples — flagging pronouns, vessel names, and countries as person names. Here is what that costs in legal and healthcare environments." category: technical publishedAt: 2026-03-23 tags:
- Presidio false positive rate
- PII detection precision
- automated redaction cost
- legal document review
- hybrid PII detection readingTime: 8
Updated for 2026
The 22.7% Precision Problem
A 2024 study tested Microsoft Presidio on business files. Presidio is an open-source PII tool. Legal teams and health groups use it widely.
The study measured how often Presidio was right. Of all the items it flagged as person names, how many were actually person names?
The answer was 22.7%. About 77 out of every 100 flags were wrong. The study counted 13,536 false flags across 4,434 sample files.
The errors were not random. They followed clear patterns:
- Pronouns flagged as people ("I" at the start of a sentence)
- Ship labels flagged as people ("ASL Scorpio")
- Company labels flagged as people ("Deloitte & Touche")
- Country terms flagged as people ("Argentina," "Singapore")
None of these are rare edge cases. They show up whenever a general NLP model meets domain-specific text. The model was not built to tell them apart.
What False Flags Cost
In legal and health work, every flag needs a response. Teams face three options. All three have real costs.
Option 1: A human checks every flag. Lawyer and expert time runs $200 to $800 per hour. At 22.7% accuracy, the volume is huge. This is not viable at scale. See eDiscovery PII Automation and Legal Review Cost Reduction for how review costs grow with volume.
Option 2: Skip review and trust the output. This is also risky. When 77% of "redacted" items are not sensitive, you create legal risk. Courts have fined lawyers for over-redaction. See eDiscovery Over-Redaction Sanctions for documented cases.
Option 3: Raise the score threshold. Presidio lets users set a score_threshold to drop weak flags. A 2024 DICOM study tested this at 0.7 — a fairly high bar. The result: 38 out of 39 DICOM images still had false flags. Thresholds help. They do not fix the root cause.
Why General NLP Struggles Here
The Presidio gap comes from a mismatch between training data and real-world use.
Legal files are full of capital-letter terms. Case names, law titles, and exhibit codes all look like personal data to a general model. It flags them. Most are not personal data.
Health files add drug names, device codes, and clinical short forms. "Pt." means Patient. "Dr." means Doctor. These trip up entity detection in ways that are hard to predict.
Finance files have product codes, entity strings, and account IDs that share surface patterns with personal records.
Fine-tuning a model on domain data helps. But it takes time and effort to build and keep up to date.
How Hybrid Detection Fixes This
The false flag problem has a clear fix. Split the work by data type.
Pattern rules for structured data. Social security numbers, phone numbers, email addresses, and ID formats follow fixed rules. A string either fits the pattern and passes a check digit test, or it does not. Zero false flags for valid rule sets.
Language models for free text. First and last names, company labels, and locations in prose lack rigid structure. NLP finds them when rules cannot. Confidence scores and context checks cut the false flag rate.
Per-type score settings for fine control. Legal teams that cannot risk over-redaction set high thresholds for fuzzy matches. Research teams that need high recall set lower ones. See Binary PII Detection and Confidence Scoring for Compliance for how score tiers work in practice.
The result is far fewer errors than Presidio defaults. Recall stays strong where rules alone would miss too much.
For legal and health teams, the key question is not whether false flags exist. They always do in NLP systems. The question is whether the tool lets you set, measure, and document the tradeoff.