Too many Purview DLP false positives: tuning confidence, instance counts and exceptions

A DLP policy that cries wolf trains users to click Override without reading and buries the real incidents in noise. False positives almost always come from a rule that accepts too little evidence: a low confidence level, a single instance, or a sensitive information type that doesn't fit your data. In this post I'll show how to find out exactly what matched, then tighten the rule step by step without losing the detections you need.

Symptoms#

Users report Microsoft's default DLP messages on content they're sure isn't sensitive:

Your email conflicts with a policy in your organization.
Your email message conflicts with a policy in your organization. The message wasn't delivered to all recipients.
This item conflicts with a policy in your organization.
Access to this item is blocked for everyone except its owner, last modifier, and the primary site admin.
This file conflicts with a policy in your organization. Go to the File menu for more information.

Around them you see the same pattern: alerts full of invoice numbers, timestamps or test data, simulation results the data owners reject, and a growing number of overrides and false-positive reports on one rule.

Why it happens#

• Low confidence is accepted. Sensitive information types (SITs) have patterns at high, medium and low confidence (85, 75 and 65). A low-confidence pattern usually needs little or no supporting evidence, so any number of the right shape can match.
• One instance is enough. With a minimum instance count of 1, a single stray match triggers the rule.
• Proximity is too generous. Supporting keywords anywhere in a long document can make an unrelated number look sensitive. Microsoft advises against unlimited proximity for exactly this reason.
• The SIT doesn't fit the data. Generic patterns such as dates or national ID formats collide with part numbers, order IDs and log files.
• The tip comes from another rule. Users only see the tip from the most restrictive, highest-priority matching rule, and that includes policies still in simulation mode. A test policy can be the one producing the complaints.

How to fix it#

1. Find exactly what matched#

Open the alert or the event in activity explorer and note the policy, the rule and the sensitive information types detected; see investigating DLP alerts. Then reproduce the match with a copy of the content in Security & Compliance PowerShell:

Connect-IPPSSession
# Text sample: shows each detected type with its count and confidence
$r = Test-DataClassification -TextToClassify (Get-Content "C:\Temp\sample.txt" -Raw)
$r.ClassificationResults

# Whole file: extract the text first, then classify it
$x = Test-TextExtraction -FileData ([System.IO.File]::ReadAllBytes("C:\Temp\sample.docx"))
Test-DataClassification -TestTextExtractionResults $x.ExtractedResults

# What the rule asks for today
Get-DlpComplianceRule -Policy "DLP - Customer data" | Format-List Name,Priority,ContentContainsSensitiveInformation

Compare the confidence and count the engine reports with what the rule accepts. That tells you which of the next steps applies.

2. Raise the confidence level and instance count#

Edit the rule, open the Content contains › Sensitive information types condition and, for each type, require High confidence (or at least Medium) and a realistic minimum instance count. A pattern that works well splits the work across rules: a few instances only notify, more instances shared externally block. Microsoft's own example uses five instances as the dividing line between notifying and blocking.

3. Add narrow exceptions#

Use the rule's exception conditions for legitimate flows you can name: a partner's recipient domain, a system account, a document name pattern, a specific group. Keep each exception as small as possible, document why it exists and who approved it, and never exclude a whole location to silence a noisy rule.

4. Fix the classifier instead of the rule#

When the SIT itself is wrong for your data, copy the built-in type and edit the copy, or create a custom one (see custom sensitive information types). Microsoft's false-positive guidance recommends:

• Supporting elements: require keywords such as "DOB" near a date, so headers and timestamps stop matching.
• Proximity of 300 characters for new custom types, not "anywhere in the document". For large tabular data, the RelaxProximity option is set through PowerShell.
• Exclusions: exclude specific values, prefixes or suffixes that you know are harmless.
• Better technology for known data: exact data match for customer or employee records, and trainable classifiers for content without a fixed pattern.

5. Test before you publish#

Use the SIT's test function in the portal or the on-demand classification capability against a set of known-good and known-bad samples. Keep the samples; they become your regression test for every future change. In content explorer, mark SharePoint and OneDrive items as a match or not a match. That feedback can include a redacted copy for Microsoft to improve built-in types.

6. Turn overrides into tuning data#

In the rule's policy tip settings, allow users to report a false positive and override (in PowerShell, -NotifyAllowOverride with FalsePositives, WithJustification or WithoutJustification; it only offers an override when the rule also blocks access). In Outlook the path is Show Details › Override, then a business justification or a false-positive report. These reports are logged, so review them per rule every month. A rule with many reports needs tuning, not more user training.

Verify the fix#

• Test-DataClassification on your known false-positive samples no longer returns the type at the confidence and count the rule requires, while genuine samples still do.
• Put the edited policy back into simulation mode for a cycle; see from simulation mode to enforcement. Matches per day and false-positive reports should drop, and the data owners should agree with what remains.
• Re-run the user's original scenario: no policy tip on the harmless content, the expected tip on real sensitive data.

Prevent it next time#

• Start every new policy in simulation mode, scoped to a pilot group first.
• Give each rule one purpose and record the confidence, instance count and exceptions you chose, with the reason.
• Prefer exact data match or labels as conditions when you know precisely what must be protected.
• Remember that the tip users see belongs to the most restrictive matching rule, including test policies, so retire abandoned simulations.

References#

• Reduce false positives, Step 1: Configure pattern-based sensitive data detection (Microsoft Learn)
• Send email notifications and show policy tips for DLP policies (Microsoft Learn)
• Test-DataClassification (Microsoft Learn)