Too many Purview DLP false positives: tuning confidence, instance counts and exceptions

Symptoms to confirm

• Users report Microsoft's default DLP messages on content they're sure isn't sensitive:

Your email conflicts with a policy in your organization.
Your email message conflicts with a policy in your organization. The message wasn't delivered to all recipients.
This item conflicts with a policy in your organization.
Access to this item is blocked for everyone except its owner, last modifier, and the primary site admin.
This file conflicts with a policy in your organization. Go to the File menu for more information.

• Around them you see the same pattern: alerts full of invoice numbers, timestamps or test data, simulation results the data owners reject, and a growing number of overrides and false-positive reports on one rule.

Likely causes

• Low confidence is accepted. Sensitive information types (SITs) have patterns at high, medium and low confidence (85, 75 and 65). A low-confidence pattern usually needs little or no supporting evidence, so any number of the right shape can match.
• One instance is enough. With a minimum instance count of 1, a single stray match triggers the rule.
• Proximity is too generous. Supporting keywords anywhere in a long document can make an unrelated number look sensitive. Microsoft advises against unlimited proximity for exactly this reason.
• The SIT doesn't fit the data. Generic patterns such as dates or national ID formats collide with part numbers, order IDs and log files.
• The tip comes from another rule. Users only see the tip from the most restrictive, highest-priority matching rule, and that includes policies still in simulation mode. A test policy can be the one producing the complaints.

Checklist

1. 1

   Find exactly what matched

   Open the alert or the event in activity explorer and note the policy, the rule and the sensitive information types detected; see investigating DLP alerts. Then reproduce the match with a copy of the content in Security & Compliance PowerShell:

   powershell

   Connect-IPPSSession
   # Text sample: shows each detected type with its count and confidence
   $r = Test-DataClassification -TextToClassify (Get-Content "C:\Temp\sample.txt" -Raw)
   $r.ClassificationResults
   
   # Whole file: extract the text first, then classify it
   $x = Test-TextExtraction -FileData ([System.IO.File]::ReadAllBytes("C:\Temp\sample.docx"))
   Test-DataClassification -TestTextExtractionResults $x.ExtractedResults
   
   # What the rule asks for today
   Get-DlpComplianceRule -Policy "DLP - Customer data" | Format-List Name,Priority,ContentContainsSensitiveInformation

2. 2

   Raise the confidence level and instance count

   Edit the rule, open the Content contains › Sensitive information types condition and, for each type, require High confidence (or at least Medium) and a realistic minimum instance count. A pattern that works well splits the work across rules: a few instances only notify, more instances shared externally block.

3. 3

   Add narrow exceptions

   Use the rule's exception conditions for legitimate flows you can name: a partner's recipient domain, a system account, a document name pattern, a specific group. Keep each exception as small as possible, document why it exists and who approved it, and never exclude a whole location to silence a noisy rule.

4. 4

   Fix the classifier instead of the rule

   When the SIT itself is wrong for your data, copy the built-in type and edit the copy, or create a custom one (see custom sensitive information types). Microsoft's false-positive guidance recommends:

   • Supporting elements: require keywords such as "DOB" near a date, so headers and timestamps stop matching.
   • Proximity of 300 characters for new custom types, not "anywhere in the document". For large tabular data, the RelaxProximity option is set through PowerShell.
   • Exclusions: exclude specific values, prefixes or suffixes that you know are harmless.
   • Better technology for known data: exact data match for customer or employee records, and trainable classifiers for content without a fixed pattern.
5. 5

   Test before you publish

   Use the SIT's test function in the portal or the on-demand classification capability against a set of known-good and known-bad samples. Keep the samples; they become your regression test for every future change.

6. 6

   Turn overrides into tuning data

   In the rule's policy tip settings, allow users to report a false positive and override (in PowerShell, -NotifyAllowOverride with FalsePositives, WithJustification or WithoutJustification; it only offers an override when the rule also blocks access). In Outlook the path is Show Details › Override, then a business justification or a false-positive report.

Verify

• Test-DataClassification on your known false-positive samples no longer returns the type at the confidence and count the rule requires, while genuine samples still do.
• Put the edited policy back into simulation mode for a cycle; see from simulation mode to enforcement. Matches per day and false-positive reports should drop, and the data owners should agree with what remains.
• Re-run the user's original scenario: no policy tip on the harmless content, the expected tip on real sensitive data.

Prevent it next time

• Start every new policy in simulation mode, scoped to a pilot group first.
• Give each rule one purpose and record the confidence, instance count and exceptions you chose, with the reason.
• Prefer exact data match or labels as conditions when you know precisely what must be protected.
• Remember that the tip users see belongs to the most restrictive matching rule, including test policies, so retire abandoned simulations.