Exact data match (EDM) in Purview: schema, hashing, upload and the errors that stop it

Exact data match (EDM) lets Microsoft Purview DLP recognize your actual customer or employee records instead of anything that merely looks like an ID number. It's also the classifier with the most moving parts: a schema, an EDM sensitive information type (SIT), a security group, the EDM Upload Agent, hashing, indexing and testing. In this post I'll walk through where each step breaks, the messages you'll see, and how to prove the data is uploaded and matching.

Symptoms#

• In the new EDM experience, the upload status shows a failure instead of Data upload complete and Indexing complete.
• An audit search or alert shows the UploadDataFailed activity.
• EdmUploadAgent.exe stops with an error, or the upload works but the EDM SIT never matches in testing, DLP or auto-labeling.

Data upload failed
Data indexing failed
UploadDataFailed

Admins have also reported this agent error on Microsoft Q&A when running /UploadData:

Command failed. Error Type: System.Configuration.ConfigurationErrorsException Error Message: Invalid AllowedBadLinesPercentage setting: .

Why it happens#

EDM is a chain, and each link has its own typical failure:

That last row matters most: Microsoft calls an incorrectly defined SIT on the primary element the most common cause of EDM detection failures. A regex that includes surrounding whitespace (\s), or captures only part of a value, produces text that never matches the hash.

How to fix it#

1. Fix the account and authorization#

Add the uploading account to EDM_DataUploaders (a Groups Administrator can do this). On the upload computer (Windows 10 or later, Windows Server 2019 or later, or Windows Server 2016 with .NET 4.6.2), open an elevated Command Prompt in the agent's install folder and run EdmUploadAgent.exe /Authorize, signing in with that account.

2. Download the current schema and validate the data#

cd "C:\Program Files\Microsoft\EdmUploadAgent"
EdmUploadAgent.exe /SaveSchema /DataStoreName PatientRecords /OutputDir C:\EDM\Schema
EdmUploadAgent.exe /ValidateData /DataFile C:\EDM\Data\PatientRecords.csv /Schema C:\EDM\Schema\PatientRecords.xml

Re-download the schema after every schema edit. If validation complains, fix the export rather than the agent: surround values that contain apostrophes or commas (Tom O'Neil, 's-Gravenhage) with double quotes, or better, export tab-delimited and use the /ColumnSeparator option. For messy CSV quoting, the agent's config file also supports the optional ParseQuotesExhaustiveScenariosByDefault setting.

3. Hash and upload with full paths#

Microsoft recommends two computers: hash on a secure one, upload from an internet-connected one, with the same agent version on both. Always give full paths and an explicit bad-lines threshold:

REM Secure computer
EdmUploadAgent.exe /CreateHash /DataFile C:\EDM\Data\PatientRecords.csv /HashLocation C:\EDM\Hash /Schema C:\EDM\Schema\PatientRecords.xml /AllowedBadLinesPercentage 5
REM Upload computer (copy the .EdmHash and .EdmSalt files across first)
EdmUploadAgent.exe /UploadHash /DataStoreName PatientRecords /HashFile C:\EDM\Hash\PatientRecords.EdmHash

On a single computer, /UploadData combines both steps with the same /DataFile, /HashLocation, /Schema and /AllowedBadLinesPercentage switches. For the reported Invalid AllowedBadLinesPercentage setting error, use the current agent and pass the switch with a value, as Microsoft's syntax does.

4. Check the session, then fix detection#

EdmUploadAgent.exe /GetDataStore
EdmUploadAgent.exe /GetSession /DataStoreName PatientRecords

If the upload completed but nothing matches:

• Test the SIT used by each primary element on its own first. If it misses values or grabs extra characters, use \b instead of \s and capture the whole value (see custom sensitive information types).
• Avoid loosely defined SITs (any number, any email address) as primary elements; the new experience warns about them.
• Multi-word values in evidence columns need multi-token matching or a dictionary-based SIT.
• Check proximity: evidence keywords far from the matched rows don't count.
• Existing SharePoint and OneDrive files are only reclassified when modified or reindexed, so request a recrawl.

Verify the fix#

1. /GetDataStore lists the store with a fresh last-updated time, and the portal shows Indexing complete.
2. Wait at least an hour after indexing, or after any EDM SIT change, before testing.
3. In the Microsoft Purview portal, open Information Protection › Classifiers › EDM classifiers (with the New EDM experience toggle on), select the SIT, choose Test and upload a file with a few real rows. In the classic experience, test from Sensitive info types instead.
4. Or test text from PowerShell: Test-DataClassification -ClassificationNames "Patient records EDM" -TextToClassify "...".

Good looks like a match for the EDM SIT itself, not only for its primary element SITs (results show both). Then add it to a DLP policy in simulation; see designing Purview DLP policies.

Prevent it next time#

• Automate the refresh with Task Scheduler (Microsoft publishes a sample script), keeping the same file name and location and staying within five refreshes in 24 hours.
• Alert on failures (worldwide and GCC clouds):
  PowerShell

  New-ProtectionAlert -Name "EdmUploadFailAlertPolicy" -Category Others -NotifyUser dlp-team@contoso.com -ThreatType Activity -Operation UploadDataFailed -Description "EDM upload failed" -AggregationType None -Severity High

• Keep sample data harmless: the new experience can build the schema from a 10–20 row sample file, which isn't hashed, so use made-up values in the real format.
• Licensing: Microsoft lists EDM-based classification in Office 365 E5, Microsoft 365 E5, Microsoft Purview Suite and Microsoft E5/A5 Information Protection and Governance.

References#

• Hash and upload the sensitive information source table for EDM SITs (Microsoft Learn)
• Test an exact data match sensitive information type (Microsoft Learn)
• Create notifications for exact data match activities (Microsoft Learn)