Patching a DAG without downtime: Exchange Server maintenance mode step by step

Patching a member of a database availability group (DAG) is routine, but one skipped step is all it takes to strand mail in the queue of a rebooting server, leave a node able to grab the Primary Active Manager role, or end up with databases that never move back. In this post I'll walk through Microsoft's supported maintenance-mode sequence for Exchange Server SE, explain what each command actually does, and show what "good" looks like before and after you patch.

How maintenance mode works#

Three layers on a DAG member have to be quiet before you install anything:

• Transport. Setting the HubTransport component to Draining stops the server accepting new messages, and Redirect-Message hands whatever is still queued to another Mailbox server.
• The DAG and its cluster. StartDagServerMaintenance.ps1 blocks database activation on the server, pauses its cluster node so it can't hold or take the Primary Active Manager (PAM) role, moves every active database to other members and, if the server owns the cluster core group, moves that too. If any step fails, the script undoes everything except database moves that already succeeded.
• Client access. Setting ServerWideOffline to Inactive switches off the remaining components, so client protocols and health probes stop treating the server as available.

Every component state is recorded against a requester. Use -Requester Maintenance on the way in and on the way out: a component that one requester set to inactive stays inactive until that same requester sets it active again.

Prerequisites#

• An elevated Exchange Management Shell on the server you're patching (EX01 in the examples). The DAG scripts can also run remotely, but the machine running them needs the Failover Cluster Management tools (RSAT-Clustering).
• A healthy passive copy of every database that's active on EX01, and another Mailbox server (EX02.contoso.com here) to receive redirected mail.
• The CU or SU that matches your current build, downloaded in advance and checked against Microsoft's Exchange build numbers page.
• A note of EX01's current activation policy, because the stop script resets it to Unrestricted.

Start with a quick health check:

Get-MailboxDatabase | Get-MailboxDatabaseCopyStatus |
  Format-Table Name,Status,CopyQueueLength,ReplayQueueLength -AutoSize
Test-ReplicationHealth -Identity EX01
Get-MailboxServer EX01 | Format-List DatabaseCopyAutoActivationPolicy

Good looks like every active copy Mounted, every passive copy Healthy, queue lengths at or near zero and every replication check Passed. If any database has only one healthy copy, fix that before you start.

Step-by-step#

1. Drain transport#

Set-ServerComponentState EX01 -Component HubTransport -State Draining -Requester Maintenance
Restart-Service MSExchangeTransport

The restart makes the Transport service pick up the draining state straight away. Microsoft's procedure restarts only MSExchangeTransport; Front End Transport is a stateless proxy and doesn't queue mail.

2. Move databases and pause the cluster node#

cd $ExScripts
.\StartDagServerMaintenance.ps1 -ServerName EX01 -MoveComment Maintenance -PauseClusterNode

The script can run for several minutes without printing anything, so be patient. Both DAG scripts accept -WhatIf if you want a dry run first.

3. Redirect queued messages#

Redirect-Message -Server EX01 -Target EX02.contoso.com -Confirm:$false
Get-Queue -Server EX01 | Where-Object { $_.MessageCount -gt 0 }

The target must be the FQDN of another Mailbox server that isn't in maintenance. Repeat the Get-Queue check until it returns nothing, or only Shadow Redundancy queues.

4. Take the server offline#

Set-ServerComponentState EX01 -Component ServerWideOffline -State Inactive -Requester Maintenance

If your load balancer probes each protocol's healthcheck.htm page, those probes now fail and EX01 drops out of the pool. If it only checks that a port is open, disable EX01 in the load balancer yourself.

5. Confirm, then patch#

Get-ServerComponentState EX01 | Format-Table Component,State -AutoSize
Get-MailboxServer EX01 | Format-List DatabaseCopyAutoActivationPolicy
Get-ClusterNode EX01 | Format-List Name,State
Get-MailboxDatabaseCopyStatus -Server EX01 | Where-Object { $_.Status -eq "Mounted" }

You're ready when only Monitoring and RecoveryActionsEnabled are still active, the policy shows Blocked, the node is Paused and the last command returns nothing. Now install from an elevated prompt: the .msp file for a security update, or Setup.exe /Mode:Upgrade /IAcceptExchangeServerLicenseTerms_DiagnosticDataON (or _DiagnosticDataOFF) for a cumulative update. Restart as often as the update needs; the paused node and the component states survive reboots.

6. Bring the server back#

Set-ServerComponentState EX01 -Component ServerWideOffline -State Active -Requester Maintenance
cd $ExScripts
.\StopDagServerMaintenance.ps1 -ServerName EX01
Set-ServerComponentState EX01 -Component HubTransport -State Active -Requester Maintenance
Restart-Service MSExchangeTransport

The stop script resumes the cluster node, sets the activation policy to Unrestricted and resumes the server's database copies. If your original policy was different, put it back, for example Set-MailboxServer EX01 -DatabaseCopyAutoActivationPolicy IntrasiteOnly. If you use POP3 or IMAP4, make sure the PopProxy and ImapProxy components are active, then start those services manually.

7. Rebalance, then move to the next server#

Since Exchange 2016 CU2, the DAG's PreferenceMoveFrequency setting rebalances active databases automatically. To rebalance immediately instead:

cd $ExScripts
.\RedistributeActiveDatabases.ps1 -DagName DAG01 -BalanceDbsByActivationPreference

Then repeat the sequence on the next member. Patch one server at a time, and don't leave members on different builds for longer than the rollout takes.

Verify#

Get-ServerComponentState EX01 | Where-Object { $_.State -ne "Active" }
Get-ClusterNode | Format-Table Name,State
Get-MailboxDatabaseCopyStatus -Server EX01 |
  Format-Table Name,Status,CopyQueueLength,ReplayQueueLength -AutoSize
Test-ServiceHealth EX01
Get-Command ExSetup.exe | ForEach-Object { $_.FileVersionInfo.FileVersion }

Good looks like this:

• Only ForwardSyncDaemon and ProvisioningRps are listed; both are inactive by default on-premises.
• Every cluster node is Up.
• EX01's copies are Healthy or Mounted, with low queue lengths.
• RequiredServicesRunning is True for every role listed.
• The ExSetup version matches the build Microsoft lists for the update you installed, and a test message flows through EX01.

Tips and gotchas#

• Components still inactive after a failed update? Setup records its own requester. Microsoft's fix is to set ServerWideOffline, Monitoring and RecoveryActionsEnabled back to Active with -Requester Functional. The LocalStates and RemoteStates properties of Get-ServerComponentState show which requester set each state.
• Two-member DAGs. While one member is paused or rebooting, the other carries every database with no redundancy, and quorum depends on the file share witness. Check that the witness server is reachable before you reboot.
• Prefer the scripts to hand-typed sequences. They handle the PAM move and the activation block for you, and the start script rolls itself back if something fails.
• Review mitigations after patching. An update can make an Emergency Mitigation service mitigation unnecessary; see Exchange Emergency Mitigation service and AMSI for the checks.

References#

• Manage database availability groups in Exchange Server (Microsoft Learn)
• Set-ServerComponentState (Microsoft Learn)