DFS Replication in Windows Server 2012 R2: If You Only Knew the Power of the Dark Shell

Hi folks, Ned here again. By now, you know that DFS Replication has some major new features in Windows Server 2012 R2. Today we dig into the most comprehensive new feature, DFSR Windows PowerShell.

Yes, your thoughts betray you

“IT pros have strong feelings about Windows PowerShell, but if they can be turned, they’d be a powerful ally. “

- Darth Ned

It’s not surprising if you’re wary. We’ve been beating the Windows PowerShell drum for years now, but sometimes, new cmdlets don’t offer better ways to do things, only different ways. If you were already comfortable with the old command-line tools or attached to the GUI, why bother learning more of the same? I spent many years in the field before I came to Redmond and I’ve felt this pain.

As the DFSR development team, we wanted to be part of the solution. It led to a charter for our Windows PowerShell design process:

1. The old admin tools work against one node at a time – DFSR Windows PowerShell should scale without extensive scripting.

2. Not everyone is a DFSR expert – DFSR Windows PowerShell should default to the recommended configuration.

3. Parity with old tools is not enough – DFSR Windows PowerShell should bring new capabilities and solve old problems.

We then devoted ourselves to this, sometimes arguing late into the night about a PowerShell experience that you would actually want to use.

Today we walk through all of these new capabilities and show you how, with our combined strength, we can end this destructive conflict and bring order to the galaxy.

Join me, and I will complete your training

Let’s start with the simple case of creating a replication topology with two servers that will be used to synchronize a single folder. In the old DFSR tools, you would have two options here:

1. Run DFSMGMT.MSC, browsing and clicking your way through adding the servers and their local configurations.

2. Run the DFSRADMIN.EXE command-line tool N times, or run N arguments as part of the BULK command-line option.

To setup only two servers with DFSMGMT, I have to go through all these dialogs:

To setup a simple hub and two-spoke environment with DFSRADMIN, I need to run these 12 commands:

dfsradmin rg new /rgname:"software"

dfsradmin rf new /rgname:software /rfname:rf01

dfsradmin mem new /rgname:software /memname:srv01

dfsradmin mem new /rgname:software /memname:srv02

dfsradmin mem new /rgname:software /memname:srv03

dfsradmin conn new /rgname:software /sendmem:srv01 /recvmem:srv02

dfsradmin conn new /rgname:software /sendmem:srv02 /recvmem:srv01

dfsradmin conn new /rgname:software /sendmem:srv01 /recvmem:srv03

dfsradmin conn new /rgname:software /sendmem:srv03 /recvmem:srv01

dfsradmin membership set /rgname:software /rfname:rf01 /memname:srv01 /localpath:c:\rf01 /isprimary:true

dfsradmin membership set /rgname:software /rfname:rf01 /memname:srv02 /localpath:c:\rf01

dfsradmin membership set /rgname:software /rfname:rf01 /memname:srv03 /localpath:c:\rf01

Facepalm. Instead of making bulk operations easier, the DFSRADMIN command-line has given me nearly as many steps as the GUI!

Worse, I have to understand that the options presented by these old tools are not always optimal – for instance, DFS Management creates the memberships disabled by default, so that there is no replication. The DFSRADMIN tool requires remembering to create connections in both directions; if I don’t, I have created an unsupported and disconnected topology that may eventually cause data loss problems. These are major pitfalls to DFSR administrators, especially when first learning the product.

Now watch this with DFSR Windows PowerShell:

New-DfsReplicationGroup -GroupName "RG01" | New-DfsReplicatedFolder -FolderName "RF01" | Add-DfsrMember -ComputerName SRV01,SRV02,SRV03

I just added RG, RF, and members with one pipelined command with minimal repeated parameters, instead of five individual commands with repeated parameters. If you are really new to Windows PowerShell, I suggest you start here to understand pipelining. Now:

Add-DfsrConnection -GroupName "rg01" -SourceComputerName srv01 -DestinationComputerName srv02

Add-DfsrConnection -GroupName "rg01" -SourceComputerName srv01 -DestinationComputerName srv03

I just added the hub and spoke connections here with a pair of commands instead of four, as the PowerShell creates bi-directionally by default instead of one-way only. Now:

Set-DfsrMembership -GroupName "rg01" -FolderName "rf01" -ComputerName srv01 -ContentPath c:\rf01 –PrimaryMember $true

Set-DfsrMembership -GroupName "rg01" -FolderName "rf01" -ComputerName srv02,srv03 -ContentPath c:\rf01

Finally, I added the memberships that enable replication and specify the content to replicate, using only two commands instead of three.

Out of the gate, DFSR Windows PowerShell saves you a significant amount of code generation and navigation. Better yet, it defaults to recommended configurations. It supports collections of servers, not just one at a time. With tabbed autocomplete, parameters always in the same order, mandatory parameters where required, and everything else opt-in, it is very easy to pick up and start working right away. We even added multiple aliases with shortened parameters and even duplicates of DFSRADMIN parameters.

Watch here as Windows PowerShell autocompletes all my typing and guides me through the minimum required commands to setup my RG:

Let’s scale this up - maybe I want to create a 100 server, read-only, hub-and-spoke configuration for distributing software. I can create a simple one-server-per-line text file named “spokes.txt” containing all my spoke servers – perhaps exported from AD with Get-AdComputer– then create my topology with DFSR Windows PowerShell. It’s as simple as this:

$rg = "RG01"
$rf = "RF01"
$hub = "SRV01"
$spokes = Get-content c:\temp\spokes.txt

New-DfsReplicationGroup -RG $rg | New-DfsReplicatedFolder -RF $rf | Add-DfsrMember -MemList ($spokes + $hub)

$spokes | % {Add-DfsrConnection -GroupName $rg -SMem $hub -RMem $_}

Set-DfsrMembership -RG $rg -RF rf01 -ComputerName $hub -ContentPath c:\rf01 –PrimaryMember $true -force

Set-DfsrMembership -RG $rg -RF rf01 -ComputerName $spokes -ContentPath c:\rf01 –Force -ReadOnly $true

Done! 100 read-only servers added in a hub and spoke, using four commands, a text file, and some variables and aliases used to save my poor little nubbin fingers. Not impressed? If this were DFSRADMIN.EXE, it would take 406 commands to generate the same configuration. And if you used DFSMGMT.MSC, you’d have to navigate through this:

That was not fun

With the underlying DFSR Windows PowerShell, you now have very easy scripting options to tie together cmdlets into basic “do everything for me with one command” functions, if you prefer. Here’s a simple example put together by our Windows PowerShell developer, Daniel Ong, that shows this off:

Configure DFSR using Windows PowerShell

It’s pretty nifty, check out this short demo video.

The sample is useable for simpler setup cases and also demonstrates (with plenty of comments!) exactly how to write your very own DFSR scripts.

I find your lack of faith disturbing

Still not convinced, eh? Ok, we’ve talked topology creation – now let’s see the ongoing management story.

Let’s say I’m the owner of an existing set of replication groups and replicated folders scattered across dozens or hundreds of DFSR nodes throughout the domain. This is old stuff, first set up years ago when bandwidth was low and latency high. Consequently, there are custom DFSR replication schedules all over the connections and RGs. Now I finally have brand new modern circuits to all my branch offices and the need for weird schedules is past. I start to poke around in DFSMGMT and see that undoing all these little nuggets is going to be a real pain in the tuchus, as there are hundreds of customizations.

What would DFSR Windows PowerShell do?

Get-DfsrConnection -GroupName * | Set-DfsrConnectionSchedule -ScheduleType UseGroupSchedule

Set-DfsrGroupSchedule -GroupName * -ScheduleType Always

With those two simple lines, I just told DFSR to:

1. Set all connections in all replication groups to use the replication group schedule instead of their custom connection schedules

2. Then set all the replication group schedules to full bandwidth, open 24 hours a day, 7 days a week.

Now that I have an updated schedule, I must wait for all the DFSR servers to poll active directory individually and pick up these changes, right? No! This can take up to an hour, and I have things do. I want them all to update right now:

Get-DfsrMember -GroupName * | Update-DfsrConfigurationFromAD

Oh baby! If I was still using DFSRDIAG.EXE POLLAD, I’d be on server 8 of 100 by the time that cmdlet returned from doing all of them.

Since things are going so well, I think I’ll kick back and read some DFSR best practices info from Warren Williams. Hmmm. I should configure a larger staging quota in my software distribution environment, as these ISO and EXE files are huge and causing performance bottlenecks. According to the math, I need at least 32 GB of staging space on this replicated folder. Let’s make that happen:

Get-DfsrMember -GroupName "rg01" | Set-DfsrMembership -FolderName "rf01" -StagingPathQuotaInMB (1024 * 32) -force

That was painless – I don’t have to figure out the server names and I don’t have to whip out Calc to figure out that 32GB is 32,768 megabytes. This wildcarding and pipelining capability is powerful stuff in the right hands.

It’s not all AD here, by the way – we greatly extended the ease of operations without the need for WMIC.EXE, DFSRDIAG.EXE, or crappy scripts made by Ned years ago. For instance, if you’re troubleshooting with Microsoft Support and they say, “I want you to turn up the DFSR debug logging verbosity and number of logs on all your servers”, you can now do this with a single easy command:

Get-DfsrMember -GroupName * | Set-DfsrServiceConfiguration -DebugLogSeverity 5 -MaximumDebugLogFiles 1250

Or what if I just set up replication and accidentally chose the empty folder as the primary copy, resulting in all my files moving into the hidden PreExisting folder, I can now easily move them back:

Restore-DfsrPreservedFiles -Path "C:\RF01\DfsrPrivate\PreExistingManifest.xml" -RestoreToOrigin

Dang, that hauls tail! Restore-DfsrPreservedFiles is so cool that it rates its own blog post (coming soon).

I sense something; a presence I've not felt since…

This new setup should be humming now – no schedule issues, big staging, no bottlenecks. Let’s see just how fast it is – I’ll create a series of propagation reports for all replicated folders in an RG, let it fan out overnight on all nodes, and then look at it in the morning:

Start-DfsrPropagationTest -GroupName "rg01" -FolderName * -ReferenceComputerName srv01

<snore, snore, snore>

Write-DfsrPropagationReport -GroupName "rg01"-FolderName * -ReferenceComputerName srv01  -verbose

Now I have as many propagation reports as I have RFs. I can scheduled this easily too which means I can have an ongoing, lightweight, and easily understood view of what replication performance is like in my environment. If I change –GroupName to use “*”, and I had a reference computer that lived everywhere (probably a hub), I can easily create propagation tests for the entire environment.

While we’re on the subject of ongoing replication:

Tell me the first 100 backlogged files and the count, for all RFs on this server, with crazy levels of detail:

Get-DfsrBacklog -GroupName rg01 -FolderName * -SourceComputerName srv02 -DestinationComputerName srv01 -verbose

Whoa, less detail please:

Get-DfsrBacklog -GroupName rg01 -FolderName * -SourceComputerName srv02 -DestinationComputerName srv01 -verbose | ft FullPathName

Seriously, I just want the count!

(Get-DfsrBacklog -GroupName "RG01" -FolderName "RF01" -SourceComputerName SRV02 -DestinationComputerName SRV01 -Verbose 4>&1).Message.Split(':')[2]

Boing boing boing

Tell me the files currently replicating or immediately queued on this server, sorted with on-the-wire files first:

Get-DfsrState -ComputerName srv01 | Sort UpdateState -descending | ft path,inbound,UpdateState,SourceComputerName -auto -wrap

Compare a folder on two servers and tell me if all their immediate file and folder contents are identical and they are synchronized:

net use x: \\Srv01\c$\Rf01

Get-DfsrFileHash x:\* | Out-File c:\Srv01.txt

net use x: /d

net use x: \\Srv02\c$\Rf01

Get-DfsrFileHash x:\* | Out-File c:\Srv02.txt

net use x: /d

Compare-Object -ReferenceObject (Get-Content C:\Srv01.txt) -DifferenceObject (Get-Content C:\Srv02.txt)

Tell me all the deleted or conflicted files on this server for this RF:

Get-DfsrPreservedFiles -Path C:\rf01\DfsrPrivate\ConflictAndDeletedManifest.xml | ft preservedreason,path,PreservedName -auto

Wait, I meant for all RFs on that computer:

Get-DfsrMembership -GroupName * -ComputerName srv01 | sort path | % { Get-DfsrPreservedFiles -Path ($_.contentpath + "\dfsrprivate\conflictanddeletedmanifest.xml") } | ft path,PreservedReason

Tell me every replicated folder for every server in every replication group in the whole domain with all their details, and I don’t want to type more than one command or parameter or use any pipelines or input files or anything! TELL ME!!!

I guess I got a bit excited there. You know how it is.

These are the cmdlets you’re looking for

For experienced DFSR administrators, here’s a breakout of the Dfsradmin.exe and Dfsrdiag.exe console applications to their new Windows PowerShell cmdlet equivalents. Look for the highlighted superscript notes for those that don’t have direct line-up.

¹ Mainly because they were pretty dumb and we found no one using them. However, you can export the values using Get-DfsrConnectionSchedule or Get-DfsrGroupSchedule and pipeline them with Out-File or Export-CSV. Then you can use Get-Content or Import-CSV to import them with Set-DfsrConnectionSchedule or Get-DfsrGroupSchedule.

² Paradoxically, these old commands leaves servers in a non-recommended state. To remove memberships from replication altogether in an RG, use Remove-DfsrMember (this is the preferred method). To remove a server from a specific membership but leave them in an RG, set their membership state to disabled using Set-DfsrMembership –DisableMembership $true.

³ DFSR Windows PowerShell implements DFSRADMIN MEMBERSHIP NEW implicitly via the New-DfsReplicatedFolder cmdlet, which removes the need to create a new membership then populate it.

⁴ You can use the Get-Acl and Set-Acl cmdlets in tandem with the Get-AdObject Active Directory cmdlet to configure delegation on the RG objects. Or just keep using the old tool, I suppose.

⁵ The DFSRADMIN SUB DELETE command was only necessary because of the non-recommended DFSRADMIN MEMBERSHIP DELETE command. To remove DFSR memberships in a supported and recommended fashion, see note 2 above.

⁶ Use the Get-AdObject Active Directory cmdlet against the DFSR objects in AD to retrieve this information (with considerably more details).

⁷ The legacy DFSR administration tools do not have the capability to list or restore preserved files from the ConflictAndDeleted folder and the PreExisting folder. Windows Server 2012 R2 introduced these capabilities for the first time as in-box options via Windows PowerShell.

⁸ The legacy DFSR administration tools do not have the capability to clone databases. Windows Server 2012 R2 introduced these capabilities for the first time as in-box options via Windows PowerShell.

⁹ The legacy DFSR administration tools do not have the full capabilities of Set-DfsrServiceConfiguration. Administrators instead had to make direct WMI calls via WMIC or Get-WmiObject/Invoke-WmiMethod. These included the options to configure debug logging on or off, maximum debug log files, debug log verbosity, maximum debug log messages, dirty shutdown autorecovery behavior, staging folder high and low watermarks, conflict folder high and low watermarks, and purging the ConflictAndDeleted folder. These are all now implemented directly in the new cmdlet.

Give yourself to the Dark Side

It’s the age of Windows PowerShell, folks. The old DFSR tools are relic of a bygone era and the main limit now is your imagination. Once you look through the DFSR Windows PowerShell online or downloadable help, you’ll find that we gave you 82 examples just to get your juices flowing here. From those, I hope you end up creating perfectly tailored solutions to all your day-to-day DFSR administrative needs.

- Ned “No. I am your father!” Pyle