Since we introduced Data Deduplication (“Dedup” for short) in Windows Server 2012, the Dedup team has been hard at work improving this feature and our updates in Windows Server 2016 are no exception. When we started planning for Windows Server 2016, we heard very clearly from customers that performance and scale limitations prevented use in certain scenarios where the great space savings from Dedup would really be useful, so in Windows Server 2016, we focused our efforts on making sure that Dedup is highly performant and can run at scale. Here’s what’s new in 2016:
- Support for Volume Sizes Up to 64 TB
In Windows Server 2012 R2, Dedup optimizes data using a single-threaded job and I/O queue for each volume. While this works well for a lot of scenarios, you have to consider the workload type and the volume size to ensure that the Dedup Processing Pipeline can keep up with the rate of data changes, or “churn”. Typically this means that Dedup doesn’t work well for volumes greater than 10 TB in size (or less for workloads with a high rate of data changes). In Windows Server 2016, we went back to the drawing board and fully redesigned the Dedup Processing Pipeline. We now run multiple threads in parallel using multiple I/O queues on a single volume, resulting in performance that was only possible before by dividing up data into multiple, smaller volumes:![]()
The result is that our volume guidance changes to a very simple statement: Use the volume size you need, up to 64TB. - Support for File Sizes up to 1 TB
While Windows Server 2012 R2 supports the use of file sizes up to 1TB, files “approaching” this size are noted as “not good candidates” for Data Deduplication. In Windows Server 2016, we made use of new stream map structures and improved our partial file optimization, to support Dedup for file sizes up to 1 TB. These changes also improve overall Deduplication performance. - Simplified Dedup Deployment for Virtualized Backup Applications
While running Dedup for virtualized backup applications, such as DPM, is a supported scenario in Windows Server 2012 R2, we have radically simplified the deployment steps for this scenario in 2016. We have combined all the relevant Dedup configuration settings needed to support virtualized backup applications into a new preset configuration type called, as you might expect, “Virtualized Backup Server” (“Backup” in PowerShell). - Nano Server Support
Nano Server is a new headless deployment option in Windows Server 2016 that has far small system resource footprint, starts up significantly faster, and requires fewer updates and restarts than the Windows Server Core deployment option. Data deduplication is fully supported on Nano Server!
Please let us know if you have any questions about Dedup in Windows Server 2016, and I’ll be happy to answer them! We also love to hear any feature requests or scenarios you’d like to see Dedup support (like support for Dedup on ReFS) in future versions of Windows Server, so feel free to send those to us as well! You can leave comments on this post, or you can send the Dedup team a message directly at dedupfeedback@microsoft.com. Thanks!