A popular request from our customer base for Windows Server Data Deduplication has been to support the Hyper-V scenarios (i.e., VDI and virtualized backup) on storage local to the Hyper-V server. This “hyper-converged” configuration uses a single server or cluster for hosting the VDI or virtualized backup guest workloads and dedup is enabled on local storage (as an alternative to using a separate file server).
We’ve been working to validate this configuration and I have good news – we now have sufficient internal validation and experience with preview customers to support a limited version. The limitation is primarily to schedule the dedup processing for a time window where the VDI or virtualized backup workloads are idle.
The section below defines the specific requirements for this configuration along with recommendations for validation and support. There is also an updated version of the whitepaperLarge scale Virtual Desktop Infrastructure deployment using Windows Server 2012 R2 with Storage Tiers and Data Deduplicationthat includes this information in the context of VDI applications.
Support exception for limited, scheduled dedup processing on a hyper-converged Hyper-V configuration
The configuration of enabling Windows Server Data Deduplication on data volumes directly mounted on a Hyper-V compute server (as opposed to mounted from a shared folder hosted on a separate file server) is not supported in the general case. However, there is a useful, limited configuration that is supported as an exception. This configuration is designed to avoid the occurrence of priority and/or resource conflicts between the Hyper-V guest workloads and data deduplication host partition processing by using appropriate scheduling.
For this limited configuration, deduplication must be running in a supported scenario as defined in the TechNet article Plan to Deploy Data Deduplication. In particular, open VHD files used for accessing storage in guest virtual machines are only supported for VDI and virtualized backup scenarios.
This limited configuration consists of the following:
Hyper-V compute server or cluster dedicated to running VDI or virtualized backup guest workloads
All server nodes are running Windows Server 2012 R2 with the November 2014 update rollup for Windows Server 2012 R2 (KB 3000850) or later.
Guest VHDs are stored in locally attached data volumes. In a cluster configuration the volumes are mounted as Cluster Shared Volumes.
Windows Server Data Deduplication is enabled on these locally attached data volumes.
All deduplication tasks on a given server or cluster are scheduled to run in a limited “idle” timeframe when all VDI or virtualized backup workloads are idle.
It is not required to actively enforce a block on running VDI or virtualized backup workloads during this scheduled time, but the system is required to be configured so that operation of VDI or virtualized backup is idle or paused during the scheduled dedup processing window.
Volume sizing and deduplication performance monitoring are done in accordance with the procedures defined in the article Sizing Volumes for Data Deduplication in Windows Server.
If deduplication processing is not able to keep up with the daily data churn, either the “downtime” deduplication processing window must be extended or fewer simultaneous active VDI or virtualized backup workloads must be configured on the node. Also, if additional system downtime is available on weekends you can try scheduling extended dedup processing windows for weekend days.
In addition to meeting the above requirements, the following steps are also strongly recommended:
For VDI workloads:
In order to validate the VDI workload capacity of this limited configuration, run a load test using the supported VDI guest images. Tools such as LoginVSI or the equivalent should be used to directly validate the configuration prior to use in production.
Review the recommendations in the article Deduplication Tuning for Deployments with High Saving Rates. If the savings rates measured in the VDI deployment fall in the range covered in this article, adjust the deduplication settings as instructed.
For virtualized backup workloads, a full backup cycle should be validated with production-equivalent workloads to ensure that the backup and deduplication jobs complete.
An active Microsoft Service Premier Support contract is recommended to facilitate further communication, telemetry, and advice that can be useful when defining these configurations.
By the way, the support for these configurations will of course continue with the Windows Server 2016 release. It would be great if you could also use the current Technical Preview and evaluate both how this works for your hyper-converged configurations as well as how Windows Server 2016 helps you scale up, with full support for 64TB volumes and full use of up to 1TB files. See Data Deduplication in Windows Server Technical Preview 4 for all the details.
And as always we would love to hear your feedback and results. Please send email to dedupfeedback@microsoft.com and let us know about your experience with these configurations and, of course, any questions you may have.
Thanks!