Using System Insights to forecast clustered storage usage

This post was authored by Garrett Watumull, PM on the Windows Server team at Microsoft. Follow him @GarrettWatumull on Twitter.

In Windows Server 2019, we introduced System Insights, a new predictive analytics feature for Windows Server. System Insights ships with four default capabilities designed to help you proactively and efficiently forecast resource consumption. It collects historical usage data and implements robust data analytics to accurately predict resource usage, without requiring you to write any scripts or create custom visualizations.

System Insights is designed to run on all Windows Server instances, across physical and guest instances, across hypervisors, and across clouds. Because most Windows Server instances are unclustered, we focused on implementing storage forecasting capabilities for local storage – the volume consumption forecasting capability predicts storage consumption for local volumes, and the total storage consumption forecasting capability predicts storage consumption across all local drives.

After hearing your feedback, however, we realized we needed to extend this functionality to clustered storage. And with the latest Windows Admin Center and Windows Server GA releases, we’re excited to announce support for forecasting on clustered storage. Cluster administrators can now use System Insights to forecast clustered storage consumption.

How it works

When you install System Insights on a failover cluster, the default behavior of System Insights remains unchanged – the storage capabilities only analyze local volumes and disks. You can, however, easily enable forecasting on clustered storage, and System Insights will immediately start collecting all accessible clustered storage information:

• If you are using Cluster Shared Volumes (CSV), System Insights collects all clustered volume and disk information in your cluster. System Insights can rely on a nice property of CSV, where each node in a cluster is presented with a consistent, distributed namespace. Or, in other words, each node can access all volumes in the cluster, even when those volumes are mounted on other nodes.
• If you aren’t using Cluster Shared Volumes (CSV), System Insights collects all clustered disk information, but it can only collect the information about the clustered volume that is currently mounted on that node.

Once enough data has been collected, System Insights will start forecasting on clustered storage data.

Lastly, before describing how to enable this functionality, there are a couple last things to point out:

• First, all clustered storage forecasting data is stored on node-local storage. If you want to collect clustered storage data on multiple nodes, you must enable this on each node. This ensures that you have multiple copies of the clustered storage data in case a node fails.
• Even though clustered storage data is stored on node-local storage, the data footprint of System Insights should still be relatively modest. Each volume and disk consume 300KB and 200KB of storage respectively. You can read more about the System Insights data sources here.
• These changes don’t affect the CPU or networking capabilities, as these capabilities analyze the server’s local CPU or network usage whether that server is clustered or stand-alone.

Setting it up

Windows Admin Center

Windows Admin Center provides a simple, straightforward method to enable forecasting on clustered storage. If you’ve enabled failover clustering on your server, you’ll see this dialog when you first open System Insights:

Clicking Install turns on data collection and forecasting for clustered storage. Alternatively, you can also use the new Clustered storage button to adjust the clustered storage forecasting settings. This button is now visible if failover clustering is enabled on a server:

Once you click on Clustered storage, you can adjust the data collection settings, as well as the specific forecasting behavior of the volume and the total storage consumption capabilities. For each storage forecasting capability, you can specify local or clustered storage predictions:

PowerShell

For those of you looking to use PowerShell instead, we’ve exposed three registry keys to enable this functionality. Together, these help you manage clustered storage data collection, volume forecasting behavior, and total storage forecasting behavior:

To turn on/off clustered data collection, use the following registry key:

• Path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\SystemDataArchiver\
• Name: ClusterVolumesAndDisks
• Type: DWORD
• Values:
  • 0: Off
  • 1: On

To adjust the behavior of the volume consumption capability, use the following registry key:

• Path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\SystemInsights\Capabilities\Volume consumption forecasting\
• Name: ClusterVolumes
• Type: DWORD
• Values:
  • 0: Local volumes
  • 1: Clustered volumes
  • 2: Both

To adjust the behavior of the total storage consumption capability, use the following registry key:

• Path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\SystemInsights\Capabilities\Total storage consumption forecasting\
• Name: ClusterVolumesAndDisks
• Type: DWORD
• Values:
  • 0: Local volumes and disks
  • 1: Clustered volumes and disks

You can use the New-ItemProperty or the Set-ItemProperty cmdlets to configure these registry keys. For example:

# Create the registry path and then start collecting clustered storage data.

$ClusteredStoragePath = “HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\SystemDataArchiver”
New-ItemPath -Path $ClusteredStoragePath
New-ItemProperty -Path $ClusteredStoragePath -Name ClusterVolumesAndDisks -Value 1 -PropertyType DWORD

# Create the registry path and then forecast on both clustered and local volumes.

$VolumeForecastingPath = “HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\SystemInsights\Capabilities\Volume consumption forecasting”
New-ItemPath -Path $VolumeForecastingPath
New-ItemProperty -Path $VolumeForecastingPath -Name ClusterVolumes -Value 2 -PropertyType DWORD

# Update the registry key so the volume forecasting capability only predicts clustered volume usage.

Set-ItemProperty -Path $VolumeForecastingPath -Name ClusterVolumes -Value 1 -PropertyType DWORD

Conclusion

We’re really excited to introduce this new functionality in System Insights and Windows Admin Center. With the latest releases, cluster users can now use System Insights to proactively predict clustered storage consumption, and these settings can be managed both in Windows Admin Center and PowerShell.

Please keep providing feedback, so we can keep adding new functionality to System Insights that’s relevant to you:

• UserVoice
• Email: system-insights-feed@microsoft.com