Fixing Windows Admin Center ‘Can’t verify whether “cluster_name” is online’

So you’re trying to add your Hyper-Converged Cluster to Windows Admin Center and it’s giving you the “Can’t verify whether “cluster_name” is online” treatment. You’ve checked DNS, upgraded WAC/Honolulu and tested installing it on multiple servers and workstations. Nothing helped. I have good news! for you:

I ran into this immediately after Project Honolulu became public and have been banging my head continuously. Here’s what to do:

Check the Event Viewer\Applications and Services Logs\Microsoft-ServerMangementExperience for the following entry:

400 - CimException: The xsi:type attribute (p1:MSCluster_Property_Node_PrivateProperties) does not identify an existing class.

This indicates your that WAC is connecting fine to your cluster but is running into an issue where it’s missing some cluster property.

I’ll have to give props to Robert Hochmayr here as he pointed me to the solution:

There are two private properties that are set on the cluster and its nodes which through some combination of events (like adding nodes to the cluster at a later point in time) are missing from nodes. You can find out by running the following PowerShell command on one of your S2D cluster nodes:

get-clusternode | Get-ClusterParameter

The output will look something like this:

Object Name Value Type
------ ---- ----- ----
S2D-01 S2DCacheBehavior 88 UInt64
S2D-01 S2DCacheDesiredState 2 UInt32
S2D-03 S2DCacheDesiredState 2 UInt32
S2D-03 S2DCacheBehavior 88 UInt64

Note that this was a four node cluster.. Nodes S2D-02 and S2D-04 are missing!

Off to the registry to fix it:

At HKLM\Cluster\Nodes\x\Parameters there should be two entries for the above cluster parameters. On my systems, the full registry key Parameters was missing from nodes 1 and 4 (go figure…). I added them *on each host* by running the following command lines:

REG ADD HKEY_LOCAL_MACHINE\Cluster\Nodes\1\Parameters /f /v  "S2DCacheBehavior" /t REG_QWORD /d "88"
REG ADD HKEY_LOCAL_MACHINE\Cluster\Nodes\1\Parameters /f /v "S2DCacheDesiredState" /t REG_DWORD /d "2"
REG ADD HKEY_LOCAL_MACHINE\Cluster\Nodes\4\Parameters /f /v  "S2DCacheBehavior" /t REG_QWORD /d "88"
REG ADD HKEY_LOCAL_MACHINE\Cluster\Nodes\4\Parameters /f /v "S2DCacheDesiredState" /t REG_DWORD /d "2"

Checking I now get the correct PowerShell output:

get-clusternode | Get-ClusterParameter

Object Name Value Type
------ ---- ----- ----
S2D-01 S2DCacheBehavior 88 UInt64
S2D-01 S2DCacheDesiredState 2 UInt32
S2D-02 S2DCacheBehavior 88 UInt64
S2D-02 S2DCacheDesiredState 2 UInt32
S2D-03 S2DCacheBehavior 88 UInt64
S2D-03 S2DCacheDesiredState 2 UInt32
S2D-04 S2DCacheBehavior 88 UInt64
S2D-04 S2DCacheDesiredState 2 UInt32

Once this was added I was immediately able to add the cluster to Windows Admin Center. No reboots or service restarts were needed.

-Jan

Related Post

Netgear ReadyNAS PRONetgear ReadyNAS PRO

For the last couple of months, I have been part of the betatest for the Netgear ReadyNAS PRO Business Edition. As the product has been released, I can now tell my story.

First of all, some history:

I’ve been a user of the PRO’s predecessor, the ReadyNAS NV+ for almost 3 years now. Its primary use has been that of a file and media streaming server. I have also had another NV+ in use as part of my employers server infrastructure, where it also works as a file server for archive and storage.
There were a couple of things that made me choose the ReadyNAS NV+ over its competition, but mostly I was convinced by reviews such as this, along with user feedback in forums and comments. For the most part, I can truthfully say I have been very happy with the NV+, as it has been rock-solid. There were no crashes or problems and configuration – at least for my usage scenario – has been dead simple.
There was only one minor gripe: Once you have a couple TBs of data, or your job includes moving hundreds of gigs of data on a regular basis, network throughput really comes to the foreground. With the NV+, under most if not all real-world applications, throughput maxed out around 30MB/s. This might not sound all that bad, considering most people still use 100Mbit networking, which tops out at around 12MB/s – but Gigabit Ethernet can move 120+MB/s and I’m an impatient fellow.

Along comes the ReadyNAS PRO, which is in most user regards an evolutionary step over the NV+:

  • 6 hot-plug drive bays instead of 4
  • two Gigabit Ethernet ports (with support for teaming and failover) instead of one
  • x86 architecture instead of SPARC
  • support for RAID6
  • support for X-RAID2 (more on that later)

Everything else, from a user perspective is pretty much the same:

  • The ReadyNAS Pro uses the same interface (FrontView)
  • it supports the same file protocols (SMB/CIFS, AFP, NFS, FTP(S), HTTP(S))
  • it supports the same streaming media solutions (UPnP AV, iTunes, SqueezeCenter, Home Media Streaming Server)
  • DHCP, WINS, Printer Sharing
  • Active Directory support
  • UPS support

So why choose the PRO over its much cheaper predecessor, the NV+? One word: Performance!

The ReadyNAS PRO is by far the fastest networking device I’ve seen yet. To be honest, I have no idea how fast it really is, and from my experience during the beta phase, no-one else really knows either.
Why is that, you ask? Well, it turns out that no-one within the group of testers had a client available that could serve data at faster than gigabit speeds. I was in fact planning on using IEEE 802.3ad to hook up a much faster server as well as the PRO to my network – but incompatibilities with my network hardware kept me from testing it. I’m still trying to track down some Cisco hardware to have another go at this but for the time being, I will refer you to the official testing data from Netgear, which I can personally tell you to be true.

In use, the limiting factor for me has been my client computer (a ThinkPad T61 with a 200GB 7.200rpm Hitachi HDD), where I’m regularly seeing transfer rates between 50-60MB/s – a huge improvement over the NV+ which gave me 30MB/s only on a good day (I had some issues with data fragmentation as the NV+ was nearly constantly filled to max capacity…). On a side note: If you use Vista, *don’t* have any kind of media files or players open because that just murders network throughput (more on this issue here).

So what else is there to say:
The one factor that lead me to the ReadyNAS line of products was its proprietary X-RAID technology. Let me explain about this for a short while:
RAID stands for Redundant Array of Independent Disks. This means that a disk failure will not lead to you losing data. There are basically three types of RAID terminology you need to know about:

  • RAID 1: Mirroring – best for two disks, you loose 50% of your gross capacity
  • RAID 5: Striping – starting with three disks, parity information is spread so that any one disk failure will not lead to data loss
  • RAID 6: Striping with “hot spare” – starting with four disks, parity information is spread so that and *two* disk failure will not lead to data loss

There are also other kinds of RAID you can use like RAID 10, 50 or 60 but I won’t go into those as they don’t really apply for a NAS box. Back to the topic though: Typically, once you’ve decided on one version of RAID for your setup, you cannot go back. You cannot upgrade to larger disks, you cannot change much of anything whatsoever. That is, unless you have X-RAID:
The ReadyNAS has this trick up its sleeve which allows you to go from one disk to two (when it sets up a mirrored RAID 1 volume), then three (converting the RAID 0 to a RAID 5), up to 4 (with the PRO 6) disks. During the latter stages, all the NAS does is to expand the volume and restripe the parity information so it is evenly spread among disks. Netgear calls this horizontal expansion.
Once you hit the maximum amount of disks you could put into your box, there was only one way to upgrade to even higher capacity: Swap out each disk one-by-one for a larger capacity model, wait for resync, and once all the disks have been swapped, wait for the volume expansion to take place. With the ReadyNAS PRO, X-RAID2 is adding a new trick: vertical expansion. The moment your drive swaping allows for a redundant setup of two larger capacity disks, the NAS will expand the volume to include whatever new diskspace is available in a redundant setup. What’s that mean? Here’s an example:

Say you have 4 500GB hdds already in place and you need even more space, adding two 1TB drives would usually have lead to an additional 1TB of space – because in a RAID5 volume, all disks can only as much to the volume as the smallest disk in the array. With X-RAID2, the ReadyNAS PRO detects that there are two drives with an unused capacity of 500GB each, which it then uses to extend the volume by another 500GB, which are in a mirrored setting. Neat.

Then again, who cares? In reality, most people will probably populate the NAS with all the disks they plan to put in it from the start and never think about it again until something breaks or they run out of space. Only then does this feature become important. What’s it good for then? Peace of mind!

I have pretty much summed up most I had to say, except for one thing: It’s quiet. Really quiet. Netgear did a great job designing the hardware for this thing as all my disks have been at below 40°C and I’ve had this thing on my desk without the fans feeling annoying.

To sum it up: If you’re looking for a fast, stable and userfriendly device to store your data, the ReadyNAS PRO will be the best choice currently on the market. It may be more expensive than its competition – but ease of use and maturity do count for something, most certainly in a business environment. Finally, a five year warranty along with the stellar support from the ReadyNAS team over at www.readynas.com/forum should put your mind at ease… it’s certainly convinced me.

-Jan Olbrecht

My acknowledgements and thanks go to the ReadyNAS PRO beta tester group and the Jedi Council over at the ReadyNAS Forum.

Issues preparing for upgrade to SCVMM 1801Issues preparing for upgrade to SCVMM 1801

I just ran into a small hickup upgrading my System Center Virtual Machine Manger 2016 to the new version 1801. I wasn’t able to find any documented cases of anyone running into this so here we go:

The upgrade to SCVMM 1801 actually requires uninstalling SCVMM 2016. This failed on my installation just at the point where I was clicking on the “Remove features” button in the setup dialog.

In C:\ProgramData\VMMLogs\SetupWizard.log I found the following entry:

11:52:28:Uncaught Exception: Threw Exception.Type: Microsoft.VirtualManager.Utils.CarmineException, Exception.Message: Unable to detect cluster configuration of the node.
Ensure that the user has permissions to detect cluster node configuration.
11:52:28:StackTrace: at Microsoft.VirtualManager.Setup.ClusterServiceHelper.get_IsAClusterNode()
 at Microsoft.VirtualManager.Setup.AddRemoveComponentsPage.EnterPage()
 at Microsoft.VirtualManager.SetupFramework.PageNavigation.WaitEnterSet(Page page)
 at Microsoft.VirtualManager.Setup.AddRemovePage.RemoveComponent_Click(Object sender, RoutedEventArgs e)
 at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
 at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
 at System.Windows.Controls.Primitives.ButtonBase.OnClick()
 at System.Windows.Controls.Button.OnClick()
 at System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(MouseButtonEventArgs e)
 at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
 at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
 at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
 at System.Windows.UIElement.ReRaiseEventAs(DependencyObject sender, RoutedEventArgs args, RoutedEvent newEvent)
 at System.Windows.UIElement.OnMouseUpThunk(Object sender, MouseButtonEventArgs e)
 at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
 at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
 at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
 at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
 at System.Windows.UIElement.RaiseTrustedEvent(RoutedEventArgs args)
 at System.Windows.Input.InputManager.ProcessStagingArea()
 at System.Windows.Input.InputManager.ProcessInput(InputEventArgs input)
 at System.Windows.Input.InputProviderSite.ReportInput(InputReport inputReport)
 at System.Windows.Interop.HwndMouseInputProvider.ReportInput(IntPtr hwnd, InputMode mode, Int32 timestamp, RawMouseActions actions, Int32 x, Int32 y, Int32 wheel)
 at System.Windows.Interop.HwndMouseInputProvider.FilterMessage(IntPtr hwnd, WindowMessage msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
 at System.Windows.Interop.HwndSource.InputFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
 at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
 at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
 at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
 at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
 at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
 at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
 at MS.Win32.UnsafeNativeMethods.DispatchMessage(MSG& msg)
 at System.Windows.Threading.Dispatcher.PushFrameImpl(DispatcherFrame frame)
 at System.Windows.Application.RunDispatcher(Object ignore)
 at System.Windows.Application.RunInternal(Window window)
 at Microsoft.VirtualManager.Setup.Program.UiRun()
 at Microsoft.VirtualManager.Setup.Program.Main()

So what this means is that the setup is unsuccessfully trying to determine whether SCVMM is installed in a clustered setup. Which mine is not. Has never been.

The solution is to actually *install* the Windows Feature Failover Clustering. This way the check can run and setup will continue. There is no need to actually configure clustering.