VSAN ESXCLI MG and Cluster Information

I have a confession to make: I’m lazy. Quite lazy; but not that lazy that I’ve started programing yet, although I’m close to it. If you ask me to do something (or I know I have to do something), I’ll google the “best and least burdensome” way to do it. The more spoon-feeding the process, the better.

So I’ve been getting my hands dirty with Virtual SAN (the official name, since VSAN is already taken by Cisco to represent multiple SANs in their MDS; kind of a play on VLAN). Virtual SAN is configured from the vSphere Web Client (although you can also do it via the CLI, as explained here in William Lam’s article) and hard as I looked, there is limited non-storage information that you get regarding the state of the Virtual SAN cluster and its member. I can’t tell if this is by design or what, but I don’t like it.

When configuring a Virtual SAN, I want to know things like who is the Master, what MGs are being used and which are the Virtual SAN members. True to my nature, I googled around a bit (not that hard really, just the top 4-5 hits on Google) and I couldn’t quite find what I was looking for. Thus I had no choice to go into the CLI and get the answers myself. I’m putting this here as 1) a parking lot for me to reference in the future and 2) a page to help fellow slackers when they need to find out the state of their Virtual SAN deployment.

Without further ado, these are the states I’m interested in looking up, follow buy the commands that will provide you the information:

1. VMkernel port used for Virtual SAN (this you can get from the vSphere Web Client by looking at the host’s VMkernel ports in Manageà Networking )
2. Multicast Group (MG) used by Virtual SAN cluster.
3. The Port Numbers used by Virtual SAN’s MG traffic.
4. The Virtual SAN host role (Master, Backup or Agent)

ESXCLI VSAN NETWORK LIST

This provides me a, b and c above. This command provides Multicast information, including the Time To Live (TTL) for the Multicast traffic. The default value is 5, which means that if Virtual SAN is being deployed over Layer 3, you can’t have this host being more than 5 routers away from the other hosts. Of course, this setting can be changed with the command esxcli vsan network ipv4 set --intrface-name vmk# --multicast-ttl {0<#<256}.

[root@com-a2-esxi01:~] esxcli vsan network list Interface VmkNic Name: vmk4 IP Protocol: IPv4 Interface UUID: f08ca556-d051-9a7d-e293-0073219a0946 Agent Group Multicast Address: 224.2.3.4 Agent Group Multicast Port: 23451 Master Group Multicast Address: 224.1.2.3 Master Group Multicast Port: 12345 Multicast TTL: 5 [root@com-a2-esxi01:~]

On a side note, you should seriously consider changing the MGs if you are deploying Virtual SAN over Layer 3. Chances are pretty good that the Data Center network team has a MG addressing plan that does not include the default MGs for Virtual SAN. Remember that all Virtual SAN cluster members must have the same Master MG address and Agent MG address. The command to change the default MGs is esxcli vsan network  --interface-name vmk# --agent-mc-addr x.x.x.x --master-mc-addr y.y.y.y.

Elver’s Opinion: If you have multiple Virtual SAN clusters over Layer 3, then change the default MGs for each Virtual SAN cluster.

ESXCLI VSAN CLUSTER LIST

This provides me with 4 above. This command tells you who the Master and Backup Virtual SAN nodes are and, if this host is either of them or an Agent. There are only three node roles in Virtual SAN. I believe this command doesn’t need explanation other than to clarify that I have a three node Virtual SAN cluster.

[root@com-a1-esxi01:~] esxcli vsan cluster get Cluster Information Enabled: true Current Local Time: 2016-01-25T14:32:40Z Local Node UUID: 55ccc92f-d9ce-8fb0-3508-0073219a0944 Local Node State: AGENT Local Node Health State: HEALTHY Sub-Cluster Master UUID: 55ccc92f-d9ce-8fb0-3508-0073219a0945 Sub-Cluster Backup UUID: 55ccc92f-d9ce-8fb0-3508-0073219a0946 Sub-Cluster UUID: 523c6a0c-8f38-90c9-edb8-09b706227091 Sub-Cluster Membership Entry Revision: 166 Sub-Cluster Member UUIDs: 55ccc92f-d9ce-8fb0-3508-0073219a0945, 55ccc92f-d9ce-8fb0-3508-0073219a0946, 55ccc92f-d9ce-8fb0-3508-0073219a0944 Sub-Cluster Membership UUID: 3334a656-a49a-e14e-b62d-0073219a0945 [root@com-a1-esxi01:~]

.elver