When design the stretched VxRail vSAN cluster, there is an option to utilize Witness Traffic Separation (WTS) to separate the vSAN witness traffic from vSAN vmkernel interfaces. This post is to discuss what is the WTS and how to deploy it.
What is vSAN Witness Traffic Separation (WTS)?
Witness Traffic Separation (WTS) is officially introduced from vSAN 6.5. The WTS is designed to separate VMware vSAN data traffic from witness traffic in two-node vSAN cluster and the stretched cluster.
Without WTS, for vSAN stretched cluster, the vSAN VLAN need to be stretched across data sites and witness site. However, in some scenarios, the VLAN could not be extended outside the Core DC. The WTS is useful when the vSAN VLAN cannot be stretched to the witness site, also this could allow the witness appliance to be deployed in public cloud, such as AWS etc.
It is noted that, when deploying the witness appliance in public cloud, it is important to check the network requirement including latency, firewall requirement and etc.
In VxRail stretched cluster solution, the WTS can be utilized by introducing an additional VMKernel interface in each ESXi data node to carry the witness traffic.
For example, there are two data centers (DC1 and DC2) with four VxRail nodes in each site, by default, each VxRail ESXi have five VMKernel interfaces pre-configured by VxRail as below:
- Vmk0 – External mgmt. -Stretched VLAN
- Vmk1 – iDRAC network -No uplink
- Vmk2 – Internal mgmt. -Stretched VLAN
- Vmk3 – VSAN Data -Stretched VLAN
- Vmk4 – vMotion -Stretched VLAN
To configure WTS, the additional vmk5 need to be added and configured as below:
- DC1 ESXi – vmk5 -Witness Traffic -DC1 Only WTS VLAN(WTS-VLAN1)
- DC2 ESXi – vmk5 -Witness Traffic -DC2 Only WTS VLAN(WTS-VLAN2)
The Witness appliance need to be deployed in Witness site with a vSAN kernel interface configured as below:
- Witness ESXi – vmk0 -Mgmt. Traffic
- Witness ESXi – vmk1 -vSAN Traffic -vSAN VLAN (WITNESS-VLAN)
The vmk5 from each vSAN ESXi data node should be able to communicate with vmk1 in witness server in bi-directions via L3 routing.
The high-level configuration steps for WTS.
Step 1: Build a standard VxRail vSAN cluster via VxRail Wizard. The steps to build a standard cluster and stretched cluster is same in VxRail wizard.
Step 2: Deploy the Witness sever or Witness Appliance in witness site with two vmkernel interfaces. (vmk0 for management and vmk1 for vSAN traffic type.)
Step 3: In VxRail vSAN data nodes in each site, add vmkernel interface in each node for WTS as below:
- In each ESXi data node, add additional VMKernel interface vmk5 in VLAN WTS-VLAN1(DC1) or WTS-VLAN2(DC2) with no traffic type configured.
- In each ESXi data node, use below command to configure and check the new VMkernel interface with traffic type “Witness”
esxcli vsan network ip add -i vmk5 -T=witness esxcli vsan network list
Step 4: In Witness appliance, Check there is VMKernel interface in VLAN WTS-VLAN3 and IP with traffic type “vSAN” configured. Login Witness appliance and use below command to check.
esxcli vsan network list
Step 5: In vCenter, configure vSAN stretched cluster in vCenter vSAN “Fault Domain” configuration page.
Step 6: Add additional Isolation Response Addresses for vSAN cluster High Availability. Choose one or two IP addresses in the stretched vSAN VLAN as the isolation addresses, normally the vSAN VLAN gateway could be used.
For now, you should have the VxRail Stretched Cluster with WTS configured. Enjoy!