AHV Basics – Part 2 Network Segmentation

This is NOT an AHV only feature (ESXi and Hyper-V too) but a relevant follow up to the last post.

What is Network Segmentation?

Network segmentation is a security feature introduced in AOS 5.5 that separates management traffic from backplane traffic by creating separate virtual networks, one for each of these traffic types, on the default external virtual switch on each host. To enable the CVMs in a cluster to communicate over these networks, the CVMs are multihomed. Multihoming is facilitated by the addition of a virtual network interface card (vNIC) to the CVM and placing the new interface on the backplane network. Additionally, the hypervisor is assigned an interface on the backplane network.

The traffic associated with the CVM interfaces and host interfaces on the backplane network can be secured further by placing those interfaces on a separate VLAN.

With the addition Nutanix platforms that support RDMA (Remote Direct Memory Access) you are also able to create a separate virtual network for RDMA enabled NICs. If the node has RDMA enabled NICs foundation will pass the NICs through to the CVM during the imaging process. The CVM will use only the first two of the RDMA enabled NICs for Stargate to Stargate communication. (Stargate is responsible for all data management and I/O operations and is the main interface fin the hypervisor).

Different Traffic Types in a Segmented Network

There are generally two different traffic types that enter and leave a Nutanix cluster:

Backplane Traffic

This is intra cluster traffic that is need for a cluster to function. This comprises of traffic between CVMs, traffic between CVMs and Hosts and storage traffic.

Management Traffic

Management traffic is administrative traffic, or traffic associated with Prism and SSH connections, remote logging, SNMP etc. The current implementation simplifies the definition of management traffic to be any traffic that is not on the backplane network, and therefore also includes communications between user VMs and CVMs.

Segmented and Un-Segmented Networks

The default out of the box setting is an unsegmented network in a Nutanix cluster, the Controller VM has two virtual network interfaces—eth0 and eth1. Interface eth0 is connected to the built-in external virtual switch, which is in turn connected to the external network through a bond or NIC team that contains the host’s physical uplinks. Interface eth1 is connected to an internal network that enables the CVM to communicate with the hypervisor. In this network, all traffic, whether backplane traffic or management traffic, uses interface eth0. These interfaces are on the default VLAN on the virtual switch.

Unsegmented network

In a segmented network, management traffic uses interface eth0 and the backplane traffic uses interface eth2. The backplane network uses either the default VLAN or, optionally, a separate VLAN that you specify when segmenting the network.

Segmented network

Network Segmentation was introduced with AOS 5.5 and is the minimum supported AOS version. AHV, ESXi and Hyper-V are all supported hypervisors for this feature.

Enabling the feature, couldn’t be simpler and is of course driven from Prism (CLI is available too).

From Prism, click the gear icon int he top right corner, then click Network Configuration

NS Prism

This will open the Network Configuration window, from here, click ‘Configure’ next to the Backplane LAN, eth2 interface

NS Network Segmentation

In the Create Interface window, configure the following:

1) Subnet IP – This should be non routable and should have a sufficient number of iP addresses. Two IP;s are required per node.

2) Netmask – The subnet mask for the above specified IP

3) VLAN ID – Specify the VLAN ID for this subnet. It’s not mandatory to configure a subnet ID but strongly recommended.

4) Click Verify and Save

NS create interface

But what about when I expand my cluster?

When you expand a cluster on which the network is segmented, network segmentation is extended to the added nodes. For each node you add to the cluster, two IP addresses are allocated from the specified non-routable network address space. If IP addresses are not available in the specified network, a message is displayed on the tasks page in the Prism web console, and you must reconfigure the network before you retry cluster expansion. When you change the subnet, any IP addresses that are currently assigned to the interfaces on the backplane network change, and the procedure, therefore, involves stopping the cluster.

Final Point

This feature is meant to meet specific security requirements where VLAN separation is mandatory. If that separation is not mandatory – Nutanix recommends using the same VLAN for management and backplane traffic. As ever, features should be enabled to meet requirements and not be treated as ‘nerd knobs!’.