Storage Knowledge Base : Netapp MultiMode VIFs and Load Balancing

Netapp MultiMode VIFs allow you the administrator to add some reliability and performance to your filer networking configuration. As with everything in the tech world, there are many different ways of doing things. There are many ways to build and configure highly available and high performance networks. Netapps support Ethernet technology that is known in other environments as Ethernet trunking, bonding, Etherchanneling or port channeling (Cisco). The joining of 2 or more physical links together into one big fast and available link. Unix systems have been doing this for many years. Netapps use MultiMode VIF’s for ethernet channeling. This used to be a complicated technology that involved special handshaking with switches, and not all switches handled the technology well as there were and are still different angles on this type of technology. Load balanced or passive active so that if one link fails the other one takes over. Hopefully cabled to a different switch. This technology is now used all the time, it has become very common and is used to increase performance on the Ethernet Fabric is MultiMode VIFs. A MultiMode VIF is NetApp’s term for EtherChannel or Port Channeling. Earlier I used the terms trunking or bonding which are not really the accurate terms. Trunking usually refers to a form of VLAN trunking. So I won’t use the trunk term any more.

Netapp Defined types of MultiMode VIF’s

Static MultiMode VIF

A Netapp static MultiMode VIF can be compored to a hard coded or static EtherChannel. A hard coded physical multi link into the channel. The ports do not negotiate, there is no auto detection of the physical port status. The interfaces are forced or told to channel. On the switch side, the static etherchannel has to be enabled on each port.

The Netapp command to to create a static MultiMode VIF is

ourfiler> vif create multi

Dynamic MultiMode VIF.

A Dynamic MultiMode VIF is a LACP EtherChannel. LACP is an acronmy (as are all things tech) for Link Aggregation Control Protocol. This complies to the IEEE 802.3ad standard for port channeling. LACP allows a device to exchange PDUs which are channeled together. A dynamic MultiMode VIF is intelligent, it is dynamic in that link status’s will be checked and appropriate actions will be taken ifa physical link is down. There is hand shaking, communication and negotiations between devices. An perfect example of hand shaking and communication is in the event of some type of physical connection problem. The switch would remove the interface from the channel and with link aggregation protocol, a notification from the switch to the Netapp controller would be sent telling it that the interface was removed. This allows the Netapp controller to gracefully handle the situation. It would remove it’s end of the link and not try to blindly sent frames down a broken link. This is in contrast to the static MultiMode VIF’s. A statis MultiMode VIF would just continue trying to transmit and would generate errors. These errors would alert the highly trained admins at which point they would manually remove the link, or fix it. Take some kind of action.

The command to create a dynamic MultiMode VIF is

ourfiler> vif create lacp

MultiMode VIF Load Balancing Options

MultiMode VIF Load Balancing with MAC Load-Balancing

MAC based load balancing is a rarer type of algorithm now a days. There are many conditions which can cause traffic to be distributed unevenly, as in large bursts of traffic being sent on just a single link. MAC based load balancing makes calculations based on thesource and destination pairs MAC addresses. The source being the connecting host and the destination MAC would be the one associated to the MultiMode VIF, seen in the output of the ifconfig command. When your filer is on a more complex network with multiple subnets and routers, MAC based load balancing starts to show it’s cracks. Ethernet frames get changed when connections go through routers, therefore flat networks work the best for this technology. If your filer is on a flat network, not routers etc.. then this type of MultiMode VIF load balancing won’t experience as many problems and is a viable option for a load balancing solution.

MultiMode VIF Load Balanding with Source and Destination Pairs

The MultiMode VIF load-balancing method is the most common and the one used by many of the Netapp MultiMode VIF setups today. Somewhat complex calculations are made to determine which link to use. The source and destinations are compared and divided by the actualnumber of physical interfaces or links in the VIF. The resulting calculation should equal or match one of the physical interfaces, which is then used. The ports are bonded resulting in high availability, not gained throughput. If you had 2 x 1 Gig Ethernet interfaces bonded into one MultiMode VIF using EtherChannel, this wouldn’t mean you would have 2 Gbps throughput. It simply means that youwould have 1 Gbps throughput with 1 optional communication link for redundancy. Because the physical links are bonded into one VIF,all the physical links are used, but bandwidth doesn’t increase proportionaly to the sum of the number of links. So no single transmission burst on a physical link can exeed the the 1 Gbps speed.

MultiMode VIF Load Balanding with Round Robin

Round Robin type load balancing has been around for a long time. It has it’s good and bad points. One of the shortcomings of round robin technology is that it runs the risk of packets arriving out of order causing untold and unecessary messes. Oddly enough, in some rare cases, you can find some installations that utilize this feature and they have worked for years without any problems. RoundRobin as the name implies, sends out ethernet frames over the active links in the channel. This theoretically provides an evenly distributed transmission. Problems arise when a frame on one link arrives after a second frame is sent on another link. This could easily occur because of link congestion or something similar, and errors would be generated. It would essentially confuse the hell out of the system. Round robin is supported but very rarely used. Will leave it at that.

MultiMode VIF IP based Load Balanding

MultiMode VIF IP based load balancing I believe is the default load balancing type used. I also think that it is the most common type of MultiMode VIF found in the enterprise environments today. The principles behind it are more or less the same as MAC based algorithms I described above. The only real difference is that source and destination IP addresses are used instead of MAC addresses. Using MultiMode VIF IP based load balancing results in a more evenly distributed network compared to MAC based load balancing. MultiMode VIF IP based load balancing uses the last octet of the IP address. Care must be taken when your environment has multiple subnets,this can affect the load balancing performance. IP aliasing can also be used as a technic to distrubute traffic over different links. Netapp VIF’s and physical interfaces have the ability to have an alias placed on the interface. Really, they are additional IP addresses mapped to VIF. Typically you don’t want more aliases than physical links on the VIF. Years ago, we used to connect each NIC to a seperate segment or VLAN. So the filer had 8 different IP addresses for 8 different subnets, and 8 different segments too. This distributed the load nicely. In some cases we had bonded NIC’s. That was in the 90′s. The MultiMode VIF’s are slicker for many reasons. It’s important to coordinate your CIFS or NFS clients to use all the aliased IP adddresses that have been assigned. ie. nas0, nas1, nas2 and nas3. Split your clients up to use these name/ip combos evenly. Multiple physical interfaces are partnered together for a good reason. The first reason is for failover in the event of a physical link failure of some kind. This ensures that failover of a controller will move the failed controllers interfaces to the remaining controller. When an alias is mapped to an interface, and if you have partnered the physical, the aliases always travels to the clustered controller in the event of a failover. The aliases are not clustered or partnered, just the physical devices. This is important and ultimately the desired result.

Sample Static MultiMode VIF

Ssh into filer01 and setup the MultiMode VIF using the first 4 interfaces, e0a-d.

# ssh filer01 -l netappguy

filer01> vif create multi our-vif01 -b ip e0a e0b e0c e0d

filer01> ifconfig our-vif01 172.16.31.10 netmask 255.255.255.0 mtusize 1500 partner cisco-partner-001

filer01> ifconfig our-vif01 alias 172.16.31.11 netmask 255.255.255.0

filer01> ifconfig our-vif01 alias 172.16.31.12 netmask 255.255.255.0

filer01> ifconfig our-vif01 alias 172.16.31.13 netmask 255.255.255.0

LACP Dynamic MultiMode VIF

Ssh into filer01 and setup the MultiMode VIF using the first 4 interfaces, e0a-d.

# ssh filer01 -l netappguy

filer01> vif create lacp our-vif01 -b ip e0a e0b e0c e0d