LACP is a channel having two physical switches with a single link between. But hey what happens if we add a second physical link there? Oh yes, you will get a bridging loop. However switches have inbuilt mechanisms, which is here Spanning Tree Protocol, or STP, that stop this loop by blocking one of the ports. Thereby it ensures a loop-free topology. However that brings you back to the original problem right, i.e., a single physical connection between switches.
A channel or sometimes called port channel or EtherChannel (a Cisco specific implementation) is an aggregation of multiple physical interfaces that creates a logical interface. We use the term LAG here.
You can bundle up to eight individual active physical links into the LAG to provide increased bandwidth and redundancy. You can also load balance traffic across these physical interfaces. The LAG stays operational as long as at least one physical interface within the channel is operational and the LAG is operationally down when all member ports are operationally down.
Link Aggregation Control Protocol (LACP) lets you configure up to 16 interfaces into the channel. A maximum of eight interfaces can be active, and a maximum of eight interfaces can be placed in a standby state.
Note that some vendors recommend disabling spanning tree on any ports involved in Link Aggregation.
LACP enabled the LAG by detecting each side’s configuration and making sure that they are compatible. This is done via LACP Data Units, or LACPDUs.
This mode places a port in an active negotiating state. In this state, the port initiates negotiations with other ports by sending LACP packets.
This mode places a port in a passive negotiating state. In this state, the port responds to the LACP packets that it receives but does not initiate LACP packet negotiation (default).
Once the logical channel is established, then traffic is distributed across all links using a hashing algorithm based on some of the properties of the traffic.
Using all links
The hashing algorithm determines the LAG member used for traffic. LACP can use different properties of the outgoing traffic to distribute traffic across all the links participating in a LAG.
If one of the links fail, then the hashing algorithm will run again, and the traffic will be distributed across all the remaining members of the LAG.
What happens on Link Failure?
In instances where a link fails, the hashing algorithm performs the calculation again to redistribute traffic across the remaining links.
LACP in vSphere 5.1
So you may ask now what Link Aggregation Control Protocol (LACP) on a vSphere distributed switch provides? Answer to this question is, it is a method to control the bundling of several physical ports together to form a single logical channel. It does so by allowing network devices to negotiate automatic bundling of links by sending LACP packets to a peer.
LACP works by sending frames down all links that have the protocol enabled. If it finds a device on the other end of the link that also has LACP enabled, it will also independently send frames along the same links enabling the two units to detect multiple links between themselves and then combine them into a single logical link.
VMware vDS 5.1 provides Teaming and Failover option to provide load balancing as well as resiliency across uplinks. Of the various load balancing algorithms supported by teaming, only IP Hash allows upstream physical switches to support etherchannels. Data Centers are moving towards 10GbE network connectivity with most common deployment of 2x10G uplinks. In this setup, customers demand active-active connectivity with etherchannel as that allows them to use both links for load balancing as well as resiliency in case of a single link failure. In addition, etherchannels allow customers to not use Spanning Tree Protocol (STP) in access layer, which increases availability and failover times of the network.
LACP in vSphere 5.5
But vDS 5.1 does not have support for multiple LAGs from a single ESXi and also not on a single vDS. There are multiple drivers for this deficiencies to be rectified including competitive, adoption at scale and feature completeness. vDS competitors today, Cisco N1kv, open vSwitch and Hyper-v, all support LACP with multiple LAG. LACP allows for dynamic provisioning without the need to manually configure each host and physical access switch. So in a cloud environment, this allows for scale up deployment. Finally, not supporting LACP with multiple LAG allows competitors to generate a lot of FUD in the minds of customers about the maturity of vDS as a networking switch. VMware has addressed these gaps in order to compete as a networking switch in the industry in vSphere 5.5 (read here). In a nutshell this is what we support today as following.