EtherChannel in practice: Layer 3 bundles, load balancing and troubleshooting

Quick recap and Layer 2 configuration

EtherChannel joins two to eight parallel links into one logical port-channel. Spanning tree sees a single link, so none of the members is blocked. LACP (IEEE 802.1AX, originally 802.3ad) and the Cisco PAgP protocol negotiate the bundle; the modes are covered in Layer 2 redundancy. LACP can also hold up to eight extra ports as hot standby members.

interface range GigabitEthernet1/0/49 - 50
 switchport mode trunk
 switchport trunk allowed vlan 10,20,99
 channel-group 1 mode active
interface Port-channel1
 description Uplink bundle to DIST-1
show etherchannel summary

Settings applied to the port-channel interface are copied to the members. Make later changes on the port-channel, not on individual members, to keep them identical.

Layer 3 EtherChannel

Between Layer 3 switches, or from a switch to a router, the bundle can be a routed link. The physical ports must become routed ports with no switchport before they join the group, and the IP address goes on the port-channel.

interface range TenGigabitEthernet1/1/1 - 2
 no switchport
 channel-group 10 mode active
interface Port-channel10
 no switchport
 ip address 10.0.0.1 255.255.255.252
show ip interface brief | include Port-channel

A routed bundle is a single interface to OSPF or EIGRP, so losing one member does not cause a routing change, only a bandwidth drop. Use port-channel min-links (LACP) if you would rather the whole bundle went down when too few members remain.

How traffic is shared

EtherChannel does not split one conversation across links. The switch calculates a hash from fields of each frame, such as source and destination MAC or IP addresses, and the result picks a member link. All frames of one flow therefore follow the same link and stay in order. The consequence is that a single large transfer never goes faster than one member link.

port-channel load-balance src-dst-ip
show etherchannel load-balance

If traffic is mostly between few addresses, for example one backup server talking to one storage array, the hash may put almost everything on one link. Including Layer 4 port numbers in the hash (where the platform supports it) or a different combination of fields spreads load better. Bundles of two, four or eight links share load most evenly.

Troubleshooting

Read the flags in show etherchannel summary: P means bundled, I stand-alone (no negotiation partner), s suspended (settings do not match the bundle), D down, and for the port-channel SU or RU means Layer 2 or Layer 3 and in use. Common causes:

• Mode mismatch: LACP on one side and PAgP on the other, both sides passive or auto, or "on" facing a negotiating mode. "On" with no protocol is the riskiest, because a mis-cabled link is not detected.
• Member mismatch: different speed, duplex, access VLAN, allowed VLANs, native VLAN or trunk mode on one member.
• Cabling: members of one bundle going to two different switches that are not a stack or virtual pair.

Spanning tree's EtherChannel misconfiguration guard (spanning-tree etherchannel guard misconfig, on by default on many switches) err-disables ports if it detects that one side bundles and the other does not, to prevent a loop. Fix the cause before re-enabling them. show lacp neighbor and show interfaces port-channel 1 etherchannel give more detail.

Real-world example: after a weekend upgrade at a logistics warehouse, one of two uplinks in a bundle showed flag "s". The engineer compared show running-config interface on both members and found the replacement port still had the old allowed VLAN list. Re-applying the trunk settings on the port-channel fixed it, and the bundle returned to 20 Gb/s for the scanners and cameras.