Hi Oliver, Verify the port-channel load-balance configuration and make sure it's set to dest-ip-port or try out other options for more granular distribution of traffic. The default load-balancing mode for Layer 3 interfaces is the source and destination IP address. The load-balance configuration needs to be done in the default VDC.
http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/interfaces/configuration/guide/if_portchannel.html After changing the load-balance method to include L4 ports, you could run multiple iterations of iperf which will by default use a random source port so you can check if traffic is hashed to multiple links. Best regards, Andras On Mon, Dec 17, 2012 at 12:09 AM, Oliver Garraux <[email protected]>wrote: > Hey, > > Has anyone seen issues with the load balancing on port-channels going > to hosts connected to FEX's attached to Nexus 7k's? > > We have a few hosts that are connected via port-channels to either a > 2248 or 2232 FEX. The FEX's are hanging off of a 7k. This is working > as far as I can tell, in that all of the ports appear to be > operational in the port-channel. On the switch though, I'm seeing > 99.99% of the traffic being sent out of one of the links in the 4 port > port-channels (with "show port-channel traffic"). Some of the > port-channels show 100% of traffic over one link, a few are just > 99.99% or something like 99.92%. I'm seeing reasonably balanced stats > for received traffic. > > One of the 7k's is using L3+L4 hashing, one is using just L3. With > "show port-channel load-balance forwarding-path", I can see that if I > change one of the source IP's, the 7k says that it will choose a > different link in the port-channel. I'm still waiting on one of the > server guys to let me know what IP's are connecting to these servers, > but it seems somewhat implausible to me that this is caused by there > just being a very small number of large flows that happen to get > hashed onto the same link. I'm seeing this on all of the host facing > port-channels that we have off of the FEX's, and I see this > historically looking through our monitoring system, as well as right > now. > > These are 7k's running 5.2(3a). I'm noticing this on two different > 7k's, with both 2248 and 2232 FEX's, and across 10+ different > port-channels to hosts. I'm not seeing this on any of the routed L3 > port-channels to other network devices though. The load balancing on > the port-channels between the 7k and the FEX's themselves also looks > reasonable. > > Thanks, > > Oliver > > ------------------------------------- > > Oliver Garraux > Check out my blog: blog.garraux.net > Follow me on Twitter: twitter.com/olivergarraux > _______________________________________________ > cisco-nsp mailing list [email protected] > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ > _______________________________________________ cisco-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
