All, (This is vaguely related to my question earlier in the week about ASR capacity)
We use quite a few 6704-10GE blades on our network, and I'm seeing some random congestion type issues. In some cases, I've made the problem go away by shuffling ports between blades to spread the load, but I'm left wondering exactly where the problems lie. >From talking to people on IRC, etc, I'm told that the 6704 runs out of steam around 24-26Gbps of throughput when handling imix traffic. I'm also told that this is largely driven by pps, rather than bps. If we take, for example, a 6504 on our network. It has a Sup2T in slot 1, 6704-10GE(CFC) in slot 2, 6724-SFP(CFC) in slot 3, and 6904-40G(DFC4) in slot 4. I've got a 4*10G portchannel towards our core consisting of Te2/1, Te4/5, Te4/6 & Te4/8 Te2/3 and Te4/9 form a 2*10G portchannel towards an IXP Te2/2 is a 10G link towards a transit provider. The traffic profile on the 4*10G portchannel seems to max out at about 24Gbps. I don't see any obvious packet drops or latency increase, just that the traffic doesn't go any higher than that. I suspect I'm hitting a limit on the 6704 which is causing this, but I can't figure out what that limit is. If I take a snapshot of the 3 active ports on the 6704 at peak time, I see: Te2/1: In = 2.7Gbps/580kpps, Out = 5.7Gbps/613kpps Te2/2: In = 7.0Gbps/865kpps, Out = 1.8Gbps/520kpps Te2/3: In = 7.3Gbps/789kpps, Out = 2.5Gbps/666kpps Summing that all up, I've got ~27Gbps of traffic flowing through the card, and just over 4Mpps. I also see this: rtr#show fabric drop Polling interval for drop counters and timestamp is 1 in seconds Packets dropped by fabric for different queues: Counters last cleared time: <22:54 08 Oct 14> slot channel Low-Q-drops High-Q-drops 1 0 0 0 1 1 0 0 2 0 35759 @00:57 09Oct14 0 2 1 76766 @00:57 09Oct14 0 3 0 0 0 4 0 169 @00:56 09Oct14 0 4 1 0 0 So I seem to be seeing fabric drops on the 6704 slot, on both channels (but more on channel 1, which has ports Te2/1 and Te2/2 on it). If I look at fabric utilisation, it doesn't say it's maxing out: rtr#show fabric utilization detail Fabric utilization: Ingress Egress Module Chanl Speed rate peak rate peak 1 0 20G 0% 0% 0% 0% 1 1 20G 0% 3% @19:53 08Oct14 0% 3% @19:53 08Oct14 2 0 20G 27% 50% @22:14 08Oct14 5% 13% @22:13 08Oct14 2 1 20G 33% 47% @00:33 09Oct14 23% 33% @23:09 08Oct14 3 0 20G 0% 0% 0% 0% 4 0 40G 11% 17% @22:30 08Oct14 26% 40% @00:02 09Oct14 4 1 40G 0% 0% 0% 0% So my questions... 1) For other people using the 6704-10GE blade, what sort of maximum throughput are you seeing? Have you managed to pinpoint what the limiting factor is? 2) What do the fabric drops really mean. My google-fu isn't helping a lot, and the command doesn't seem to be documented. Is there anything I can do to reduce the fabric drops? Why am I also seeing some on the 6904-40G slot, which should be a much more capable card. Many thanks in advance, Simon _______________________________________________ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/