Re: [c-nsp] ASIC to switch port mapping
Hi, On Mon, Sep 20, 2010 at 02:41:12PM +, Mishka, Jason wrote: I've seen similar problems when backups run at night. The drops happened on a 4xGE etherchannel on the individual ports in the bundle. What I found was that the hashing was being done very poorly and one of the interfaces was being assigned more than its share of the load. I know there are different hash algorithms but none appear to be able to do anything different in this case. Well, in our case we hash on src+destination IP address, and since this is audio streaming to a high number of destinations (10.000 different IPs or more) it hashes very well. If you do backups, you basically have few source IPs, one destination IP and hashing will always be less than perfectly balanced. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] ASIC to switch port mapping
Hi all, I am pretty sure the C3750/C3560 ranges of switches have 2 MB of buffer space / ASIC. How many ports are connected on an ASIC depends and differs in each model: some models have 4 ports / ASIC, other 24 ports / ASIC :-) The buffer space of 2 MB is divided in 4 queues which you can program with the mls qos settings. You can allow a queue to expand all the way up to 2 MB if needed (or you can also program the queue NOT to do that) If i remember correctly by heart, the default setting of Cisco is to cut up the 4 queueu evenly (25 25 25 25), then reserve half of each queue (dedicated, reserved 50% 50% 50% 50%) And then allow each queue to borrow space from neighboring queues to grow up to 100% meaning doubling (??). I have found that this limit is too low (100% is not allowing to use the full queue: 25 x 2 = 50% half of available space). Therefore i put the burst percentages to 400%, meaning each queue of 25% can grow 4 times up to 100% of buffer space, all of the 2 MB. Remember: all these 4 queues are shared between ports on the same ASIC Here is - an already famous document (i think lots of people have already spent hours understanding this doc :-) http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a0080883f9e.shtml regards, Geert 2010/9/14 Nick Hilliard n...@foobar.org On 13/09/2010 21:33, Benny Amorsen wrote: 3MB per PFE, according to: http://www.juniper.net/us/en/local/pdf/implementation-guides/8010073-en.pdf http://kb.juniper.net/KB10963 so, the 24 port model has 2 PFEs (i.e. 6M buffer space) and the 48, 3 PFEs (9 meg). That's not really very much, particularly the way that it's divvied up. I'm not sure how much buffer the 3560 actually has, just that it isn't enough. See a couple of postings back: http://networking.ventrefamily.com/2010/09/3560ge-and-3750ge-buffers.html quoth cisco: these platforms [Catalyst 3560G/3750G and 3650-E/3750E ] provide (minimally) 750 KB of receive buffers and (up to) 2 MB of transmit buffers for each set of 4 ports. Again, this looks like a simplification. It would help enormously if Cisco were to publish clear and extensive documentation as to how buffering on each of the 3560/3750 models actually works. Would anyone from Cisco on this list be able to oblige here? Or if its already published somewhere, could you point us in the right direction? Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Hi, On Sun, Sep 12, 2010 at 07:13:33PM -0400, Keegan Holley wrote: You can always buy more switches and move ports. The 2960 and the hundreds of other switches (and blades) just like it is a wiring closet switch for the enterprise. It should be common knowledge (no offense if this is new information to you) that they are oversubscribed, have tiny buffers and are not suitable for anything but. Oversubscription is not the issue here, tiny buffers is - and indeed that came as a big suprise as the 2950, 2970 and 3550s have larger buffers. Read up in the archives for the lengthy discussion of the problems we had - but basically it came down to microbursts overflowing the egress ports, while the average egress load never exceeded 50% (!), and with many ports unused on the switch. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de pgpKJ94rExrbQ.pgp Description: PGP signature ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
FYI The 48 port 3560E switches have 3x ASICs so you'll get the same buffer limitations across each set of 24 ports. Regards, --Dan On Fri, Sep 10, 2010 at 7:27 PM, Vincent Aniello vincent.anie...@pipelinefinancial.com wrote: So it looks like a WS-C3560E-24TD has two ASICs: switch#show platform port-asic version Port-Asic Version Info: ASIC-0: Version:4 DeviceType:0x30A ASIC-1: Version:4 DeviceType:0x30A Also, the show platform pm if-numbers command does return the ports assigned with each ASIC, I overlooked the '0' in the port column because I expected to see more than two ASICs. --Vincent From: illcrit...@gmail.com [mailto:illcrit...@gmail.com] On Behalf Of Ben Steele Sent: Friday, September 10, 2010 2:06 PM To: Vincent Aniello Cc: Nick Hilliard; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping sh platform port-asic should list your ASIC's port groupings are almost always in groups, so you can work out what ports belong to a common ASIC by dividing the amount of ports you have by the amount of ASIC's listed, keep in mind you will probably have a dedicated ASIC for the 2 10G uplinks. On Fri, Sep 10, 2010 at 6:50 PM, Vincent Aniello vincent.anie...@pipelinefinancial.com wrote: This is on a 3650E switch. Thanks. --Vincent -Original Message- From: Nick Hilliard [mailto:n...@foobar.org] Sent: Friday, September 10, 2010 1:31 PM To: Vincent Aniello Cc: Heath Jones; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping On 10/09/2010 18:18, Vincent Aniello wrote: I am trying to solve a output drops on switch ports on which bandwidth utilization does not seem to exceed the port speed. Seems like the drops are due to the buffers filling up and dropping frames. I am under the impression that each ASIC has their own buffer and if the buffer fills on a particular ASIC all ports that share that ASIC will also drop frames. If I know the switch interfaces associated with each ASIC I can redistribute the connections on the switch to better balance the load. What sort of card are you using? Nick Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-...@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On 13/09/2010 07:05, Gert Doering wrote: ports, while the average egress load never exceeded 50% (!) The average that you're talking about here is measured over 5 minutes, which is an eternity in terms of packet throughput. If you drop your measurement interval from 5 minutes to something much lower (5 - 30 seconds, you'll get a much more accurate picture of what your switch is actually doing. Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On 13/09/2010 10:44, Gert Doering wrote: Nick, grant me a bit of understanding about averaging and bursts :-) Heh, this wasn't directed at you, really. But most people don't bother looking at numbers any closer than the 5 minute average - which tells you almost nothing about what's going on. (spreading out the packets), while most other streaming software creates somewhat massive wirespeed bursts, and then waits some milliseconds, and then generates a new wirespeed burst. ew, that is pretty horrible :-( Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Hi, On Mon, Sep 13, 2010 at 11:06:48AM +0100, Nick Hilliard wrote: On 13/09/2010 10:44, Gert Doering wrote: (spreading out the packets), while most other streaming software creates somewhat massive wirespeed bursts, and then waits some milliseconds, and then generates a new wirespeed burst. ew, that is pretty horrible :-( Trade-off between server load (send out as many packets as can be stuffed into the hardware in one go) and network load (smooth out stuff, but have more context switches, interrupts, ... etc. in the server). Now if I had more time :-) it might be worth investigating the (Linux) streaming server software used, whether it can be changed to invest a bit more CPU to better smooth out the packets... OTOH, the kernel might just wreck this, and smear it all togehter again. (*Now* we really get even more off-topic for c-nsp than usual) gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de pgp8FnfUkyKpt.pgp Description: PGP signature ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Interesting enough, yesterday James Ventre posted a note where he found at least some minimal info about the 2960/3560/3750 buffer amount: http://networking.ventrefamily.com/2010/09/3560ge-and-3750ge-buffers.html Also, I have to say I have exactly the same experience as Gert - IPTV streaming box connected via 1Gbps, generating about 65Mbps, that no Cisco Enterprise level switch (aka 2960/3560/3750 or even ME3400) was able to forward to a 100Mbps port without output drops. When we sniffed it we found that in a given discrete period of 1 second, the streaming box is absolutely idle for the first 900 ms, and then quickly pushes 80Megs in the last 100 ms. My guess is that due to the functionality of the MPEG box, it needs to gather some uncompressed picture frames first, and only after that it is able to produce the MPEG outputwhich makes it hurry too much :) -pavel On Mon, Sep 13, 2010 at 12:55 PM, Gert Doering g...@greenie.muc.de wrote: Hi, On Mon, Sep 13, 2010 at 11:06:48AM +0100, Nick Hilliard wrote: On 13/09/2010 10:44, Gert Doering wrote: (spreading out the packets), while most other streaming software creates somewhat massive wirespeed bursts, and then waits some milliseconds, and then generates a new wirespeed burst. ew, that is pretty horrible :-( Trade-off between server load (send out as many packets as can be stuffed into the hardware in one go) and network load (smooth out stuff, but have more context switches, interrupts, ... etc. in the server). Now if I had more time :-) it might be worth investigating the (Linux) streaming server software used, whether it can be changed to invest a bit more CPU to better smooth out the packets... OTOH, the kernel might just wreck this, and smear it all togehter again. (*Now* we really get even more off-topic for c-nsp than usual) gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025 g...@net.informatik.tu-muenchen.de ___ cisco-nsp mailing list cisco-...@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On Mon, 13 Sep 2010, Pavel Skovajsa wrote: Interesting enough, yesterday James Ventre posted a note where he found at least some minimal info about the 2960/3560/3750 buffer amount: http://networking.ventrefamily.com/2010/09/3560ge-and-3750ge-buffers.html Ugh, ugly. I was hoping to find a box that could do 10Gb/s uplink and breakout as far down as 100Mb/s. Back to hunting again. wfms ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On a 48 port 3560E, 24 ports per ASIC cat3560-2#sh platform pm if-numbers interface gid gpn lpn port slot unit slun port-type lpn-idb gpn-idb -- Gi0/1 1111/1 111local Yes Yes Gi0/2 2221/0 122local Yes Yes Gi0/3 3331/3 133local Yes Yes Gi0/4 4441/2 144local Yes Yes Gi0/5 5551/5 155local Yes Yes Gi0/6 6661/4 166local Yes Yes Gi0/7 7771/7 177local Yes Yes Gi0/8 8881/6 188local Yes Yes Gi0/9 9991/9 199local Yes Yes Gi0/1010 10 10 1/8 110 10 local Yes Yes Gi0/1111 11 11 1/11 111 11 local Yes Yes Gi0/1212 12 12 1/10 112 12 local Yes Yes Gi0/1313 13 13 1/25 113 13 local Yes Yes Gi0/1414 14 14 1/24 114 14 local Yes Yes Gi0/1515 15 15 1/15 115 15 local Yes Yes Gi0/1616 16 16 1/14 116 16 local Yes Yes Gi0/1717 17 17 1/17 117 17 local Yes Yes Gi0/1818 18 18 1/16 118 18 local Yes Yes Gi0/1919 19 19 1/19 119 19 local Yes Yes Gi0/2020 20 20 1/18 120 20 local Yes Yes Gi0/2121 21 21 1/21 121 21 local Yes Yes Gi0/2222 22 22 1/20 122 22 local Yes Yes Gi0/2323 23 23 1/23 123 23 local Yes Yes Gi0/2424 24 24 1/22 124 24 local Yes Yes Gi0/2525 25 25 2/1 125 25 local Yes Yes Gi0/2626 26 26 2/0 126 26 local Yes Yes Gi0/2727 27 27 2/3 127 27 local Yes Yes Gi0/2828 28 28 2/2 128 28 local Yes Yes Gi0/2929 29 29 2/5 129 29 local Yes Yes Gi0/3030 30 30 2/4 130 30 local Yes Yes Gi0/3131 31 31 2/7 131 31 local Yes Yes Gi0/3232 32 32 2/6 132 32 local Yes Yes Gi0/3333 33 33 2/9 133 33 local Yes Yes Gi0/3434 34 34 2/8 134 34 local Yes Yes Gi0/3535 35 35 2/11 135 35 local Yes Yes Gi0/3636 36 36 2/10 136 36 local Yes Yes Gi0/3737 37 37 2/25 137 37 local Yes Yes Gi0/3838 38 38 2/24 138 38 local Yes Yes Gi0/3939 39 39 2/15 139 39 local Yes Yes Gi0/4040 40 40 2/14 140 40 local Yes Yes Gi0/4141 41 41 2/17 141 41 local Yes Yes Gi0/4242 42 42 2/16 142 42 local Yes Yes Gi0/4343 43 43 2/19 143 43 local Yes Yes Gi0/4444 44 44 2/18 144 44 local Yes Yes Gi0/4545 45 45 2/21 145 45 local Yes Yes Gi0/4646 46 46 2/20 146 46 local Yes Yes Gi0/4747 47 47 2/23 147 47 local Yes Yes Gi0/4848 48 48 2/22 148 48 local Yes Yes Gi0/4949 49 49 0/27 149 49 local Yes Yes Gi0/5050 50 50 0/19 150 50 local Yes Yes Gi0/5151 51 51 0/9 151 51 local Yes Yes Gi0/5252 52 52 0/1 152 52 local Yes Yes Te0/1 53 53 53 0/14 1153 local Yes Yes Te0/2 54 54 54 0/0 1254 local Yes Yes St1 440 440 016/0 100internal Yes Yes -- http://dcp.dcptech.com -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp- boun...@puck.nether.net] On Behalf Of Vincent Aniello Sent: Friday, September 10, 2010 1:41 PM To: Håvard Staub Nyhus Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping Unless there is only a single ASIC in a 3560E switch I do not believe this command returns the ASICs associated with each port. Here is the output on my switch: switch1#show platform pm if-numbers interface gid gpn lpn port slot unit slun port-type lpn-idb gpn-idb -- Gi0/1 1111/1 111local Yes Yes Gi0/2 2221/0 122local Yes Yes Gi0/3 3331/3 133local Yes Yes Gi0/4 4441/2 144local Yes Yes Gi0/5 5551/5 155local Yes Yes Gi0/6 6661/4 166local Yes Yes Gi0/7 77
Re: [c-nsp] ASIC to switch port mapping
Seriously look at the juniper ex platforms if you are open to other vendors. They sound to be exactly what your are asking for. On Mon, 13 Sep 2010, Pavel Skovajsa wrote: Interesting enough, yesterday James Ventre posted a note where he found at least some minimal info about the 2960/3560/3750 buffer amount: http://networking.ventrefamily.com/2010/09/3560ge-and-3750ge-buffers.html Ugh, ugly. I was hoping to find a box that could do 10Gb/s uplink and breakout as far down as 100Mb/s. Back to hunting again. wfms ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Ugh, ugly. I was hoping to find a box that could do 10Gb/s uplink and breakout as far down as 100Mb/s. Back to hunting again. Have a look at the new ME 3600X / 3800X series. Steinar Haug, Nethelp consulting, sth...@nethelp.no ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
The 4900 is 16MB shared for the whole box. The Arista 7048 (not stackable) is about the only thing close to the S60, with 768MB. Phil On Sep 13, 2010, at 2:03 PM, Nick Hilliard wrote: On 13/09/2010 17:28, Chris Evans wrote: Seriously look at the juniper ex platforms if you are open to other vendors. They sound to be exactly what your are asking for. From what I remember, the EX4200 has rather small buffers - not terribly different in size to the 3560/3750 range. This is from memory, so I could be mistaken. Juniper are rather coy on the topic, which is always a sign of relative paucity. If the box had buffer capacity which was worth mentioning, they'd mention it in the marketing blurb. If you're looking for a GE stackable with large buffers, I understand that the Force10 S60 has 1.25G shareable between 48 x 1G + 2 x 10G (about 10 megs buffer per gigabit capacity - , lots!). If you want Cisco, the 4900 range is reasonably well provisioned, although nothing like the S60. nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Gert Doering g...@greenie.muc.de writes: Now if I had more time :-) it might be worth investigating the (Linux) streaming server software used, whether it can be changed to invest a bit more CPU to better smooth out the packets... OTOH, the kernel might just wreck this, and smear it all togehter again. (*Now* we really get even more off-topic for c-nsp than usual) You can use pspacer to achieve something close to perfect smoothing of bursty traffic. /Benny ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Nick Hilliard n...@foobar.org writes: From what I remember, the EX4200 has rather small buffers - not terribly different in size to the 3560/3750 range. This is from memory, so I could be mistaken. Juniper are rather coy on the topic, which is always a sign of relative paucity. If the box had buffer capacity which was worth mentioning, they'd mention it in the marketing blurb. 3MB per PFE, according to: http://www.juniper.net/us/en/local/pdf/implementation-guides/8010073-en.pdf See table 2. I'm not sure how much buffer the 3560 actually has, just that it isn't enough. /Benny ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Hi, On Mon, Sep 13, 2010 at 10:23:24PM +0200, Benny Amorsen wrote: You can use pspacer to achieve something close to perfect smoothing of bursty traffic. Thanks for the link. I'll give it a try - it's not perfectly what we want (because it needs to know the target bitrate to shape to, instead of auto-smoothing the traffic, at least if I understood the docs right) - but in many cases we know that, like this streaming server will do 300 mbit/s max and that one 200 mbit/s max because those are contractual limits. Should have quite some interesting effects on the counters on the switch side. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de pgpF7x5xwkoB1.pgp Description: PGP signature ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On 13/09/2010 21:33, Benny Amorsen wrote: 3MB per PFE, according to: http://www.juniper.net/us/en/local/pdf/implementation-guides/8010073-en.pdf http://kb.juniper.net/KB10963 so, the 24 port model has 2 PFEs (i.e. 6M buffer space) and the 48, 3 PFEs (9 meg). That's not really very much, particularly the way that it's divvied up. I'm not sure how much buffer the 3560 actually has, just that it isn't enough. See a couple of postings back: http://networking.ventrefamily.com/2010/09/3560ge-and-3750ge-buffers.html quoth cisco: these platforms [Catalyst 3560G/3750G and 3650-E/3750E ] provide (minimally) 750 KB of receive buffers and (up to) 2 MB of transmit buffers for each set of 4 ports. Again, this looks like a simplification. It would help enormously if Cisco were to publish clear and extensive documentation as to how buffering on each of the 3560/3750 models actually works. Would anyone from Cisco on this list be able to oblige here? Or if its already published somewhere, could you point us in the right direction? Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Hi, On Fri, Sep 10, 2010 at 01:18:19PM -0400, Vincent Aniello wrote: I am trying to solve a output drops on switch ports on which bandwidth utilization does not seem to exceed the port speed. Seems like the drops are due to the buffers filling up and dropping frames. I am under the impression that each ASIC has their own buffer and if the buffer fills on a particular ASIC all ports that share that ASIC will also drop frames. If I know the switch interfaces associated with each ASIC I can redistribute the connections on the switch to better balance the load. There's material in the c-nsp archives about the buffer size issue on the 2960, 3560 and related switches. Short answer: the buffers are so tiny that Cisco doesn't even document the size anywhere. So as soon as you have microburst traffic with more ingress ports than egress, you'll see drops (turn on QoS, cut the buffers into 4 even smaller queues, increase(!) drops). Cisco does not think that this is a problem, and I have been told that the new generation 2960S and 3560E have the same size buffers. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de pgpIBzU93abPe.pgp Description: PGP signature ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
They are closet switches. If you need bigger buffers get a platform meant for heavier use such as the 4948. There are other vendors with nice offerings at a lower cost too so don't think Cisco is the only answer. Hi, On Fri, Sep 10, 2010 at 01:18:19PM -0400, Vincent Aniello wrote: I am trying to solve a output drops on switch ports on which bandwidth utilization does not seem to exceed the port speed. Seems like the drops are due to the buffers filling up and dropping frames. I am under the impression that each ASIC has their own buffer and if the buffer fills on a particular ASIC all ports that share that ASIC will also drop frames. If I know the switch interfaces associated with each ASIC I can redistribute the connections on the switch to better balance the load. There's material in the c-nsp archives about the buffer size issue on the 2960, 3560 and related switches. Short answer: the buffers are so tiny that Cisco doesn't even document the size anywhere. So as soon as you have microburst traffic with more ingress ports than egress, you'll see drops (turn on QoS, cut the buffers into 4 even smaller queues, increase(!) drops). Cisco does not think that this is a problem, and I have been told that the new generation 2960S and 3560E have the same size buffers. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025 g...@net.informatik.tu-muenchen.de ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
(I assume the response was to this or similar) On Sun, 2010-09-12 at 13:43 +0200, Gert Doering wrote: Cisco does not think that this is a problem, and I have been told that the new generation 2960S and 3560E have the same size buffers. On Sun, 2010-09-12 at 08:26 -0400, Chris Evans wrote: They are closet switches. If you need bigger buffers get a platform meant for heavier use such as the 4948. We've heard Cisco use that argument, and it's hilarious. The problem is _also_ there in the closets. Even end user PCs actually need bandwidth. We currently use 3750 and 3560E models in our datacenters, based on a recommendation from the (AFAIK) largest gold partner in our country. When whining about the buffer problem, everybody says Nexus 5k and a few say 4948. Seriously: Is it okay for Cisco to sell handicapped closet switches? It's not like they're cheap compared to others vendors or previous comparable Cisco switches (3550/2950/2970). -- Peter ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
IMHO the 3750/3560 series are way overpriced and underperforming switches. I'd honestly give the Juniper EX4200 series a look if you're looking for a direct class comparison, but looking for better performance at a lower cost. Our Cisco HTTS engineers have directly come out and said that they will not support 3750/3560 switches for data center usage in our environment. Cat4948 is the platform to choose if you're looking for a 1RU footprint stand alone unit. Cat4948 vs N5K is a non-comparison. That's talking apples to oranges in regards are you comparing a L2/L3 copper GigE switch with 10Gig uplink capability (depending on model) vs a purely L2 switch that can do Copper GigE with Fabric extenders (minus the few onboard ports that can do GigE on 5K chassis itself).. Depending on the count of devices that you need, the N5K solution is waaay more expensive. Not to mention it doesn't do L3 until the Nexus 5500 series comes out later this year and only when enabled in software Q1 of next year. The N5K being a cut-through switch with VoQ doesn't have large buffers either. They use those features to make big buffers not much of a necessity, but there is always going to be that multiple in single out issue. It's just easier with the 5K as you can throw more bandwidth at it with VPC/port-channels and its large hashing bucket capability. To put it simply, these platforms service different requirements, but overlap in ways. On Sun, Sep 12, 2010 at 1:05 PM, Peter Rathlev pe...@rathlev.dk wrote: (I assume the response was to this or similar) On Sun, 2010-09-12 at 13:43 +0200, Gert Doering wrote: Cisco does not think that this is a problem, and I have been told that the new generation 2960S and 3560E have the same size buffers. On Sun, 2010-09-12 at 08:26 -0400, Chris Evans wrote: They are closet switches. If you need bigger buffers get a platform meant for heavier use such as the 4948. We've heard Cisco use that argument, and it's hilarious. The problem is _also_ there in the closets. Even end user PCs actually need bandwidth. We currently use 3750 and 3560E models in our datacenters, based on a recommendation from the (AFAIK) largest gold partner in our country. When whining about the buffer problem, everybody says Nexus 5k and a few say 4948. Seriously: Is it okay for Cisco to sell handicapped closet switches? It's not like they're cheap compared to others vendors or previous comparable Cisco switches (3550/2950/2970). -- Peter ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On 9/12/2010 1:05 PM, Peter Rathlev wrote: Seriously: Is it okay for Cisco to sell handicapped closet switches? It's not like they're cheap compared to others vendors or previous comparable Cisco switches (3550/2950/2970) We have held on to 2950/3550s for that very purpose, where their newer counterparts present excessive drops. Rather than being pushed toward surplus, It is particularly annoying on an EMI (L3) switch actually doing routing. 2960s are especially prone to drops (esp if mls qos enabled). Jeff ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Hi Jeff On 12.09.2010, at 19:32, Jeff Kell jeff-k...@utc.edu wrote: We have held on to 2950/3550s for that very purpose, where their newer counterparts present excessive drops. Rather than being pushed toward surplus, It is particularly annoying on an EMI (L3) switch actually doing routing. 2960s are especially prone to drops (esp if mls qos enabled). Does this include 2960Gs? thanks Andrew ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Hi, On Sun, Sep 12, 2010 at 08:41:49PM +0200, Andrew Miehs wrote: 2960s are especially prone to drops (esp if mls qos enabled). Does this include 2960Gs? Yes. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de pgpjOhqMdGhCn.pgp Description: PGP signature ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
You can always buy more switches and move ports. The 2960 and the hundreds of other switches (and blades) just like it is a wiring closet switch for the enterprise. It should be common knowledge (no offense if this is new information to you) that they are oversubscribed, have tiny buffers and are not suitable for anything but. The fact is that these switches cost anywhere from $800 - $2200 and support is also cheap. This allows us all to get all the users and printers connected on the cheap. 4900's, Juniper EX's and the hundreds of other switches (and blades)that are not oversubscribed, have large queues and can switch at line rate are about $4k - $20k. It may actually be cheaper to just buy another 2960 than to upgrade to something beefier. Is this really user traffic? Is the user actually pushing 1g of traffic or are the ASICs just filling up faster than the frames can be switched off the buffers? I've never actually seen queues overrun by something that wasn't server/enterprise grade. On Sun, Sep 12, 2010 at 4:45 PM, Gert Doering g...@greenie.muc.de wrote: Hi, On Sun, Sep 12, 2010 at 08:41:49PM +0200, Andrew Miehs wrote: 2960s are especially prone to drops (esp if mls qos enabled). Does this include 2960Gs? Yes. gert -- USENET is *not* the non-clickable part of WWW! // www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025 g...@net.informatik.tu-muenchen.de ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
This is a VERY interesting topic. We need to have more attention at buffers size in our next aquisition. Thanks guy. 2010/9/12 Keegan Holley keegan.hol...@sungard.com You can always buy more switches and move ports. The 2960 and the hundreds of other switches (and blades) just like it is a wiring closet switch for the enterprise. It should be common knowledge (no offense if this is new information to you) that they are oversubscribed, have tiny buffers and are not suitable for anything but. The fact is that these switches cost anywhere from $800 - $2200 and support is also cheap. This allows us all to get all the users and printers connected on the cheap. 4900's, Juniper EX's and the hundreds of other switches (and blades)that are not oversubscribed, have large queues and can switch at line rate are about $4k - $20k. It may actually be cheaper to just buy another 2960 than to upgrade to something beefier. Is this really user traffic? Is the user actually pushing 1g of traffic or are the ASICs just filling up faster than the frames can be switched off the buffers? I've never actually seen queues overrun by something that wasn't server/enterprise grade. On Sun, Sep 12, 2010 at 4:45 PM, Gert Doering g...@greenie.muc.de wrote: Hi, On Sun, Sep 12, 2010 at 08:41:49PM +0200, Andrew Miehs wrote: 2960s are especially prone to drops (esp if mls qos enabled). Does this include 2960Gs? Yes. gert -- USENET is *not* the non-clickable part of WWW! // www.muc.de/~gert/ http://www.muc.de/%7Egert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025 g...@net.informatik.tu-muenchen.de ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ -- []'s Lívio Zanol Puppim ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On Sunday, September 12, 2010 07:43:26 pm Gert Doering wrote: Cisco does not think that this is a problem, and I have been told that the new generation 2960S and 3560E have the same size buffers. Probably because these are Enterprise switches, and enterprise-anything shouldn't be trying to do provider- anything :-). Oh well... Mark. signature.asc Description: This is a digitally signed message part. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] ASIC to switch port mapping
I am trying to determine the switch ports assigned to each ASIC in various Cisco switches, in particular a 3750 and 3560E. Can anyone enlighten me on how to go about doing this? Thanks. --Vincent Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Hi Vincent 1) Obtain screwdriver 2) Remove case 3) Trace tracks... :) On a serious note, it is actually probably the best way to do it. What are you trying to achieve/solve/learn? Heath On 10 September 2010 15:13, Vincent Aniello vincent.anie...@pipelinefinancial.com wrote: I am trying to determine the switch ports assigned to each ASIC in various Cisco switches, in particular a 3750 and 3560E. Can anyone enlighten me on how to go about doing this? Thanks. --Vincent Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On 10/09/2010 16:20, Heath Jones wrote: On a serious note, it is actually probably the best way to do it. +1 Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
I am trying to determine the switch ports assigned to each ASIC in various Cisco switches, in particular a 3750 and 3560E. Can anyone enlighten me on how to go about doing this? show platform pm if-numbers In the port column, look at the first number. -- Håvard Staub Nyhus +47 41 88 00 99 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Is there also a command for the 6500 that does this? It's of interest to me because some features like VLAN translation work on groups of ports on a common ASIC... -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Håvard Staub Nyhus Sent: Friday, September 10, 2010 10:33 AM To: Vincent Aniello Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping I am trying to determine the switch ports assigned to each ASIC in various Cisco switches, in particular a 3750 and 3560E. Can anyone enlighten me on how to go about doing this? show platform pm if-numbers In the port column, look at the first number. -- Håvard Staub Nyhus +47 41 88 00 99 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ smime.p7s Description: S/MIME cryptographic signature ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On 10/09/2010 17:16, Murphy, William wrote: Is there also a command for the 6500 that does this? It's of interest to me because some features like VLAN translation work on groups of ports on a common ASIC... show interface Gi x/y capabilities | i ASIC Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
I was hoping there was an easier way. J I am trying to solve a output drops on switch ports on which bandwidth utilization does not seem to exceed the port speed. Seems like the drops are due to the buffers filling up and dropping frames. I am under the impression that each ASIC has their own buffer and if the buffer fills on a particular ASIC all ports that share that ASIC will also drop frames. If I know the switch interfaces associated with each ASIC I can redistribute the connections on the switch to better balance the load. Thanks. --Vincent From: Heath Jones [mailto:hj1...@gmail.com] Sent: Friday, September 10, 2010 11:20 AM To: Vincent Aniello Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping Hi Vincent 1) Obtain screwdriver 2) Remove case 3) Trace tracks... :) On a serious note, it is actually probably the best way to do it. What are you trying to achieve/solve/learn? Heath On 10 September 2010 15:13, Vincent Aniello vincent.anie...@pipelinefinancial.com wrote: I am trying to determine the switch ports assigned to each ASIC in various Cisco switches, in particular a 3750 and 3560E. Can anyone enlighten me on how to go about doing this? Thanks. --Vincent Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Unless there is only a single ASIC in a 3560E switch I do not believe this command returns the ASICs associated with each port. Here is the output on my switch: switch1#show platform pm if-numbers interface gid gpn lpn port slot unit slun port-type lpn-idb gpn-idb -- Gi0/1 1111/1 111local Yes Yes Gi0/2 2221/0 122local Yes Yes Gi0/3 3331/3 133local Yes Yes Gi0/4 4441/2 144local Yes Yes Gi0/5 5551/5 155local Yes Yes Gi0/6 6661/4 166local Yes Yes Gi0/7 7771/9 177local Yes Yes Gi0/8 8881/8 188local Yes Yes Gi0/9 9991/11 199local Yes Yes Gi0/1010 10 10 1/10 110 10 local Yes Yes Gi0/1111 11 11 1/13 111 11 local Yes Yes Gi0/1212 12 12 1/12 112 12 local Yes Yes Gi0/1313 13 13 1/15 113 13 local Yes Yes Gi0/1414 14 14 1/14 114 14 local Yes Yes Gi0/1515 15 15 1/17 115 15 local Yes Yes Gi0/1616 16 16 1/16 116 16 local Yes Yes Gi0/1717 17 17 1/19 117 17 local Yes Yes Gi0/1818 18 18 1/18 118 18 local Yes Yes Gi0/1919 19 19 1/23 119 19 local Yes Yes Gi0/2020 20 20 1/22 120 20 local Yes Yes interface gid gpn lpn port slot unit slun port-type lpn-idb gpn-idb -- Gi0/2121 21 21 1/25 121 21 local Yes Yes Gi0/2222 22 22 1/24 122 22 local Yes Yes Gi0/2323 23 23 1/27 123 23 local Yes Yes Gi0/2424 24 24 1/26 124 24 local Yes Yes Gi0/2525 25 25 0/20 125 25 local Yes Yes Gi0/2626 26 26 0/21 126 26 local Yes Yes Gi0/2727 27 27 0/6 127 27 local Yes Yes Gi0/2828 28 28 0/7 128 28 local Yes Yes Te0/1 29 29 29 0/14 1129 local Yes Yes Te0/2 30 30 30 0/0 1230 local Yes Yes St1 440 440 016/0 100internal Yes Yes Thanks. --Vincent -Original Message- From: Håvard Staub Nyhus [mailto:hny...@gmail.com] Sent: Friday, September 10, 2010 11:33 AM To: Vincent Aniello Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping I am trying to determine the switch ports assigned to each ASIC in various Cisco switches, in particular a 3750 and 3560E. Can anyone enlighten me on how to go about doing this? show platform pm if-numbers In the port column, look at the first number. -- Håvard Staub Nyhus +47 41 88 00 99 Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
This is on a 3650E switch. Thanks. --Vincent -Original Message- From: Nick Hilliard [mailto:n...@foobar.org] Sent: Friday, September 10, 2010 1:31 PM To: Vincent Aniello Cc: Heath Jones; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping On 10/09/2010 18:18, Vincent Aniello wrote: I am trying to solve a output drops on switch ports on which bandwidth utilization does not seem to exceed the port speed. Seems like the drops are due to the buffers filling up and dropping frames. I am under the impression that each ASIC has their own buffer and if the buffer fills on a particular ASIC all ports that share that ASIC will also drop frames. If I know the switch interfaces associated with each ASIC I can redistribute the connections on the switch to better balance the load. What sort of card are you using? Nick Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 9/10/2010 12:16 PM, Murphy, William wrote: Is there also a command for the 6500 that does this? It's of interest to me because some features like VLAN translation work on groups of ports on a common ASIC... Check out http://networking.ventrefamily.com/2010/08/asic-to-port-mappings.html - -- Devon -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.12 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyKZ2oACgkQWP2WrBTHBS9+mwCePGEuVvswToXHOQicBHMapDA/ 90YAoKDDfbsMg80Qmf5ezBlEoSdJbTCD =4vx3 -END PGP SIGNATURE- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
sh platform port-asic should list your ASIC's port groupings are almost always in groups, so you can work out what ports belong to a common ASIC by dividing the amount of ports you have by the amount of ASIC's listed, keep in mind you will probably have a dedicated ASIC for the 2 10G uplinks. On Fri, Sep 10, 2010 at 6:50 PM, Vincent Aniello vincent.anie...@pipelinefinancial.com wrote: This is on a 3650E switch. Thanks. --Vincent -Original Message- From: Nick Hilliard [mailto:n...@foobar.org] Sent: Friday, September 10, 2010 1:31 PM To: Vincent Aniello Cc: Heath Jones; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping On 10/09/2010 18:18, Vincent Aniello wrote: I am trying to solve a output drops on switch ports on which bandwidth utilization does not seem to exceed the port speed. Seems like the drops are due to the buffers filling up and dropping frames. I am under the impression that each ASIC has their own buffer and if the buffer fills on a particular ASIC all ports that share that ASIC will also drop frames. If I know the switch interfaces associated with each ASIC I can redistribute the connections on the switch to better balance the load. What sort of card are you using? Nick Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
So it looks like a WS-C3560E-24TD has two ASICs: switch#show platform port-asic version Port-Asic Version Info: ASIC-0: Version:4 DeviceType:0x30A ASIC-1: Version:4 DeviceType:0x30A Also, the show platform pm if-numbers command does return the ports assigned with each ASIC, I overlooked the '0' in the port column because I expected to see more than two ASICs. --Vincent From: illcrit...@gmail.com [mailto:illcrit...@gmail.com] On Behalf Of Ben Steele Sent: Friday, September 10, 2010 2:06 PM To: Vincent Aniello Cc: Nick Hilliard; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping sh platform port-asic should list your ASIC's port groupings are almost always in groups, so you can work out what ports belong to a common ASIC by dividing the amount of ports you have by the amount of ASIC's listed, keep in mind you will probably have a dedicated ASIC for the 2 10G uplinks. On Fri, Sep 10, 2010 at 6:50 PM, Vincent Aniello vincent.anie...@pipelinefinancial.com wrote: This is on a 3650E switch. Thanks. --Vincent -Original Message- From: Nick Hilliard [mailto:n...@foobar.org] Sent: Friday, September 10, 2010 1:31 PM To: Vincent Aniello Cc: Heath Jones; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping On 10/09/2010 18:18, Vincent Aniello wrote: I am trying to solve a output drops on switch ports on which bandwidth utilization does not seem to exceed the port speed. Seems like the drops are due to the buffers filling up and dropping frames. I am under the impression that each ASIC has their own buffer and if the buffer fills on a particular ASIC all ports that share that ASIC will also drop frames. If I know the switch interfaces associated with each ASIC I can redistribute the connections on the switch to better balance the load. What sort of card are you using? Nick Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
That's a very interesting problem, I'd love to hear what you find! Lots of different sized frames from different ingress interfaces all heading to a few egress interfaces on the same asic perhaps? Are you seeing an interface that is highly utilized - I'm imagining you are and this is driving the theory? On 10 September 2010 18:18, Vincent Aniello vincent.anie...@pipelinefinancial.com wrote: I was hoping there was an easier way. J I am trying to solve a output drops on switch ports on which bandwidth utilization does not seem to exceed the port speed. Seems like the drops are due to the buffers filling up and dropping frames. I am under the impression that each ASIC has their own buffer and if the buffer fills on a particular ASIC all ports that share that ASIC will also drop frames. If I know the switch interfaces associated with each ASIC I can redistribute the connections on the switch to better balance the load. Thanks. --Vincent *From:* Heath Jones [mailto:hj1...@gmail.com] *Sent:* Friday, September 10, 2010 11:20 AM *To:* Vincent Aniello *Cc:* cisco-nsp@puck.nether.net *Subject:* Re: [c-nsp] ASIC to switch port mapping Hi Vincent 1) Obtain screwdriver 2) Remove case 3) Trace tracks... :) On a serious note, it is actually probably the best way to do it. What are you trying to achieve/solve/learn? Heath On 10 September 2010 15:13, Vincent Aniello vincent.anie...@pipelinefinancial.com wrote: I am trying to determine the switch ports assigned to each ASIC in various Cisco switches, in particular a 3750 and 3560E. Can anyone enlighten me on how to go about doing this? Thanks. --Vincent Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On 10/09/2010 18:40, Vincent Aniello wrote: Unless there is only a single ASIC in a 3560E switch I do not believe this command returns the ASICs associated with each port. Here is the output on my switch: oh, 3650. hmmm. Unfortunately, these switches have very small buffers indeed. If you microburst above what their buffers can handle, then yes - you will drop packets. I'd suggest that you check your outbound port usage. If there are any ports which are consistently or inconsistently high, or if you're seeing any ports with buffer drops, that you upgrade those connections. Alternatively, get a switch with larger per-port buffers. Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
Any recommendations on a switch with larger buffers? I would like to stick with the 1U form factor. Also, are you saying that the microbursts cause the switch to exceed 1Gb port speed which causes the drops? Cisco claims that the WS-C3560E-24TD-E switch has a 128-Gbps wire rate non-blocking switching fabric capacity. I may be oversimplifying this, but to me, Cisco's claim implies that I should be able to run every port on this switch simultaneously at a full 1Gb in both directions. Thanks. --Vincent -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Nick Hilliard Sent: Friday, September 10, 2010 4:11 PM To: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping On 10/09/2010 18:40, Vincent Aniello wrote: Unless there is only a single ASIC in a 3560E switch I do not believe this command returns the ASICs associated with each port. Here is the output on my switch: oh, 3650. hmmm. Unfortunately, these switches have very small buffers indeed. If you microburst above what their buffers can handle, then yes - you will drop packets. I'd suggest that you check your outbound port usage. If there are any ports which are consistently or inconsistently high, or if you're seeing any ports with buffer drops, that you upgrade those connections. Alternatively, get a switch with larger per-port buffers. Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
4948 is a good choice. Any recommendations on a switch with larger buffers? I would like to stick with the 1U form factor. Also, are you saying that the microbursts cause the switch to exceed 1Gb port speed which causes the drops? Cisco claims that the WS-C3560E-24TD-E switch has a 128-Gbps wire rate non-blocking switching fabric capacity. I may be oversimplifying this, but to me, Cisco's claim implies that I should be able to run every port on this switch simultaneously at a full 1Gb in both directions. Thanks. --Vincent -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Nick Hilliard Sent: Friday, September 10, 2010 4:11 PM To: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASIC to switch port mapping On 10/09/2010 18:40, Vincent Aniello wrote: Unless there is only a single ASIC in a 3560E switch I do not believe this command returns the ASICs associated with each port. Here is the output on my switch: oh, 3650. hmmm. Unfortunately, these switches have very small buffers indeed. If you microburst above what their buffers can handle, then yes - you will drop packets. I'd suggest that you check your outbound port usage. If there are any ports which are consistently or inconsistently high, or if you're seeing any ports with buffer drops, that you upgrade those connections. Alternatively, get a switch with larger per-port buffers. Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Disclaimer: Any references to Pipeline performance contained herein are based on internal testing and / or historic performance levels which Pipeline expects to maintain or exceed but nevertheless does not guarantee. Congested networks, price volatility, or other extraordinary events may impede future trading activities and degrade performance statistics. Pipeline is a member of FINRA and SIPC. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On 9/10/2010 5:47 PM, Vincent Aniello wrote: Any recommendations on a switch with larger buffers? I would like to stick with the 1U form factor. Also, are you saying that the microbursts cause the switch to exceed 1Gb port speed which causes the drops? Cisco claims that the WS-C3560E-24TD-E switch has a 128-Gbps wire rate non-blocking switching fabric capacity. I may be oversimplifying this, but to me, Cisco's claim implies that I should be able to run every port on this switch simultaneously at a full 1Gb in both directions. Stick one 100Mb or 10Mb device on it and all those bets are off. It's still store-and-forward and you can only buffer 1Gb-input-to-less-Mb-output for so long. Jeff ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASIC to switch port mapping
On 10/09/2010 22:47, Vincent Aniello wrote: Any recommendations on a switch with larger buffers? I would like to stick with the 1U form factor. Also, are you saying that the microbursts cause the switch to exceed 1Gb port speed which causes the drops? Cisco claims that the WS-C3560E-24TD-E switch has a 128-Gbps wire rate non-blocking switching fabric capacity. I may be oversimplifying this, but to me, Cisco's claim implies that I should be able to run every port on this switch simultaneously at a full 1Gb in both directions. Yes, probably you can inject a constant 2,976,190 x 64 byte frames per second on all ports and it won't lose any packets. Thing is, that's not a very interesting measurement as far as switch performance is concerned, because normal traffic is very bursty and doesn't behave like lab traffic injected on a traffic generator. What's interesting is what happens when you have line rate going on on a single port, and then you receive frames on other ports which are destined to the full port. From a simplistic point of view, when this happens on a store-n-forward switch like a 3560, you need buffers to store the incoming packets - because the switch cannot send them to the destination port which is full. If for some reason you run out of buffer space on the input ports, the switch will drop incoming packets. Of course, there are several ways of mitigating against this sort of problem (virtual queues, QoS etc), but they only get you so far. Cisco are notoriously cagey about exactly how much rx/tx buffer space is available on the 3560/3750 series boxes and how it's divvied up, but there is lots of speculation about this on cisco-nsp. Here's some: http://puck.nether.net/pipermail/cisco-nsp/2010-March/068810.html Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
