Re: [c-nsp] Transfer speed issues on 3560G
ethtool might help getting the Speed/Duplex settings for that interface on the server: e.g. ethtool eth0 -Azher Brandon Ewing wrote: This is a strange issue that I have noticed on a 3560G that we have deployed. We have two servers, on different ports, controlled by different ASICs. Each port negotiates a 1000mb/s link, but I cannot get more than 11MB/s (88mb/s) of traffic between the two ports. I conducted the following tests: Transferring a 1GB file from one server to the other, written to /dev/null Single transfer averaged 11.2MB/s Performed two simultaneous transfers from one server to the other, written to /dev/null. One transfer completed at 2MB/s, the other at 9.3MB/s. This happens consistently, every test. Performed two simultaneous transfers -- one between the two servers on the switch, and one from another server off-switch to one of the servers. Both transfers maxed out at 11MB/s This doesn't appear to be a TCP windowing issue, due to the fact that doubling the number of TCP sessions did not result in a net increase of overall speed. It appears that any flow in between two ports can only reach 100mb/s. Anyone have any idea where I can look to find the root cause? ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Transfer speed issues on 3560G
Thanks to all the replies, on and off list. There is no QoS configured on the switch currently. mls qos isn't in the config. Adding srr-queue bandwidth commands to the ports did not improve the situation. The servers in question are not on the same vLAN, we're routing between SVIs. I also tested with UDP, and got the same results as before. If anyone has any additional ideas as to what to check, it would be appreciated. -- Brandon Ewing(nicot...@warningg.com) pgpt2OBdBnRIa.pgp Description: PGP signature ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Transfer speed issues on 3560G
Check whether you are not running into high CPU issues due to IRQ, due to wrong SDM profile used.See http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00801e7bb9.shtml -pavel On Fri, Jun 25, 2010 at 5:52 PM, Brandon Ewing nicot...@warningg.comwrote: Thanks to all the replies, on and off list. There is no QoS configured on the switch currently. mls qos isn't in the config. Adding srr-queue bandwidth commands to the ports did not improve the situation. The servers in question are not on the same vLAN, we're routing between SVIs. I also tested with UDP, and got the same results as before. If anyone has any additional ideas as to what to check, it would be appreciated. -- Brandon Ewing( nicot...@warningg.com) ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Transfer speed issues on 3560G
On Fri, Jun 25, 2010 at 6:46 AM, Brandon Ewing nicot...@warningg.com wrote: This is a strange issue that I have noticed on a 3560G that we have deployed. We have two servers, on different ports, controlled by different ASICs. Each port negotiates a 1000mb/s link, but I cannot get more than 11MB/s (88mb/s) of traffic between the two ports. I conducted the following tests: pls check the port speed, it seems a 100mb/s link. We have a old server, the NIC works at 100mb/s when connect to a 6509 or other host. -- Zhang Huanjie ja...@ustc.edu.cn +86-551-3601897 University of Science and Technology of China ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] Transfer speed issues on 3560G
This is a strange issue that I have noticed on a 3560G that we have deployed. We have two servers, on different ports, controlled by different ASICs. Each port negotiates a 1000mb/s link, but I cannot get more than 11MB/s (88mb/s) of traffic between the two ports. I conducted the following tests: Transferring a 1GB file from one server to the other, written to /dev/null Single transfer averaged 11.2MB/s Performed two simultaneous transfers from one server to the other, written to /dev/null. One transfer completed at 2MB/s, the other at 9.3MB/s. This happens consistently, every test. Performed two simultaneous transfers -- one between the two servers on the switch, and one from another server off-switch to one of the servers. Both transfers maxed out at 11MB/s This doesn't appear to be a TCP windowing issue, due to the fact that doubling the number of TCP sessions did not result in a net increase of overall speed. It appears that any flow in between two ports can only reach 100mb/s. Anyone have any idea where I can look to find the root cause? -- Brandon Ewing(nicot...@warningg.com) pgpxxUBJInuQ0.pgp Description: PGP signature ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Transfer speed issues on 3560G
On Thu, 24 Jun 2010, Brandon Ewing wrote: This is a strange issue that I have noticed on a 3560G that we have deployed. We have two servers, on different ports, controlled by different ASICs. Each port negotiates a 1000mb/s link, but I cannot get more than 11MB/s (88mb/s) of traffic between the two ports. I conducted the following tests: Transferring a 1GB file from one server to the other, written to /dev/null Single transfer averaged 11.2MB/s What are you using to do the transfers? What are the srr-queue bandwidth settings for the ports involved? Are you doing anything special with CoS or DSCP? The 3560 has some silly default behavior for shaping/sharing of bandwidth between the 4 queues per port. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Transfer speed issues on 3560G
Duplex mis-match? Have you checked the interface stats on both ends? Have you tried to force 1000/full on all interfaces concerned? -b -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Jon Lewis Sent: Thursday, June 24, 2010 5:55 PM To: Brandon Ewing Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Transfer speed issues on 3560G On Thu, 24 Jun 2010, Brandon Ewing wrote: This is a strange issue that I have noticed on a 3560G that we have deployed. We have two servers, on different ports, controlled by different ASICs. Each port negotiates a 1000mb/s link, but I cannot get more than 11MB/s (88mb/s) of traffic between the two ports. I conducted the following tests: Transferring a 1GB file from one server to the other, written to /dev/null Single transfer averaged 11.2MB/s ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Transfer speed issues on 3560G
Sorry about top posting. Try to transfer a large file via ftp between the two hosts using the hash '-h' switch. If the hashes are choppy, then that would be indicative of a dup mis-match. -b -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Bill Blackford Sent: Thursday, June 24, 2010 6:50 PM To: Brandon Ewing Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Transfer speed issues on 3560G Duplex mis-match? Have you checked the interface stats on both ends? Have you tried to force 1000/full on all interfaces concerned? -b -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Jon Lewis Sent: Thursday, June 24, 2010 5:55 PM To: Brandon Ewing Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Transfer speed issues on 3560G On Thu, 24 Jun 2010, Brandon Ewing wrote: This is a strange issue that I have noticed on a 3560G that we have deployed. We have two servers, on different ports, controlled by different ASICs. Each port negotiates a 1000mb/s link, but I cannot get more than 11MB/s (88mb/s) of traffic between the two ports. I conducted the following tests: Transferring a 1GB file from one server to the other, written to /dev/null Single transfer averaged 11.2MB/s ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Transfer speed issues on 3560G
I wonder what kind of speed you will get if you connected the two server's NIC directly to each other and did the test so you can take the switch out of the equation. On Thu, Jun 24, 2010 at 10:00 PM, Bill Blackford bblackf...@nwresd.k12.or.us wrote: Sorry about top posting. Try to transfer a large file via ftp between the two hosts using the hash '-h' switch. If the hashes are choppy, then that would be indicative of a dup mis-match. -b -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Bill Blackford Sent: Thursday, June 24, 2010 6:50 PM To: Brandon Ewing Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Transfer speed issues on 3560G Duplex mis-match? Have you checked the interface stats on both ends? Have you tried to force 1000/full on all interfaces concerned? -b -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Jon Lewis Sent: Thursday, June 24, 2010 5:55 PM To: Brandon Ewing Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Transfer speed issues on 3560G On Thu, 24 Jun 2010, Brandon Ewing wrote: This is a strange issue that I have noticed on a 3560G that we have deployed. We have two servers, on different ports, controlled by different ASICs. Each port negotiates a 1000mb/s link, but I cannot get more than 11MB/s (88mb/s) of traffic between the two ports. I conducted the following tests: Transferring a 1GB file from one server to the other, written to /dev/null Single transfer averaged 11.2MB/s ___ cisco-nsp mailing list cisco-...@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-...@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-...@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
