This isn't exactly the problem I am seeing.. I actually set up a windows
server and it shows the same result
as the centos server which leads me to believe it's not a driver issue
with centos.
None of our connections are overflowing, the transfer doesn't even start
out fast. It's going through all gigabit or higher
ports the entire way.
One particular transfer I get 1.3MB/s every time, consistently, and if i
disable TSO/GSO i get 8-9MB/s average but during
the transfer the rate jumps up and down a lot. (this is on centos
using ethtool -K eth0 tso off gso off)
Windows gets the 1.3MB/s but i haven't tried disabling tso/gso yet.
If I set the port to 100mbits, both max it out no problem.
Locally where latency is < 1ms both come near maxing out the gigabit
port (probably hard drive limitation)
Is there any utility that will test this end to end ? I've used iperf
to do loss/transfer tests. What kills me is that the servers
i'm using to test with can download at 300-400mbits from the server on
the other end that im using to test with but can only upload
at 10mbits. One would think, that if a server on level3 for example in
one location and another server on level3 in another location
both on gigabit ports, should get a good rate both directions. And of
course if i set it to 100m, it gets 100m and not 10m..
I'm still stumped by this issue.
Wouldn't having a server on the other end at 1gbit and using a 100m port
to upload with cause more packet drops than having gigabit on
the uploading server? since it maxes out the 100m port
Arie Vayner (avayner) wrote:
Joe, this is exactly the phenomena I was referring to. It can be
controlled with applying shaping on platforms that can support this kind
of QOS policy (requires large buffers).
Usually available on WAN routers, specific switches or requires specific
modules on some other switches.
Arie
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Joe Loiacono
Sent: Monday, June 28, 2010 17:11
To: Paul
Cc: [email protected]; [email protected]
Subject: Re: [c-nsp] Centos upload speed slower on 1000m than 100m over
WANlinks
OK I'm jumping in on this thread late as I just got back from some
vacation, don't know if this particluar observation has been discussed,
but ...
We've seen this problem a lot when moving up to new local connection
speeds. The problem for us has been that unless the entire path can
support the new speed (e.g., 1G) switches down the path that connect to
slower speeds (e.g. 100M) will overflow and put your data transfer into
TCP slow-start recovery. As soon as the sending NIC is 'downgraded'
(e.g.,
back to 100 M) the overflows disappear, slow-start is avoided, and
performance improves. Bitterly ironic.
Joe
From:
Paul <[email protected]>
To:
[email protected]
Date:
06/27/2010 03:08 AM
Subject:
[c-nsp] Centos upload speed slower on 1000m than 100m over WAN links
I'm not even sure this is the right forum but since we use mainly Cisco
equipment I'll give this a shot. :)
I have tried several centos based servers and compiled various kernels
and the results have been extremely weird.
90% of the cases the remote hosts can download from a server at
1-5megabytes per second, and most of these are over
the internet ranging from 30-200ms away. Local (1ms or less) is super
fast 100MB/s for example.
Ok that sounds normal since it's going over the internet, etc. But
here's the )(!...@*! part..
If I set the port speed to 100 megabits full duplex on the switch and
server , the clients that get 1-5MB/s now get 11MB/s which is
approximately the limit of the 100mbit port.
Totally stumped here, tried different nics, servers, even 4 different
switches. Is a very interesting problem and I'm probing to see
if anyone else has encountered it.
So far the only OS i have tried is centos, but different versions and
kernels and hardware.
All the switches/routers are Cisco based, but I seriously doubt that has
anything to do with this. :P
--
GloboTech Communications
Phone: 1-514-907-0050 x 215
Toll Free: 1-(888)-GTCOMM1
Fax: 1-(514)-907-0750
[email protected]
http://www.gtcomm.net
_______________________________________________
cisco-nsp mailing list [email protected]
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
