On Tue, 2013-07-09 at 14:57 +0100, Sam Crawford wrote: > Hi all, > > This issue persists unfortunately. Attached is a log from an instrumented > TCP server (the sender), logging CWND values and the retransmits. This has > been run on two identical servers on the same switch - one at 100Mbit and > the other at 1Gbit. You can see that a small amount of losses occur after > 1-2 seconds with the 1Gbit setup, limiting the congestion window to ~200 > MSSs. The 100Mbit server is able to hit a CWND of 1092 stably. These > results are highly repeatable. > > TSO/GSO/GRO are disabled on all hosts. Packet captures from both ends are > available upon request. > > Any suggestions gratefully received!
What you are seeing is pretty normal, as the standard pfifo_fast qdisc allows a queue of up to 1000 packets. Using 100Mbit with such amount of queueing 'allows' RTT to grow at insane levels. And since you are below the nominal WAN bandwidth, you get no packet losses and 'optimal' tcp throughput. As soon as you allow tcp sender to send more packets than real bandwidth, you experiment packet losses, and if RTT is big, performance sunks badly (Depending if SACK and/or tcp timestamps are enabled) I advise you use a rate limiter, using HTB + fq_codel. (Allow your LAN traffic to reach 1Gb, but shape the traffic meant for WAN to 100Mbits) BTW, 100ms RTT doesn't need 8MB of TCP buffers to fill the pipe. You only add bufferbloat. You theoretically need 1.25 MB (10.000.000 bits) And the ramp up should be much faster than 60 sec ! ------------------------------------------------------------------------------ See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired