Re: Super TSO performance drop

[Dmitry Yusupov]

For jumbo traffic, if cong. window is big, than TSO defer will not
happen that often. Hence, most of the traffic will be non-TSO and that
is why we saw performance degradation on our setup. This was the case
for 10G network where we tend to set tcp window very big, i.e. 1M+
This patch forces to defer even more. I'm not sure if this is clean
enough, but here goes results:

Without patch:

OS on Tx system TSO   ThroughputCPU idle
--- ---   ---   --
2.6.13-rc6  ON5.93Gb/s  58%
OFF   6.16Gb/s  52%

With patch:

OS on Tx system TSO   ThroughputCPU idle
--- ---   ---   --
2.6.13-rc6  ON6.59Gb/s  57%
(patched)   OFF   6.16Gb/s  52%


Signed-off-by: Dmitry Yusupov [EMAIL PROTECTED]

diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -925,10 +925,6 @@ static int tcp_tso_should_defer(struct s
 
limit = min(send_win, cong_win);
 
-   /* If sk_send_head can be sent fully now, just do it.  */
-   if (skb-len = limit)
-   return 0;
-
if (sysctl_tcp_tso_win_divisor) {
u32 chunk = min(tp-snd_wnd, tp-snd_cwnd * tp-mss_cache);
 


On Thu, 2005-08-11 at 16:15 -0700, David S. Miller wrote:
 From: Leonid Grossman [EMAIL PROTECTED]
 Date: Thu, 11 Aug 2005 19:05:22 -0400
 
  Basically, it looks like with SuperTSO most of the traffic in our tests
  comes down to the driver with mss 0 (TSO is mostly off).
  With the original TSO, is was always on (see below).
  Could you describe how SuperTSO algorithm for switching TSO on and
  off works, it seems to be fairly complex for us to mess with? For a
  fast network with extremely low error rate, I'd expect TSO to be on.
 
 We build packets as large as the user gives us in sendmsg(),
 then when ACK's come in openning up the window we either
 send as much as we can at that moment, or we defer to wait
 for more space so we can build larger TSO frames.
 
 The logic for that decision is in:
 
 net/ipv4/tcp_output.c:tcp_tso_should_defer()
 
 Keep also in mind that if there is packet loss, TSO is disabled
 completely.
 

-
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Super TSO performance drop

[Dmitry Yusupov]

On Thu, 2005-08-11 at 16:15 -0700, David S. Miller wrote:
 From: Leonid Grossman [EMAIL PROTECTED]
 Date: Thu, 11 Aug 2005 19:05:22 -0400
 
  Basically, it looks like with SuperTSO most of the traffic in our tests
  comes down to the driver with mss 0 (TSO is mostly off).
  With the original TSO, is was always on (see below).
  Could you describe how SuperTSO algorithm for switching TSO on and
  off works, it seems to be fairly complex for us to mess with? For a
  fast network with extremely low error rate, I'd expect TSO to be on.
 
 We build packets as large as the user gives us in sendmsg(),
 then when ACK's come in openning up the window we either
 send as much as we can at that moment, or we defer to wait
 for more space so we can build larger TSO frames.

But even with new TSO schema, this logic can not send more than
MAX_SKB_FRAGS which at best case ~60K. With old TSO schema.
I thought with SuperTSO we could send  than 60K. And than it could be
very beneficial for us on some platforms where DMA setup is very cheap,
since we could xmit up to 256 pages at once.

 The logic for that decision is in:
 
 net/ipv4/tcp_output.c:tcp_tso_should_defer()

ok, I'll take a look.

 Keep also in mind that if there is packet loss, TSO is disabled
 completely.

I'll double check that too.

Thanks!
Dima

-
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html