Re: 2.6.15 localhost performance hit

2006-03-20 Thread Rick Jones
And there you have it - four separate sends for logically associated data, when there should have been just the one sendmsg or writev call. Given the sizes involved, that is roughly three to four times as much CPU consumed to send that data is should have been - for small packets you can

Re: 2.6.15 localhost performance hit

2006-03-20 Thread Skunk Worx
Rick Jones wrote: It _would_ be very interesting to see if disabling tcp_abc does workaround the problem. Yes it does, we tried that first. Then we reenabled tcp_abc and increased the toolkit keep_alive. Either method works around the issue from our point of view. --- John - To

Re: 2.6.15 localhost performance hit

2006-03-20 Thread Stephen Hemminger
On Mon, 20 Mar 2006 11:56:32 -0800 Skunk Worx [EMAIL PROTECTED] wrote: Rick Jones wrote: It _would_ be very interesting to see if disabling tcp_abc does workaround the problem. Yes it does, we tried that first. Then we reenabled tcp_abc and increased the toolkit keep_alive. Either

Re: 2.6.15 localhost performance hit

2006-03-17 Thread Skunk Worx
Stephen Hemminger wrote: On Wed, 15 Mar 2006 20:13:01 -0800 Skunk Worx [EMAIL PROTECTED] wrote: Hello, I've taken a performance hit over localhost between kernels 2.6.14 and 2.6.15 in my client/server application. This came up with java debugging already. The problem is when the sender

Re: 2.6.15 localhost performance hit

2006-03-17 Thread Rick Jones
Skunk Worx wrote: Rick Jones wrote: From strace : 15:27:04.568800 recv(3, ?xml version=\1.0\ encoding=\UT..., 555, ) = 555 0.000121 vs. 15:18:24.515891 recv(3, ?xml version=\1.0\ encoding=\UT..., 566, ) = 566 0.038414 Will watch replies and post more when I know more. Kinda new at

Re: 2.6.15 localhost performance hit

2006-03-17 Thread Rick Jones
I am curious about why this is seen only on loopback. Over ether, with client and server on two machines, there is no 40ms. ack penalty. If anything, I would expect to see things the other way around--penalty on ether, no penalty on loopback. What is the MTU of the loopback interface? The

Re: 2.6.15 localhost performance hit

2006-03-17 Thread Skunk Worx
Rick Jones wrote: Skunk Worx wrote: Rick Jones wrote: From strace : 15:27:04.568800 recv(3, ?xml version=\1.0\ encoding=\UT..., 555, ) = 555 0.000121 vs. 15:18:24.515891 recv(3, ?xml version=\1.0\ encoding=\UT..., 566, ) = 566 0.038414 Will watch replies and post more when I know

Re: 2.6.15 localhost performance hit

2006-03-16 Thread Rick Jones
This came up with java debugging already. Interesting how this has come-up a second time isn't it. The problem is when the sender writes a message in separate write() system calls, each one becomes a separate packet. And indeed, that is doubleplusungood and should be fixed in the

Re: 2.6.15 localhost performance hit

2006-03-16 Thread Ben Greear
Stephen Hemminger wrote: On Wed, 15 Mar 2006 20:13:01 -0800 Skunk Worx [EMAIL PROTECTED] wrote: Hello, I've taken a performance hit over localhost between kernels 2.6.14 and 2.6.15 in my client/server application. I'm trying to gut things down to a simple test case, in the meantime, this

Re: 2.6.15 localhost performance hit

2006-03-16 Thread Stephen Hemminger
Also, even if the TCP connection stalls for the ACK, why does it take multiple miliseconds for the ACK to come back over the localhost? Thanks, Ben NODELAY effects whether we send the second message right away (without waiting for the first ACK). Normal (small writer):

Re: 2.6.15 localhost performance hit

2006-03-16 Thread Rick Jones
From strace : 15:27:04.568800 recv(3, ?xml version=\1.0\ encoding=\UT..., 555, ) = 555 0.000121 vs. 15:18:24.515891 recv(3, ?xml version=\1.0\ encoding=\UT..., 566, ) = 566 0.038414 Will watch replies and post more when I know more. Kinda new at this. Do you have the strace from the

Re: 2.6.15 localhost performance hit

2006-03-16 Thread Ben Greear
Stephen Hemminger wrote: Also, even if the TCP connection stalls for the ACK, why does it take multiple miliseconds for the ACK to come back over the localhost? Thanks, Ben NODELAY effects whether we send the second message right away (without waiting for the first ACK). And it also

Re: 2.6.15 localhost performance hit

2006-03-16 Thread Rick Jones
I thought that in the java case the problem only occurred because TCP_NODELAY was set? Perhaps/probably, since they were sending something in five spearate peices and so were semistuck until cwnd grew to five packets from the initial three. However, had it not set TCP_NODELAY, the first of

Re: 2.6.15 localhost performance hit

2006-03-16 Thread Skunk Worx
Rick Jones wrote: From strace : 15:27:04.568800 recv(3, ?xml version=\1.0\ encoding=\UT..., 555, ) = 555 0.000121 vs. 15:18:24.515891 recv(3, ?xml version=\1.0\ encoding=\UT..., 566, ) = 566 0.038414 Will watch replies and post more when I know more. Kinda new at this. Do you have

Re: 2.6.15 localhost performance hit

2006-03-16 Thread Stephen Hemminger
On Thu, 16 Mar 2006 19:29:32 -0800 Skunk Worx [EMAIL PROTECTED] wrote: Rick Jones wrote: From strace : 15:27:04.568800 recv(3, ?xml version=\1.0\ encoding=\UT..., 555, ) = 555 0.000121 vs. 15:18:24.515891 recv(3, ?xml version=\1.0\ encoding=\UT..., 566, ) = 566 0.038414

Re: 2.6.15 localhost performance hit

2006-03-15 Thread Stephen Hemminger
On Wed, 15 Mar 2006 20:13:01 -0800 Skunk Worx [EMAIL PROTECTED] wrote: Hello, I've taken a performance hit over localhost between kernels 2.6.14 and 2.6.15 in my client/server application. I'm trying to gut things down to a simple test case, in the meantime, this is what I've been

Re: 2.6.15 localhost performance hit

2006-03-15 Thread Stephen Hemminger
On Wed, 15 Mar 2006 20:13:01 -0800 Skunk Worx [EMAIL PROTECTED] wrote: Hello, I've taken a performance hit over localhost between kernels 2.6.14 and 2.6.15 in my client/server application. I'm trying to gut things down to a simple test case, in the meantime, this is what I've been