Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-11 Thread James Antill
"Stephen D. Williams" <[EMAIL PROTECTED]> writes: > James Antill wrote: > ... > > >The > > > time went from 3.7 to 4.4 seconds per 10. > > > > Ok here's a quick test that I've done. This passes data between 2 > >

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-11 Thread Stephen D. Williams
James Antill wrote: ... > >The > > time went from 3.7 to 4.4 seconds per 10. > > Ok here's a quick test that I've done. This passes data between 2 > processes. Obviously you can't compare this to your code or Michael's, >

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-11 Thread James Antill
"Stephen D. Williams" [EMAIL PROTECTED] writes: James Antill wrote: ... The time went from 3.7 to 4.4 seconds per 10. Ok here's a quick test that I've done. This passes data between 2 processes. Obviously you

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-10 Thread James Antill
"Stephen D. Williams" <[EMAIL PROTECTED]> writes: > James Antill wrote: > > > > I seemed to miss the original post, so I can't really comment on the > > tests. However... > > It was a thread in January, but just ran accross it looking for > something else. See below for results. Ahh, ok.

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-10 Thread Stephen D. Williams
James Antill wrote: > > "Stephen D. Williams" <[EMAIL PROTECTED]> writes: > > > An old thread, but important to get these fundamental performance > > numbers up there: > > > > 2.4.2 on an 800mhz PIII Sceptre laptop w/ 512MB ram: > > > > elapsed time for 10 pingpongs is > > 3.81327 > >

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-10 Thread Stephen D. Williams
James Antill wrote: "Stephen D. Williams" [EMAIL PROTECTED] writes: An old thread, but important to get these fundamental performance numbers up there: 2.4.2 on an 800mhz PIII Sceptre laptop w/ 512MB ram: elapsed time for 10 pingpongs is 3.81327 10/3.81256

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-10 Thread James Antill
"Stephen D. Williams" [EMAIL PROTECTED] writes: James Antill wrote: I seemed to miss the original post, so I can't really comment on the tests. However... It was a thread in January, but just ran accross it looking for something else. See below for results. Ahh, ok. Michael

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-09 Thread James Antill
"Stephen D. Williams" <[EMAIL PROTECTED]> writes: > An old thread, but important to get these fundamental performance > numbers up there: > > 2.4.2 on an 800mhz PIII Sceptre laptop w/ 512MB ram: > > elapsed time for 10 pingpongs is > 3.81327 >

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-09 Thread Stephen D. Williams
An old thread, but important to get these fundamental performance numbers up there: 2.4.2 on an 800mhz PIII Sceptre laptop w/ 512MB ram: elapsed time for 10 pingpongs is 3.81327 10/3.81256 ~26229.09541095746689888159 1/.379912

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-09 Thread Stephen D. Williams
An old thread, but important to get these fundamental performance numbers up there: 2.4.2 on an 800mhz PIII Sceptre laptop w/ 512MB ram: elapsed time for 10 pingpongs is 3.81327 10/3.81256 ~26229.09541095746689888159 1/.379912

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-04-09 Thread James Antill
"Stephen D. Williams" [EMAIL PROTECTED] writes: An old thread, but important to get these fundamental performance numbers up there: 2.4.2 on an 800mhz PIII Sceptre laptop w/ 512MB ram: elapsed time for 10 pingpongs is 3.81327 10/3.81256

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-24 Thread Bernd Eckenfels
> can someone explain what is nagle or pinpoint explanation :) nagel's algorithm is used to "wait" with sending of small packets until more data is available, because sending biger packets has less overhead. greetings Bernd - To unsubscribe from this list: send the line "unsubscribe

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-24 Thread Boris Dragovic
can someone explain what is nagle or pinpoint explanation :) lynx On Sun, 21 Jan 2001, Chris Wedgwood wrote: > On Sat, Jan 20, 2001 at 07:35:12PM -0500, Dan Maas wrote: > > Bingo! With this fix, 2.2.18 performance becomes almost identical to 2.4.0 > performance. I assume 2.4.0

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-24 Thread Boris Dragovic
can someone explain what is nagle or pinpoint explanation :) lynx On Sun, 21 Jan 2001, Chris Wedgwood wrote: On Sat, Jan 20, 2001 at 07:35:12PM -0500, Dan Maas wrote: Bingo! With this fix, 2.2.18 performance becomes almost identical to 2.4.0 performance. I assume 2.4.0

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-24 Thread Bernd Eckenfels
can someone explain what is nagle or pinpoint explanation :) nagel's algorithm is used to "wait" with sending of small packets until more data is available, because sending biger packets has less overhead. greetings Bernd - To unsubscribe from this list: send the line "unsubscribe

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Michael Lindner
OK, 2.4.0 kernel installed, and a new set of numbers: testkernel ping-pongs/s. @ total CPU util w/SOL_NDELAY sample (2 skts) 2.2.18 100 @ 0.1% 800 @ 1% sample (1 skt) 2.2.18 8000 @ 100% 8000 @ 50% real app

RE: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread David Schwartz
> ...and I still don't understand why the identical program, but using one > socket instead of 2 sockets, IS CPU bound, and gets on the order of > 10K/sec. on the same HW. Diffs to produce 10K/sec. 1 socket version from > my previous sample follow... It's really this simple -- this

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Michael Lindner
Chris Wedgwood wrote: > > On Sat, Jan 20, 2001 at 07:35:12PM -0500, Dan Maas wrote: > > Bingo! With this fix, 2.2.18 performance becomes almost identical to 2.4.0 > performance. I assume 2.4.0 disables Nagle by default on local > connections... > > 2.4.x has a smarter nagle

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Dan Maas
> It's not the select that waits. It's a delay in the tcp send > path waiting for more data. Try disabling it: > > int f=1; > setsockopt(s, SOL_TCP, TCP_NODELAY, , sizeof(f)); Bingo! With this fix, 2.2.18 performance becomes almost identical to 2.4.0 performance. I assume 2.4.0 disables Nagle

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Edgar Toernig
Michael Lindner wrote: >[...] > send(s, ".", 1, 0); >[...] > while (select(r+1, , 0, 0, 0) > 0) { >[...] >[select returns only after about 1 HZ] Ever heard of nagle? (If not, there's a long thread about it on the mailing list *g*) It's not the select that waits. It's a

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Michael Lindner
Dan Maas wrote: > > What kernel have you been using? I have reproduced your problem on a > standard 2.2.18 kernel (elapsed time ~10sec). However, using a 2.4.0 kernel > with HZ=1000, I see a 100x improvement (elapsed time ~0.1 sec; note that > increasing HZ alone should only give a 10x

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Bjorn Wesen
On Sat, 20 Jan 2001, Martin MaD Douda wrote: > On Fri, 19 Jan 2001, Michael Lindner wrote: > > data is generated as a result of data received via a select(), > > the next delivery occurs a clock tick later, with the machine > > mostly idle. > > The machine is in fact not idle - there is a task

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Martin MaD Douda
On Fri, 19 Jan 2001, Michael Lindner wrote: > data is generated as a result of data received via a select(), > the next delivery occurs a clock tick later, with the machine > mostly idle. ^^^ The machine is in fact not idle - there is a task running - idle task. Could the problem be

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Bernd Eckenfels
In article <[EMAIL PROTECTED]> you wrote: > My problem is that if data is NOT available when select() > starts, but becomes available immediately afterwards, select() > doesn't wake up immediately, but sleeps for 1/100 second. It does not sleep for a 1/100second, it will but the process in the

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Dan Maas
What kernel have you been using? I have reproduced your problem on a standard 2.2.18 kernel (elapsed time ~10sec). However, using a 2.4.0 kernel with HZ=1000, I see a 100x improvement (elapsed time ~0.1 sec; note that increasing HZ alone should only give a 10x improvement). Perhaps the scheduler

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Dan Maas
What kernel have you been using? I have reproduced your problem on a standard 2.2.18 kernel (elapsed time ~10sec). However, using a 2.4.0 kernel with HZ=1000, I see a 100x improvement (elapsed time ~0.1 sec; note that increasing HZ alone should only give a 10x improvement). Perhaps the scheduler

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Bernd Eckenfels
In article [EMAIL PROTECTED] you wrote: My problem is that if data is NOT available when select() starts, but becomes available immediately afterwards, select() doesn't wake up immediately, but sleeps for 1/100 second. It does not sleep for a 1/100second, it will but the process in the run

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Martin MaD Douda
On Fri, 19 Jan 2001, Michael Lindner wrote: data is generated as a result of data received via a select(), the next delivery occurs a clock tick later, with the machine mostly idle. ^^^ The machine is in fact not idle - there is a task running - idle task. Could the problem be that

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Bjorn Wesen
On Sat, 20 Jan 2001, Martin MaD Douda wrote: On Fri, 19 Jan 2001, Michael Lindner wrote: data is generated as a result of data received via a select(), the next delivery occurs a clock tick later, with the machine mostly idle. The machine is in fact not idle - there is a task running -

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Michael Lindner
Dan Maas wrote: What kernel have you been using? I have reproduced your problem on a standard 2.2.18 kernel (elapsed time ~10sec). However, using a 2.4.0 kernel with HZ=1000, I see a 100x improvement (elapsed time ~0.1 sec; note that increasing HZ alone should only give a 10x improvement).

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Edgar Toernig
Michael Lindner wrote: [...] send(s, ".", 1, 0); [...] while (select(r+1, readfds, 0, 0, 0) 0) { [...] [select returns only after about 1 HZ] Ever heard of nagle? (If not, there's a long thread about it on the mailing list *g*) It's not the select that waits. It's a

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Dan Maas
It's not the select that waits. It's a delay in the tcp send path waiting for more data. Try disabling it: int f=1; setsockopt(s, SOL_TCP, TCP_NODELAY, f, sizeof(f)); Bingo! With this fix, 2.2.18 performance becomes almost identical to 2.4.0 performance. I assume 2.4.0 disables Nagle by

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Michael Lindner
Chris Wedgwood wrote: On Sat, Jan 20, 2001 at 07:35:12PM -0500, Dan Maas wrote: Bingo! With this fix, 2.2.18 performance becomes almost identical to 2.4.0 performance. I assume 2.4.0 disables Nagle by default on local connections... 2.4.x has a smarter nagle algorithm.

RE: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread David Schwartz
...and I still don't understand why the identical program, but using one socket instead of 2 sockets, IS CPU bound, and gets on the order of 10K/sec. on the same HW. Diffs to produce 10K/sec. 1 socket version from my previous sample follow... It's really this simple -- this isn't

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-20 Thread Michael Lindner
OK, 2.4.0 kernel installed, and a new set of numbers: testkernel ping-pongs/s. @ total CPU util w/SOL_NDELAY sample (2 skts) 2.2.18 100 @ 0.1% 800 @ 1% sample (1 skt) 2.2.18 8000 @ 100% 8000 @ 50% real app

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
Chris Wedgwood wrote: > > You can measure this latency; and it's indeed very low (lmbench gives > 28 usecs on one of my machines). > > If you don't see this I would suspect an application bug -- can you > use strace or some such and confirm this is not the case? OK, two new data points (thanks

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
Dan Maas wrote: > > > OK, if this is the case, how do I alter the scheduling class? > > man sched_setscheduler > > Set SCHED_FIFO or SCHED_RR; you'll need to be root to do this AFAIK. > > I do agree though, Linux's scheduler (for SCHED_OTHER processes) is much > less "ruthless" than, say, the

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
David Schwartz wrote: > > How can you tell when select wakes up the process? What you are seeing has > nothing whatsoever to do with select and simply has to do with the fact that > the kernel does not give the CPU to a process the second that process may > want it. I guess I can't. But

RE: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread David Schwartz
> Thanks CW and DS for the prompt replies. However, although each > addressed the (flawed) example I included, neither addressed the > problem described in the text. > > I wrote: > > > If select() is waiting for data to become available on a > > > TCP socket FD, and > > > data becomes available,

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
Thanks CW and DS for the prompt replies. However, although each addressed the (flawed) example I included, neither addressed the problem described in the text. I wrote: > > If select() is waiting for data to become available on a > > TCP socket FD, and > > data becomes available, it doesn't

RE: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread David Schwartz
> If select() is waiting for data to become available on a > TCP socket FD, and > data becomes available, it doesn't return until the next clock tick. If your application has scheduling requirements, you need to communicate them to the scheduler. > #include > #include >

PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
[1.] select() sleeps for 1 tick even if data available [2.] Full description of the problem/report: If select() is waiting for data to become available on a TCP socket FD, and data becomes available, it doesn't return until the next clock tick. This produces large

PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
[1.] select() sleeps for 1 tick even if data available [2.] Full description of the problem/report: If select() is waiting for data to become available on a TCP socket FD, and data becomes available, it doesn't return until the next clock tick. This produces large

RE: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread David Schwartz
If select() is waiting for data to become available on a TCP socket FD, and data becomes available, it doesn't return until the next clock tick. If your application has scheduling requirements, you need to communicate them to the scheduler. #include sys/time.h #include

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
Thanks CW and DS for the prompt replies. However, although each addressed the (flawed) example I included, neither addressed the problem described in the text. I wrote: If select() is waiting for data to become available on a TCP socket FD, and data becomes available, it doesn't return

RE: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread David Schwartz
Thanks CW and DS for the prompt replies. However, although each addressed the (flawed) example I included, neither addressed the problem described in the text. I wrote: If select() is waiting for data to become available on a TCP socket FD, and data becomes available, it doesn't

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
David Schwartz wrote: How can you tell when select wakes up the process? What you are seeing has nothing whatsoever to do with select and simply has to do with the fact that the kernel does not give the CPU to a process the second that process may want it. I guess I can't. But on

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
Dan Maas wrote: OK, if this is the case, how do I alter the scheduling class? man sched_setscheduler Set SCHED_FIFO or SCHED_RR; you'll need to be root to do this AFAIK. I do agree though, Linux's scheduler (for SCHED_OTHER processes) is much less "ruthless" than, say, the NT

Re: PROBLEM: select() on TCP socket sleeps for 1 tick even if data available

2001-01-19 Thread Michael Lindner
Chris Wedgwood wrote: You can measure this latency; and it's indeed very low (lmbench gives 28 usecs on one of my machines). If you don't see this I would suspect an application bug -- can you use strace or some such and confirm this is not the case? OK, two new data points (thanks for