Re: kqueue microbenchmark results

2000-10-27 Thread Dan Kegel
Terry Lambert wrote: > > > > Which is precisely why you need to know where in the chain of events this > > > happened. Otherwise if I see > > > 'read on fd 5' > > > 'read on fd 5' > > > How do I know which read is for which fd in the multithreaded case > > > > That can't happen,

Re: kqueue microbenchmark results

2000-10-27 Thread Terry Lambert
> > Which is precisely why you need to know where in the chain of events this > > happened. Otherwise if I see > > > > 'read on fd 5' > > 'read on fd 5' > > > > How do I know which read is for which fd in the multithreaded case > > That can't happen, can it? Let's say the

Re: kqueue microbenchmark results

2000-10-27 Thread Alfred Perlstein
* Dan Kegel <[EMAIL PROTECTED]> [001027 09:40] wrote: > Alan Cox wrote: > > > > kqueue currently does this; a close() on an fd will remove any pending > > > > events from the queues that they are on which correspond to that fd. > > > > > > the application of a close event. What can I say, "the

Re: kqueue microbenchmark results

2000-10-27 Thread Dan Kegel
Alan Cox wrote: > > > kqueue currently does this; a close() on an fd will remove any pending > > > events from the queues that they are on which correspond to that fd. > > > > the application of a close event. What can I say, "the fd formerly known > > as X" is now gone? It would be incorrect

Re: kqueue microbenchmark results

2000-10-27 Thread Alfred Perlstein
* Jamie Lokier <[EMAIL PROTECTED]> [001027 08:21] wrote: > Alfred Perlstein wrote: > > > If a programmer does not ever wish to block under any circumstances, it's > > > his obligation to communicate this desire to the implementation. Otherwise, > > > the implementation can block if it doesn't

Re: kqueue microbenchmark results

2000-10-27 Thread Jamie Lokier
Alfred Perlstein wrote: > > If a programmer does not ever wish to block under any circumstances, it's > > his obligation to communicate this desire to the implementation. Otherwise, > > the implementation can block if it doesn't have data or an error available > > at the instant 'read' is called,

Re: kqueue microbenchmark results

2000-10-27 Thread Jamie Lokier
Alfred Perlstein wrote: If a programmer does not ever wish to block under any circumstances, it's his obligation to communicate this desire to the implementation. Otherwise, the implementation can block if it doesn't have data or an error available at the instant 'read' is called,

Re: kqueue microbenchmark results

2000-10-27 Thread Alfred Perlstein
* Jamie Lokier [EMAIL PROTECTED] [001027 08:21] wrote: Alfred Perlstein wrote: If a programmer does not ever wish to block under any circumstances, it's his obligation to communicate this desire to the implementation. Otherwise, the implementation can block if it doesn't have data or an

Re: kqueue microbenchmark results

2000-10-27 Thread Dan Kegel
Alan Cox wrote: kqueue currently does this; a close() on an fd will remove any pending events from the queues that they are on which correspond to that fd. the application of a close event. What can I say, "the fd formerly known as X" is now gone? It would be incorrect to say that

Re: kqueue microbenchmark results

2000-10-27 Thread Alfred Perlstein
* Dan Kegel [EMAIL PROTECTED] [001027 09:40] wrote: Alan Cox wrote: kqueue currently does this; a close() on an fd will remove any pending events from the queues that they are on which correspond to that fd. the application of a close event. What can I say, "the fd formerly known

Re: kqueue microbenchmark results

2000-10-27 Thread Terry Lambert
Which is precisely why you need to know where in the chain of events this happened. Otherwise if I see 'read on fd 5' 'read on fd 5' How do I know which read is for which fd in the multithreaded case That can't happen, can it? Let's say the following happens:

Re: kqueue microbenchmark results

2000-10-27 Thread Dan Kegel
Terry Lambert wrote: Which is precisely why you need to know where in the chain of events this happened. Otherwise if I see 'read on fd 5' 'read on fd 5' How do I know which read is for which fd in the multithreaded case That can't happen, can it? Let's say

Re: kqueue microbenchmark results

2000-10-26 Thread Alfred Perlstein
* Alan Cox <[EMAIL PROTECTED]> [001026 18:33] wrote: > > the application of a close event. What can I say, "the fd formerly known > > as X" is now gone? It would be incorrect to say that "fd X was closed", > > since X no longer refers to anything, and the application may have reused > > that fd

Re: kqueue microbenchmark results

2000-10-26 Thread Alan Cox
> the application of a close event. What can I say, "the fd formerly known > as X" is now gone? It would be incorrect to say that "fd X was closed", > since X no longer refers to anything, and the application may have reused > that fd for another file. Which is precisely why you need to know

Re: kqueue microbenchmark results

2000-10-26 Thread Jonathan Lemon
On Fri, Oct 27, 2000 at 01:50:40AM +0100, Alan Cox wrote: > > kqueue currently does this; a close() on an fd will remove any pending > > events from the queues that they are on which correspond to that fd. > > This seems an odd thing to do. Surely what you need to do is to post a > 'close

Re: kqueue microbenchmark results

2000-10-26 Thread Alfred Perlstein
* Alan Cox <[EMAIL PROTECTED]> [001026 17:50] wrote: > > kqueue currently does this; a close() on an fd will remove any pending > > events from the queues that they are on which correspond to that fd. > > This seems an odd thing to do. Surely what you need to do is to post a > 'close completed'

Re: kqueue microbenchmark results

2000-10-26 Thread Alan Cox
> kqueue currently does this; a close() on an fd will remove any pending > events from the queues that they are on which correspond to that fd. This seems an odd thing to do. Surely what you need to do is to post a 'close completed' event to the queue. This also makes more sense when you have a

Re: kqueue microbenchmark results

2000-10-26 Thread Terry Lambert
This is a long posting, with a humble beginning, but it has a point. I'm being complete so that no one is left in the dark, or in any doubt as to what that point is. That means rehashing some history. This posting is not really about select or Linux: it's about interfaces. Like cached state,

Re: kqueue microbenchmark results

2000-10-26 Thread Jonathan Lemon
On Thu, Oct 26, 2000 at 02:16:28AM -0700, Gideon Glass wrote: > Jonathan Lemon wrote: > > > > Also, consider the following scenario for the proposed get_event(): > > > >1. packet arrives, queues an event. > >2. user retrieves event. > >3. second packet arrives, queues event again. >

Re: kqueue microbenchmark results

2000-10-26 Thread Gideon Glass
Jonathan Lemon wrote: > > Also, consider the following scenario for the proposed get_event(): > >1. packet arrives, queues an event. >2. user retrieves event. >3. second packet arrives, queues event again. >4. user reads() all data. > > Now, next time around the loop, we get a

RE: kqueue microbenchmark results

2000-10-26 Thread David Schwartz
> * David Schwartz <[EMAIL PROTECTED]> [001025 15:35] wrote: > > > > If a programmer does not ever wish to block under any > circumstances, it's > > his obligation to communicate this desire to the > implementation. Otherwise, > > the implementation can block if it doesn't have data or an > error

Re: kqueue microbenchmark results

2000-10-26 Thread Terry Lambert
[ ... blocking read after signalling that data is available ... ] > Yes, and as you mentioned, it was _bugs_ in the operating system > that did this. I think it's reasonable for the OS to discard, for example, connection requests which are not serviced in a reasonable time window. Likewise,

Re: kqueue microbenchmark results

2000-10-26 Thread Terry Lambert
[ ... blocking read after signalling that data is available ... ] Yes, and as you mentioned, it was _bugs_ in the operating system that did this. I think it's reasonable for the OS to discard, for example, connection requests which are not serviced in a reasonable time window. Likewise, it's

RE: kqueue microbenchmark results

2000-10-26 Thread David Schwartz
* David Schwartz [EMAIL PROTECTED] [001025 15:35] wrote: If a programmer does not ever wish to block under any circumstances, it's his obligation to communicate this desire to the implementation. Otherwise, the implementation can block if it doesn't have data or an error available

Re: kqueue microbenchmark results

2000-10-26 Thread Jonathan Lemon
On Thu, Oct 26, 2000 at 02:16:28AM -0700, Gideon Glass wrote: Jonathan Lemon wrote: Also, consider the following scenario for the proposed get_event(): 1. packet arrives, queues an event. 2. user retrieves event. 3. second packet arrives, queues event again. 4. user

Re: kqueue microbenchmark results

2000-10-26 Thread Terry Lambert
This is a long posting, with a humble beginning, but it has a point. I'm being complete so that no one is left in the dark, or in any doubt as to what that point is. That means rehashing some history. This posting is not really about select or Linux: it's about interfaces. Like cached state,

Re: kqueue microbenchmark results

2000-10-26 Thread Alan Cox
kqueue currently does this; a close() on an fd will remove any pending events from the queues that they are on which correspond to that fd. This seems an odd thing to do. Surely what you need to do is to post a 'close completed' event to the queue. This also makes more sense when you have a

Re: kqueue microbenchmark results

2000-10-26 Thread Alfred Perlstein
* Alan Cox [EMAIL PROTECTED] [001026 17:50] wrote: kqueue currently does this; a close() on an fd will remove any pending events from the queues that they are on which correspond to that fd. This seems an odd thing to do. Surely what you need to do is to post a 'close completed' event to

Re: kqueue microbenchmark results

2000-10-26 Thread Jonathan Lemon
On Fri, Oct 27, 2000 at 01:50:40AM +0100, Alan Cox wrote: kqueue currently does this; a close() on an fd will remove any pending events from the queues that they are on which correspond to that fd. This seems an odd thing to do. Surely what you need to do is to post a 'close completed'

Re: kqueue microbenchmark results

2000-10-26 Thread Alan Cox
the application of a close event. What can I say, "the fd formerly known as X" is now gone? It would be incorrect to say that "fd X was closed", since X no longer refers to anything, and the application may have reused that fd for another file. Which is precisely why you need to know where

Re: kqueue microbenchmark results

2000-10-26 Thread Alfred Perlstein
* Alan Cox [EMAIL PROTECTED] [001026 18:33] wrote: the application of a close event. What can I say, "the fd formerly known as X" is now gone? It would be incorrect to say that "fd X was closed", since X no longer refers to anything, and the application may have reused that fd for

Re: kqueue microbenchmark results

2000-10-25 Thread Alfred Perlstein
* David Schwartz <[EMAIL PROTECTED]> [001025 15:35] wrote: > > If a programmer does not ever wish to block under any circumstances, it's > his obligation to communicate this desire to the implementation. Otherwise, > the implementation can block if it doesn't have data or an error available > at

Re: kqueue microbenchmark results

2000-10-25 Thread James Lewis Nance
On Wed, Oct 25, 2000 at 11:27:09AM -0400, Simon Kirby wrote: > On Wed, Oct 25, 2000 at 01:02:46AM -0500, Jonathan Lemon wrote: > > > ends up making the job of the application harder. A simple example > > to illustrate the point: what if the application does not choose > > to read all the data

RE: kqueue microbenchmark results

2000-10-25 Thread David Schwartz
> On Wed, Oct 25, 2000 at 03:11:37PM -0700, David Schwartz wrote: > > > > > Now, next time around the loop, we get a notification for an event > > > when there is no data to read. The application now must be prepared > > > to handle this case (meaning no blocking read() calls can be used). > >

Re: kqueue microbenchmark results

2000-10-25 Thread Jonathan Lemon
On Wed, Oct 25, 2000 at 03:11:37PM -0700, David Schwartz wrote: > > > Now, next time around the loop, we get a notification for an event > > when there is no data to read. The application now must be prepared > > to handle this case (meaning no blocking read() calls can be used). > > -- > >

RE: kqueue microbenchmark results

2000-10-25 Thread David Schwartz
> Now, next time around the loop, we get a notification for an event > when there is no data to read. The application now must be prepared > to handle this case (meaning no blocking read() calls can be used). > -- > Jonathan If the programmer never wants to block in a read call, he

Re: kqueue microbenchmark results

2000-10-25 Thread Jonathan Lemon
On Wed, Oct 25, 2000 at 11:40:28AM -0700, Simon Kirby wrote: > On Wed, Oct 25, 2000 at 12:23:07PM -0500, Jonathan Lemon wrote: > > > Consider a program which reads from point A, writes to point B. If > > the buffer associated with B fills up, then we don't want to continue > > reading from A. >

Re: kqueue microbenchmark results

2000-10-25 Thread Jonathan Lemon
On Wed, Oct 25, 2000 at 09:40:53PM +0200, Jamie Lokier wrote: > Simon Kirby wrote: > > And you'd need to take the descriptor out of the read() set in the > > select() case anyway, so I don't really see what's different. > > The difference is that taking a bit out of select()'s bitmap is >

Re: kqueue microbenchmark results

2000-10-25 Thread Terry Lambert
> What applications would do better by postponing some of the reading? > I can't think of any reason off the top of my head why an application > wouldn't want to read everything it can. Doing everything in smaller > chunks would increase overhead (but maybe reduce latencies very slightly > --

Re: kqueue microbenchmark results

2000-10-25 Thread Jamie Lokier
Simon Kirby wrote: > And you'd need to take the descriptor out of the read() set in the > select() case anyway, so I don't really see what's different. The difference is that taking a bit out of select()'s bitmap is basically free. Whereas the equivalent with events is a bind_event() system

Re: kqueue microbenchmark results

2000-10-25 Thread Simon Kirby
On Wed, Oct 25, 2000 at 12:23:07PM -0500, Jonathan Lemon wrote: > Consider a program which reads from point A, writes to point B. If > the buffer associated with B fills up, then we don't want to continue > reading from A. > > A/B may be network sockets, pipes, or ptys. Fine, but we

Re: kqueue microbenchmark results

2000-10-25 Thread Simon Kirby
On Wed, Oct 25, 2000 at 07:08:48PM +0200, Jamie Lokier wrote: > Simon Kirby wrote: > > > What applications would do better by postponing some of the reading? > > I can't think of any reason off the top of my head why an application > > wouldn't want to read everything it can. > > Pipelined

Re: kqueue microbenchmark results

2000-10-25 Thread Jonathan Lemon
On Wed, Oct 25, 2000 at 11:27:09AM -0400, Simon Kirby wrote: > On Wed, Oct 25, 2000 at 01:02:46AM -0500, Jonathan Lemon wrote: > > > Yes, someone pointed me to those today. I would suggest reading > > some of the relevant literature before embarking on a design. My > > paper discusses some of

Re: kqueue microbenchmark results

2000-10-25 Thread Jamie Lokier
Simon Kirby wrote: > > While an 'edge-trigger' design is indeed simpler, I feel that it > > ends up making the job of the application harder. A simple example > > to illustrate the point: what if the application does not choose > > to read all the data from an incoming packet? The app now has

Re: kqueue microbenchmark results

2000-10-25 Thread Simon Kirby
On Wed, Oct 25, 2000 at 01:02:46AM -0500, Jonathan Lemon wrote: > Yes, someone pointed me to those today. I would suggest reading > some of the relevant literature before embarking on a design. My > paper discusses some of the issues, and Mogul/Banga make some good > points too. > > While an

Re: kqueue microbenchmark results

2000-10-25 Thread Simon Kirby
On Wed, Oct 25, 2000 at 01:02:46AM -0500, Jonathan Lemon wrote: Yes, someone pointed me to those today. I would suggest reading some of the relevant literature before embarking on a design. My paper discusses some of the issues, and Mogul/Banga make some good points too. While an

Re: kqueue microbenchmark results

2000-10-25 Thread Jamie Lokier
Simon Kirby wrote: While an 'edge-trigger' design is indeed simpler, I feel that it ends up making the job of the application harder. A simple example to illustrate the point: what if the application does not choose to read all the data from an incoming packet? The app now has to

Re: kqueue microbenchmark results

2000-10-25 Thread Jonathan Lemon
On Wed, Oct 25, 2000 at 11:27:09AM -0400, Simon Kirby wrote: On Wed, Oct 25, 2000 at 01:02:46AM -0500, Jonathan Lemon wrote: Yes, someone pointed me to those today. I would suggest reading some of the relevant literature before embarking on a design. My paper discusses some of the

Re: kqueue microbenchmark results

2000-10-25 Thread Simon Kirby
On Wed, Oct 25, 2000 at 07:08:48PM +0200, Jamie Lokier wrote: Simon Kirby wrote: What applications would do better by postponing some of the reading? I can't think of any reason off the top of my head why an application wouldn't want to read everything it can. Pipelined server.

Re: kqueue microbenchmark results

2000-10-25 Thread Simon Kirby
On Wed, Oct 25, 2000 at 12:23:07PM -0500, Jonathan Lemon wrote: Consider a program which reads from point A, writes to point B. If the buffer associated with B fills up, then we don't want to continue reading from A. A/B may be network sockets, pipes, or ptys. Fine, but we can

Re: kqueue microbenchmark results

2000-10-25 Thread Jamie Lokier
Simon Kirby wrote: And you'd need to take the descriptor out of the read() set in the select() case anyway, so I don't really see what's different. The difference is that taking a bit out of select()'s bitmap is basically free. Whereas the equivalent with events is a bind_event() system call.

Re: kqueue microbenchmark results

2000-10-25 Thread Terry Lambert
What applications would do better by postponing some of the reading? I can't think of any reason off the top of my head why an application wouldn't want to read everything it can. Doing everything in smaller chunks would increase overhead (but maybe reduce latencies very slightly -- albeit

Re: kqueue microbenchmark results

2000-10-25 Thread Jonathan Lemon
On Wed, Oct 25, 2000 at 09:40:53PM +0200, Jamie Lokier wrote: Simon Kirby wrote: And you'd need to take the descriptor out of the read() set in the select() case anyway, so I don't really see what's different. The difference is that taking a bit out of select()'s bitmap is basically

Re: kqueue microbenchmark results

2000-10-25 Thread Jonathan Lemon
On Wed, Oct 25, 2000 at 11:40:28AM -0700, Simon Kirby wrote: On Wed, Oct 25, 2000 at 12:23:07PM -0500, Jonathan Lemon wrote: Consider a program which reads from point A, writes to point B. If the buffer associated with B fills up, then we don't want to continue reading from A. A/B

RE: kqueue microbenchmark results

2000-10-25 Thread David Schwartz
Now, next time around the loop, we get a notification for an event when there is no data to read. The application now must be prepared to handle this case (meaning no blocking read() calls can be used). -- Jonathan If the programmer never wants to block in a read call, he should

RE: kqueue microbenchmark results

2000-10-25 Thread David Schwartz
On Wed, Oct 25, 2000 at 03:11:37PM -0700, David Schwartz wrote: Now, next time around the loop, we get a notification for an event when there is no data to read. The application now must be prepared to handle this case (meaning no blocking read() calls can be used). --

Re: kqueue microbenchmark results

2000-10-25 Thread James Lewis Nance
On Wed, Oct 25, 2000 at 11:27:09AM -0400, Simon Kirby wrote: On Wed, Oct 25, 2000 at 01:02:46AM -0500, Jonathan Lemon wrote: ends up making the job of the application harder. A simple example to illustrate the point: what if the application does not choose to read all the data from an

Re: kqueue microbenchmark results

2000-10-25 Thread Alfred Perlstein
* David Schwartz [EMAIL PROTECTED] [001025 15:35] wrote: If a programmer does not ever wish to block under any circumstances, it's his obligation to communicate this desire to the implementation. Otherwise, the implementation can block if it doesn't have data or an error available at the

Re: kqueue microbenchmark results

2000-10-24 Thread Jonathan Lemon
On Tue, Oct 24, 2000 at 09:45:14PM -0700, Dan Kegel wrote: > If you haven't already, you might peek at the discussion on > linux-kernel. Linus seems to be on the verge of adding > something like kqueue() to Linux, but appears opposed to > supporting level-triggering; he likes the simplicity of >

Re: kqueue microbenchmark results

2000-10-24 Thread Dan Kegel
Johnathan, Thanks for running that test for me! I've added your results (plus a cautionary note about microbenchmarks and a link to your site) to http://www.kegel.com/dkftpbench/Poller_bench.html If you haven't already, you might peek at the discussion on linux-kernel. Linus seems to be on the

Re: kqueue microbenchmark results

2000-10-24 Thread Dan Kegel
Johnathan, Thanks for running that test for me! I've added your results (plus a cautionary note about microbenchmarks and a link to your site) to http://www.kegel.com/dkftpbench/Poller_bench.html If you haven't already, you might peek at the discussion on linux-kernel. Linus seems to be on the

Re: kqueue microbenchmark results

2000-10-24 Thread Jonathan Lemon
On Tue, Oct 24, 2000 at 09:45:14PM -0700, Dan Kegel wrote: If you haven't already, you might peek at the discussion on linux-kernel. Linus seems to be on the verge of adding something like kqueue() to Linux, but appears opposed to supporting level-triggering; he likes the simplicity of edge