Re: [patch 05/11] syslets: core code

2007-02-18 Thread Michael K. Edwards
On 2/18/07, Davide Libenzi wrote: Clets would execute in userspace, like signal handlers, or like "event handlers" in cooperative multitasking environments without the Unix baggage but under the special schedule() handler. or, better yet, as the next tasklet in the chain after the softirq

Re: [patch 05/11] syslets: core code

2007-02-18 Thread Davide Libenzi
On Sun, 18 Feb 2007, Pavel Machek wrote: > > > The upcall will setup a frame, execute the clet (where jump/conditions > > > and > > > userspace variable changes happen in machine code - gcc is pretty good in > > > taking care of that for us) on its return, come back through a > > >

Re: [patch 05/11] syslets: core code

2007-02-18 Thread Pavel Machek
Hi! > > The upcall will setup a frame, execute the clet (where jump/conditions and > > userspace variable changes happen in machine code - gcc is pretty good in > > taking care of that for us) on its return, come back through a > > sys_async_return, and go back to userspace. > > So, for

Re: [patch 05/11] syslets: core code

2007-02-18 Thread Pavel Machek
Hi! The upcall will setup a frame, execute the clet (where jump/conditions and userspace variable changes happen in machine code - gcc is pretty good in taking care of that for us) on its return, come back through a sys_async_return, and go back to userspace. So, for example, this

Re: [patch 05/11] syslets: core code

2007-02-18 Thread Davide Libenzi
On Sun, 18 Feb 2007, Pavel Machek wrote: The upcall will setup a frame, execute the clet (where jump/conditions and userspace variable changes happen in machine code - gcc is pretty good in taking care of that for us) on its return, come back through a sys_async_return, and go

Re: [patch 05/11] syslets: core code

2007-02-18 Thread Michael K. Edwards
On 2/18/07, Davide Libenzi davidel@xmailserver.org wrote: Clets would execute in userspace, like signal handlers, or like event handlers in cooperative multitasking environments without the Unix baggage but under the special schedule() handler. or, better yet, as the next tasklet in the

Re: [patch 05/11] syslets: core code

2007-02-17 Thread Cyrill V. Gorcunov
On Sat, Feb 17, 2007 at 01:02:00PM +0300, Evgeniy Polyakov wrote: [... snipped ...] | Yes, I only proposed to change what Ingo has right now - although it is | usable, but it does suck, but since overall syslet design is indeed good | it does not suffer from possible interface changes - so I said

Re: [patch 05/11] syslets: core code

2007-02-17 Thread Al Boldi
Evgeniy Polyakov wrote: > Ray Lee ([EMAIL PROTECTED]) wrote: > > The truth of this lies somewhere in the middle. It isn't kernel driven, > > or userspace interface driven, but a tradeoff between the two. > > > > So: > > > Userspace_API_is_the_ever_possible_last_thing_to_ever_think_about. > > >

Re: [patch 05/11] syslets: core code

2007-02-17 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 08:54:11PM -0800, Ray Lee ([EMAIL PROTECTED]) wrote: > (This is probably why, by the way, most people are staying silent on > your excellent kevent work. The kernel side is, in some ways, the easy > part. It's getting an interface that will handle all users [ users == >

Re: [patch 05/11] syslets: core code

2007-02-17 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 11:20:36PM +0300, Cyrill V. Gorcunov ([EMAIL PROTECTED]) wrote: > On Fri, Feb 16, 2007 at 07:58:54PM +0300, Evgeniy Polyakov wrote: > | Absolutely. > | And if overall system design is good, there is no problem to change > | (well, for those who fail to read to the end and

Re: [patch 05/11] syslets: core code

2007-02-17 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 11:20:36PM +0300, Cyrill V. Gorcunov ([EMAIL PROTECTED]) wrote: On Fri, Feb 16, 2007 at 07:58:54PM +0300, Evgeniy Polyakov wrote: | Absolutely. | And if overall system design is good, there is no problem to change | (well, for those who fail to read to the end and

Re: [patch 05/11] syslets: core code

2007-02-17 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 08:54:11PM -0800, Ray Lee ([EMAIL PROTECTED]) wrote: (This is probably why, by the way, most people are staying silent on your excellent kevent work. The kernel side is, in some ways, the easy part. It's getting an interface that will handle all users [ users ==

Re: [patch 05/11] syslets: core code

2007-02-17 Thread Al Boldi
Evgeniy Polyakov wrote: Ray Lee ([EMAIL PROTECTED]) wrote: The truth of this lies somewhere in the middle. It isn't kernel driven, or userspace interface driven, but a tradeoff between the two. So: Userspace_API_is_the_ever_possible_last_thing_to_ever_think_about. Period Please

Re: [patch 05/11] syslets: core code

2007-02-17 Thread Cyrill V. Gorcunov
On Sat, Feb 17, 2007 at 01:02:00PM +0300, Evgeniy Polyakov wrote: [... snipped ...] | Yes, I only proposed to change what Ingo has right now - although it is | usable, but it does suck, but since overall syslet design is indeed good | it does not suffer from possible interface changes - so I said

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Ray Lee
Evgeniy Polyakov wrote: > On Fri, Feb 16, 2007 at 08:53:30AM -0800, Ray Lee ([EMAIL PROTECTED]) wrote: >> On 2/16/07, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: >>> if its design is good, then >>> interface can be changed in a moment without any problem >> This isn't always the case. Sometimes

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Cyrill V. Gorcunov
On Fri, Feb 16, 2007 at 07:58:54PM +0300, Evgeniy Polyakov wrote: | Absolutely. | And if overall system design is good, there is no problem to change | (well, for those who fail to read to the end and understand my english | replace 'to change' with 'to create and commit') interface to the state |

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 08:53:30AM -0800, Ray Lee ([EMAIL PROTECTED]) wrote: > On 2/16/07, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > >if its design is good, then > >interface can be changed in a moment without any problem > > This isn't always the case. Sometimes the interface puts

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Ray Lee
On 2/16/07, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: if its design is good, then interface can be changed in a moment without any problem This isn't always the case. Sometimes the interface puts requirements (contract-like) upon the implementation. Case in point in the kernel, dnotify

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 07:54:22AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: > > Interfaces can be created and destroyed - they do not affect overall > > system design in anyway (well, if they do, something is broken). > > I'm sorry, but you've obviously never maintained any piece of

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Linus Torvalds
On Fri, 16 Feb 2007, Evgeniy Polyakov wrote: > > Interfaces can be created and destroyed - they do not affect overall > system design in anyway (well, if they do, something is broken). I'm sorry, but you've obviously never maintained any piece of software that actually has users. As long as

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 01:28:06PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > OTOH, the syslet concept right now already looks very ubiquitous, and > the main problem with AIO use in applications wasnt just even its broken > API or its broken performance, but the fundamental lack of all

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > On Thu, 15 Feb 2007, Linus Torvalds wrote: > > > > So I think that a good implementation just does everything up-front, > > and doesn't _need_ a user buffer that is live over longer periods, > > except for the actual results. Exactly because the

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 11:28:57AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: > THAT was the point. Interfaces are really really subtle and important. > It's absolutely not a case of "we can just write wrappers to fix up any > library issues". Interfaces can be created and destroyed -

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 11:28:57AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: THAT was the point. Interfaces are really really subtle and important. It's absolutely not a case of we can just write wrappers to fix up any library issues. Interfaces can be created and destroyed - they do

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Ingo Molnar
* Linus Torvalds [EMAIL PROTECTED] wrote: On Thu, 15 Feb 2007, Linus Torvalds wrote: So I think that a good implementation just does everything up-front, and doesn't _need_ a user buffer that is live over longer periods, except for the actual results. Exactly because the whole

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 01:28:06PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: OTOH, the syslet concept right now already looks very ubiquitous, and the main problem with AIO use in applications wasnt just even its broken API or its broken performance, but the fundamental lack of all Linux

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Linus Torvalds
On Fri, 16 Feb 2007, Evgeniy Polyakov wrote: Interfaces can be created and destroyed - they do not affect overall system design in anyway (well, if they do, something is broken). I'm sorry, but you've obviously never maintained any piece of software that actually has users. As long as you

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 07:54:22AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: Interfaces can be created and destroyed - they do not affect overall system design in anyway (well, if they do, something is broken). I'm sorry, but you've obviously never maintained any piece of software

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Ray Lee
On 2/16/07, Evgeniy Polyakov [EMAIL PROTECTED] wrote: if its design is good, then interface can be changed in a moment without any problem This isn't always the case. Sometimes the interface puts requirements (contract-like) upon the implementation. Case in point in the kernel, dnotify versus

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Evgeniy Polyakov
On Fri, Feb 16, 2007 at 08:53:30AM -0800, Ray Lee ([EMAIL PROTECTED]) wrote: On 2/16/07, Evgeniy Polyakov [EMAIL PROTECTED] wrote: if its design is good, then interface can be changed in a moment without any problem This isn't always the case. Sometimes the interface puts requirements

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Cyrill V. Gorcunov
On Fri, Feb 16, 2007 at 07:58:54PM +0300, Evgeniy Polyakov wrote: | Absolutely. | And if overall system design is good, there is no problem to change | (well, for those who fail to read to the end and understand my english | replace 'to change' with 'to create and commit') interface to the state |

Re: [patch 05/11] syslets: core code

2007-02-16 Thread Ray Lee
Evgeniy Polyakov wrote: On Fri, Feb 16, 2007 at 08:53:30AM -0800, Ray Lee ([EMAIL PROTECTED]) wrote: On 2/16/07, Evgeniy Polyakov [EMAIL PROTECTED] wrote: if its design is good, then interface can be changed in a moment without any problem This isn't always the case. Sometimes the interface

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Michael K. Edwards
On 2/15/07, Linus Torvalds <[EMAIL PROTECTED]> wrote: Would it make the interface less cool? Yeah. Would it limit it to just a few linked system calls (to avoid memory allocation issues in the kernel)? Yes again. But it would simplify a lot of the interface issues. Only in toy applications.

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Davide Libenzi
On Thu, 15 Feb 2007, Linus Torvalds wrote: > > > On Thu, 15 Feb 2007, Linus Torvalds wrote: > > > > So I think that a good implementation just does everything up-front, and > > doesn't _need_ a user buffer that is live over longer periods, except for > > the actual results. Exactly because

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Linus Torvalds wrote: > > So I think that a good implementation just does everything up-front, and > doesn't _need_ a user buffer that is live over longer periods, except for > the actual results. Exactly because the whole alloc/teardown is nasty. Btw, this doesn't

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: > > > > See? The example you tried to use to show how "simple" the interface iswas > > actually EXACTLY THE REVERSE. It shows how subtle bugs can creep in! > > So describe what are the requirements (constraints)? > > Above example has exactly one

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Eric Dumazet
On Thursday 15 February 2007 19:46, bert hubert wrote: > Both 1 and 2 are currently limiting factors when I enter the 100kqps domain > of name serving. This doesn't mean the rest of my code is as tight as it > could be, but I spend a significant portion of time in the kernel even at > moderate

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Zach Brown
2) On the client facing side (port 53), I'd very much hope for a way to do 'recvv' on datagram sockets, so I can retrieve a whole bunch of UDP datagrams with only one kernel transition. I want to highlight this point that Bert is making. Whenever we talk about AIO and kernel

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 07:46:56PM +0100, bert hubert ([EMAIL PROTECTED]) wrote: > 1) batch, and wait for, with proper error reporting: > socket(); > [ setsockopt(); ] > bind(); > connect(); > gettimeofday(); // doesn't *always* happen > send(); > recv();

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 10:25:37AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: > > static void syslet_setup(struct syslet *s, int nr, void *arg1...) > > { > > s->flags = ... > > s->arg[1] = arg1; > > > > } > > > > long glibc_async_stat(const char *path, struct stat *buf) >

Re: [patch 05/11] syslets: core code

2007-02-15 Thread bert hubert
On Thu, Feb 15, 2007 at 09:42:32AM -0800, Linus Torvalds wrote: > We know one interface: the current aio_read() one. Nobody really _likes_ [...] > Others? We don't know yet. And exposing complex interfaces that may not be > the right ones is much *worse* than exposing simple interfaces (that

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: > > So we just need to describe the way we want to see new interface - > that's it. Agreed. Absolutely. But please keep the kernel interface as part of that. Not just a strange and complex kernel interface and then _usable_ library interfaces that

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 09:42:32AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: > > > On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: > > > > Userspace_API_is_the_ever_possible_last_thing_to_ever_think_about. Period > > . // <- wrapped one > > No, I really think you're wrong. > > In many

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 09:39:33AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: > On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: > > > On Thu, Feb 15, 2007 at 09:05:13AM -0800, Davide Libenzi > > (davidel@xmailserver.org) wrote: > > > > > > I actually think that building chains of

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: > > Userspace_API_is_the_ever_possible_last_thing_to_ever_think_about. Period > . // <- wrapped one No, I really think you're wrong. In many ways, the interfaces and especially data structures are *more* important than the code. The code we can

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Davide Libenzi
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: > On Thu, Feb 15, 2007 at 09:05:13AM -0800, Davide Libenzi > (davidel@xmailserver.org) wrote: > > > > I actually think that building chains of syscalls bring you back to a > > multithreaded solution. Why? Because suddendly the service thread become

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 09:05:13AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: > On Thu, 15 Feb 2007, Linus Torvalds wrote: > > > I don't think the "atom" approach is bad per se. I think it could be fine > > to have some state information in user space. It's just that I think > >

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Ulrich Drepper
Linus Torvalds wrote: > Here's a quick question: how many people have actually ever seen them used > in "normal code"? > > Yeah. Nobody uses them. They're not all that portable (even within unixes > they aren't always there, much less in other places), they are fairly > obscure, and they are

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Davide Libenzi
On Thu, 15 Feb 2007, Linus Torvalds wrote: > I don't think the "atom" approach is bad per se. I think it could be fine > to have some state information in user space. It's just that I think > complex interfaces that people largely won't even use is a big mistake. We > should concentrate on

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 08:09:54AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: > > > In other words, the "let user space sort out the complexity" is not a > > > good > > > answer. It just means that the interface is badly designed. > > > > Well, if we can setup iocb structure, why we can

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: > > > > In other words, the "let user space sort out the complexity" is not a good > > answer. It just means that the interface is badly designed. > > Well, if we can setup iocb structure, why we can not setup syslet one? (I'm cutting wildly, to

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Wed, Feb 14, 2007 at 12:38:16PM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: > Or how would you do the trivial example loop that I explained was a good > idea: > > struct one_entry *prev = NULL; > struct dirent *de; > > while ((de = readdir(dir)) != NULL) { >

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Wed, Feb 14, 2007 at 12:38:16PM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: Or how would you do the trivial example loop that I explained was a good idea: struct one_entry *prev = NULL; struct dirent *de; while ((de = readdir(dir)) != NULL) {

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: In other words, the let user space sort out the complexity is not a good answer. It just means that the interface is badly designed. Well, if we can setup iocb structure, why we can not setup syslet one? (I'm cutting wildly, to try to get

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 08:09:54AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: In other words, the let user space sort out the complexity is not a good answer. It just means that the interface is badly designed. Well, if we can setup iocb structure, why we can not setup

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Davide Libenzi
On Thu, 15 Feb 2007, Linus Torvalds wrote: I don't think the atom approach is bad per se. I think it could be fine to have some state information in user space. It's just that I think complex interfaces that people largely won't even use is a big mistake. We should concentrate on usability

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Ulrich Drepper
Linus Torvalds wrote: Here's a quick question: how many people have actually ever seen them used in normal code? Yeah. Nobody uses them. They're not all that portable (even within unixes they aren't always there, much less in other places), they are fairly obscure, and they are just not

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 09:05:13AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: On Thu, 15 Feb 2007, Linus Torvalds wrote: I don't think the atom approach is bad per se. I think it could be fine to have some state information in user space. It's just that I think complex

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Davide Libenzi
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: On Thu, Feb 15, 2007 at 09:05:13AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: I actually think that building chains of syscalls bring you back to a multithreaded solution. Why? Because suddendly the service thread become from

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: Userspace_API_is_the_ever_possible_last_thing_to_ever_think_about. Period . // - wrapped one No, I really think you're wrong. In many ways, the interfaces and especially data structures are *more* important than the code. The code we can fix.

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 09:39:33AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: On Thu, Feb 15, 2007 at 09:05:13AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: I actually think that building chains of syscalls bring

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 09:42:32AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: Userspace_API_is_the_ever_possible_last_thing_to_ever_think_about. Period . // - wrapped one No, I really think you're wrong. In many ways, the

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: So we just need to describe the way we want to see new interface - that's it. Agreed. Absolutely. But please keep the kernel interface as part of that. Not just a strange and complex kernel interface and then _usable_ library interfaces that

Re: [patch 05/11] syslets: core code

2007-02-15 Thread bert hubert
On Thu, Feb 15, 2007 at 09:42:32AM -0800, Linus Torvalds wrote: We know one interface: the current aio_read() one. Nobody really _likes_ [...] Others? We don't know yet. And exposing complex interfaces that may not be the right ones is much *worse* than exposing simple interfaces (that

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 10:25:37AM -0800, Linus Torvalds ([EMAIL PROTECTED]) wrote: static void syslet_setup(struct syslet *s, int nr, void *arg1...) { s-flags = ... s-arg[1] = arg1; } long glibc_async_stat(const char *path, struct stat *buf) { /* What

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Evgeniy Polyakov
On Thu, Feb 15, 2007 at 07:46:56PM +0100, bert hubert ([EMAIL PROTECTED]) wrote: 1) batch, and wait for, with proper error reporting: socket(); [ setsockopt(); ] bind(); connect(); gettimeofday(); // doesn't *always* happen send(); recv();

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Zach Brown
2) On the client facing side (port 53), I'd very much hope for a way to do 'recvv' on datagram sockets, so I can retrieve a whole bunch of UDP datagrams with only one kernel transition. I want to highlight this point that Bert is making. Whenever we talk about AIO and kernel

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Eric Dumazet
On Thursday 15 February 2007 19:46, bert hubert wrote: Both 1 and 2 are currently limiting factors when I enter the 100kqps domain of name serving. This doesn't mean the rest of my code is as tight as it could be, but I spend a significant portion of time in the kernel even at moderate

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Evgeniy Polyakov wrote: See? The example you tried to use to show how simple the interface iswas actually EXACTLY THE REVERSE. It shows how subtle bugs can creep in! So describe what are the requirements (constraints)? Above example has exactly one syscall in

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Linus Torvalds
On Thu, 15 Feb 2007, Linus Torvalds wrote: So I think that a good implementation just does everything up-front, and doesn't _need_ a user buffer that is live over longer periods, except for the actual results. Exactly because the whole alloc/teardown is nasty. Btw, this doesn't

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Davide Libenzi
On Thu, 15 Feb 2007, Linus Torvalds wrote: On Thu, 15 Feb 2007, Linus Torvalds wrote: So I think that a good implementation just does everything up-front, and doesn't _need_ a user buffer that is live over longer periods, except for the actual results. Exactly because the whole

Re: [patch 05/11] syslets: core code

2007-02-15 Thread Michael K. Edwards
On 2/15/07, Linus Torvalds [EMAIL PROTECTED] wrote: Would it make the interface less cool? Yeah. Would it limit it to just a few linked system calls (to avoid memory allocation issues in the kernel)? Yes again. But it would simplify a lot of the interface issues. Only in toy applications.

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Zach Brown
But the whole point is that the notion of a "register" is wrong in the first place. [...] forget about it then. The thing we "register" is dead-simple: struct async_head_user { struct syslet_uatom __user **completion_ring; unsigned long

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Davide Libenzi
On Wed, 14 Feb 2007, Davide Libenzi wrote: > On Wed, 14 Feb 2007, Ingo Molnar wrote: > > > yeah, that's another key thing. I do plan to provide a sys_upcall() > > syscall as well which calls a 5-parameter user-space function with a > > special stack. (it's like a lightweight signal/event

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Davide Libenzi
On Wed, 14 Feb 2007, Ingo Molnar wrote: > yeah, that's another key thing. I do plan to provide a sys_upcall() > syscall as well which calls a 5-parameter user-space function with a > special stack. (it's like a lightweight signal/event handler, without > any of the signal handler legacies and

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Ingo Molnar <[EMAIL PROTECTED]> wrote: > > You are not counting the whole setup cost there, then, because your > > setup cost is going to be at a minimum more expensive than the null > > system call. > > hm, this one-time cost was never on my radar. [ It's really dwarved by > other startup

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > > case. (but with some crazier hacks i got the one-shot atom overhead > > [compared to a simple synchronous null syscall] to below 10 nsecs, > > so there's room in there for further optimizations. Our current null > > syscall latency is around

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Linus Torvalds
On Wed, 14 Feb 2007, Ingo Molnar wrote: > > i think you are banging on open doors. That async_stat() call is very > much what i'd like to see glibc to provide, not really the raw syslet > interface. Right. Which is why I wrote (and you removed) the rest of my email. If the "raw" interfaces

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Alan <[EMAIL PROTECTED]> wrote: > >this "AIO atom" in the first place, WHICH WE KNOW IS INCORRECT, > >since current users use "aio_read()" that simply doesn't have > >that and doesn't build up any such data structures. > > Do current users do this because that is all they have,

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > Or how would you do the trivial example loop that I explained was a > good idea: > > struct one_entry *prev = NULL; > struct dirent *de; > > while ((de = readdir(dir)) != NULL) { > struct one_entry *entry =

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > - it fundamentally is based on a broken notion that everything would >use this "AIO atom" in the first place, WHICH WE KNOW IS INCORRECT, >since current users use "aio_read()" that simply doesn't have that >and doesn't build up any

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Alan
> - it assumes we are going to make these complex state machines (which I >don't believe for a second that a real program will do) They've not had the chance before and there are certain chains of them which make huge amounts of sense because you don't want to keep taking completion hits.

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > But the whole point is that the notion of a "register" is wrong in the > first place. [...] forget about it then. The thing we "register" is dead-simple: struct async_head_user { struct syslet_uatom __user **completion_ring;

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Linus Torvalds
On Wed, 14 Feb 2007, Ingo Molnar wrote: > > hm, there must be some misunderstanding here. That mlock is /only/ once > per the lifetime of the whole 'head' - i.e. per sys_async_register(). > (And you can even forget i ever did it - it's 5 lines of code to turn > the completion ring into a

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Ingo Molnar <[EMAIL PROTECTED]> wrote: > * Linus Torvalds <[EMAIL PROTECTED]> wrote: > > > And the whole "lock things down in memory" approach is bad. It's > > doing expensive things like mlock(), making the overhead for > > _single_ system calls much more expensive. [...] > > hm, there

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Davide Libenzi
On Wed, 14 Feb 2007, Linus Torvalds wrote: > > > On Tue, 13 Feb 2007, Ingo Molnar wrote: > > > > the core syslet / async system calls infrastructure code. > > Ok, having now looked at it more, I can say: > > - I hate it. > > I dislike it intensely, because it's so _close_ to being usable.

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > And the whole "lock things down in memory" approach is bad. It's doing > expensive things like mlock(), making the overhead for _single_ system > calls much more expensive. [...] hm, there must be some misunderstanding here. That mlock is /only/

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Linus Torvalds
On Tue, 13 Feb 2007, Ingo Molnar wrote: > > the core syslet / async system calls infrastructure code. Ok, having now looked at it more, I can say: - I hate it. I dislike it intensely, because it's so _close_ to being usable. But the programming interface looks absolutely horrid for any

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Stephen Rothwell
Hi Ingo, On Tue, 13 Feb 2007 15:20:35 +0100 Ingo Molnar <[EMAIL PROTECTED]> wrote: > > From: Ingo Molnar <[EMAIL PROTECTED]> > > the core syslet / async system calls infrastructure code. It occurred to me that the 32 compat code for 64 bit architectures for all this could be very hairy ... --

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Guillaume Chazarain
Ingo Molnar a écrit : + if (unlikely(signal_pending(t) || need_resched())) + goto stop; So, this is how you'll prevent me from running an infinite loop ;-) The attached patch adds a cond_resched() instead, to allow infinite loops without DoS. I dropped the unlikely() as

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Evgeniy Polyakov
On Wed, Feb 14, 2007 at 11:30:55AM +0100, Arjan van de Ven ([EMAIL PROTECTED]) wrote: > > (at least on Debian > > and Mandrake there is no locked memory limit by default). > > that sounds like 2 very large bugtraq-worthy bugs in these distros.. so > bad a bug that I almost find it hard to

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Arjan van de Ven
> (at least on Debian > and Mandrake there is no locked memory limit by default). that sounds like 2 very large bugtraq-worthy bugs in these distros.. so bad a bug that I almost find it hard to believe... -- if you want to mail me at work (you don't), use arjan (at) linux.intel.com Test the

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Evgeniy Polyakov
On Wed, Feb 14, 2007 at 10:46:29AM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > This will end up badly - I used the same approach in the early kevent > > days and was proven to have swapable memory for the ring. I think it > > would be

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > This will end up badly - I used the same approach in the early kevent > days and was proven to have swapable memory for the ring. I think it > would be much better to have userspace allocated ring and use > copy_to_user() there. it is a

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Evgeniy Polyakov
On Tue, Feb 13, 2007 at 11:41:31PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > Then limit it to a single page and use gup > > 1024 (512 on 64-bit) is alot but not ALOT. It is also certainly not > ALT :-) Really, people will want to have more than 512 > disks/spindles in the same box.

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Evgeniy Polyakov
On Tue, Feb 13, 2007 at 11:41:31PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: Then limit it to a single page and use gup 1024 (512 on 64-bit) is alot but not ALOT. It is also certainly not ALT :-) Really, people will want to have more than 512 disks/spindles in the same box. I have

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Ingo Molnar
* Evgeniy Polyakov [EMAIL PROTECTED] wrote: This will end up badly - I used the same approach in the early kevent days and was proven to have swapable memory for the ring. I think it would be much better to have userspace allocated ring and use copy_to_user() there. it is a userspace

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Evgeniy Polyakov
On Wed, Feb 14, 2007 at 10:46:29AM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: * Evgeniy Polyakov [EMAIL PROTECTED] wrote: This will end up badly - I used the same approach in the early kevent days and was proven to have swapable memory for the ring. I think it would be much better

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Arjan van de Ven
(at least on Debian and Mandrake there is no locked memory limit by default). that sounds like 2 very large bugtraq-worthy bugs in these distros.. so bad a bug that I almost find it hard to believe... -- if you want to mail me at work (you don't), use arjan (at) linux.intel.com Test the

Re: [patch 05/11] syslets: core code

2007-02-14 Thread Evgeniy Polyakov
On Wed, Feb 14, 2007 at 11:30:55AM +0100, Arjan van de Ven ([EMAIL PROTECTED]) wrote: (at least on Debian and Mandrake there is no locked memory limit by default). that sounds like 2 very large bugtraq-worthy bugs in these distros.. so bad a bug that I almost find it hard to believe...

  1   2   >