Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Zach Brown
I'm finally back from my travel and conference hiatus.. you guys have been busy! :) On Feb 13, 2007, at 6:20 AM, Ingo Molnar wrote: I'm pleased to announce the first release of the "Syslet" kernel feature and kernel subsystem, which provides generic asynchrous system call support:

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Davide Libenzi
On Wed, 14 Feb 2007, Jeremy Fitzhardinge wrote: > Davide Libenzi wrote: > >> Would this work? > >> > > > > Hopefully the API will simplify enough so that emulation will becomes > > easier. > > > > The big question in my mind is how all this stuff interacts with > signals. Can a blocked

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Jeremy Fitzhardinge
Davide Libenzi wrote: >> Would this work? >> > > Hopefully the API will simplify enough so that emulation will becomes > easier. > The big question in my mind is how all this stuff interacts with signals. Can a blocked syscall atom be interrupted by a signal? If so, what thread does it

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Davide Libenzi
On Wed, 14 Feb 2007, Jeremy Fitzhardinge wrote: > Are there any special semantics that result from running the syslet > atoms in kernel mode? If I wanted to, could I write a syslet emulation > in userspace that's functionally identical to a kernel-based > implementation? (Obviously the

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Jeremy Fitzhardinge
Ingo Molnar wrote: > Syslets consist of 'syslet atoms', where each atom represents a single > system-call. These atoms can be chained to each other: serially, in > branches or in loops. The return value of an executed atom is checked > against the condition flags. So an atom can specify 'exit

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Davide Libenzi
On Wed, 14 Feb 2007, Ingo Molnar wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > Let me clarify what I meant. There is only limited number of threads, > > which are supposed to execute blocking context, so when all they are > > used, main one will block too - I asked about

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Linus Torvalds
On Wed, 14 Feb 2007, Pavel Machek wrote: > > Ouch, yet another interpretter in kernel :-(. Can we reuse acpi or > something? Hah. You make the joke! I get it! Mwahahahaa! Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Pavel Machek
Hi! > The boring details: > > Syslets consist of 'syslet atoms', where each atom represents a single > system-call. These atoms can be chained to each other: serially, in > branches or in loops. The return value of an executed atom is checked > against the condition flags. So an atom can

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Evgeniy Polyakov
On Wed, Feb 14, 2007 at 11:37:31AM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > Let me clarify what I meant. There is only limited number of threads, > > which are supposed to execute blocking context, so when all they are > > used, main one will block too - I asked about possibility to

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Alan
> > Ohh. OpenVMS lives forever ;) Me likeee ;) > > hm, i dont know OpenVMS - but googled around a bit for 'VMS > asynchronous' and it gave me this: VMS had SYS$QIO which is asynchronous I/O queueing with completions of sorts. You had to specifically remember if you wanted to a synchronous

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > Let me clarify what I meant. There is only limited number of threads, > which are supposed to execute blocking context, so when all they are > used, main one will block too - I asked about possibility to reuse the > same thread to execute queue

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Evgeniy Polyakov
On Tue, Feb 13, 2007 at 11:18:10PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > [...] it still has a problem - syscall blocks and the same thread thus > > is not allowed to continue execution and fill the pipe - so what if > > system

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-14 Thread Ingo Molnar
* Davide Libenzi wrote: > > There's another problem AFAICS: > > > > - We woke up one of the cachemiss_loop threads in pick_new_async_thread > > > > - The threads wakes up, mark itself as busy, and look at the ->work > > pointer hoping to find something to work on > > > > But we never set

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Ingo Molnar
* Davide Libenzi davidel@xmailserver.org wrote: There's another problem AFAICS: - We woke up one of the cachemiss_loop threads in pick_new_async_thread - The threads wakes up, mark itself as busy, and look at the -work pointer hoping to find something to work on But we

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Evgeniy Polyakov
On Tue, Feb 13, 2007 at 11:18:10PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: * Evgeniy Polyakov [EMAIL PROTECTED] wrote: [...] it still has a problem - syscall blocks and the same thread thus is not allowed to continue execution and fill the pipe - so what if system issues

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Ingo Molnar
* Evgeniy Polyakov [EMAIL PROTECTED] wrote: Let me clarify what I meant. There is only limited number of threads, which are supposed to execute blocking context, so when all they are used, main one will block too - I asked about possibility to reuse the same thread to execute queue of

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Alan
Ohh. OpenVMS lives forever ;) Me likeee ;) hm, i dont know OpenVMS - but googled around a bit for 'VMS asynchronous' and it gave me this: VMS had SYS$QIO which is asynchronous I/O queueing with completions of sorts. You had to specifically remember if you wanted to a synchronous I/O.

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Evgeniy Polyakov
On Wed, Feb 14, 2007 at 11:37:31AM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: Let me clarify what I meant. There is only limited number of threads, which are supposed to execute blocking context, so when all they are used, main one will block too - I asked about possibility to reuse the

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Pavel Machek
Hi! The boring details: Syslets consist of 'syslet atoms', where each atom represents a single system-call. These atoms can be chained to each other: serially, in branches or in loops. The return value of an executed atom is checked against the condition flags. So an atom can specify

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Linus Torvalds
On Wed, 14 Feb 2007, Pavel Machek wrote: Ouch, yet another interpretter in kernel :-(. Can we reuse acpi or something? Hah. You make the joke! I get it! Mwahahahaa! Linus - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Davide Libenzi
On Wed, 14 Feb 2007, Ingo Molnar wrote: * Evgeniy Polyakov [EMAIL PROTECTED] wrote: Let me clarify what I meant. There is only limited number of threads, which are supposed to execute blocking context, so when all they are used, main one will block too - I asked about possibility to

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Jeremy Fitzhardinge
Ingo Molnar wrote: Syslets consist of 'syslet atoms', where each atom represents a single system-call. These atoms can be chained to each other: serially, in branches or in loops. The return value of an executed atom is checked against the condition flags. So an atom can specify 'exit on

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Davide Libenzi
On Wed, 14 Feb 2007, Jeremy Fitzhardinge wrote: Are there any special semantics that result from running the syslet atoms in kernel mode? If I wanted to, could I write a syslet emulation in userspace that's functionally identical to a kernel-based implementation? (Obviously the performance

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Jeremy Fitzhardinge
Davide Libenzi wrote: Would this work? Hopefully the API will simplify enough so that emulation will becomes easier. The big question in my mind is how all this stuff interacts with signals. Can a blocked syscall atom be interrupted by a signal? If so, what thread does it get

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Davide Libenzi
On Wed, 14 Feb 2007, Jeremy Fitzhardinge wrote: Davide Libenzi wrote: Would this work? Hopefully the API will simplify enough so that emulation will becomes easier. The big question in my mind is how all this stuff interacts with signals. Can a blocked syscall atom be

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-14 Thread Zach Brown
I'm finally back from my travel and conference hiatus.. you guys have been busy! :) On Feb 13, 2007, at 6:20 AM, Ingo Molnar wrote: I'm pleased to announce the first release of the Syslet kernel feature and kernel subsystem, which provides generic asynchrous system call support:

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Davide Libenzi wrote: [...] > So the sys_async_exec task is going to block. Now, am I being really > tired, or the cachemiss fast return is simply not there? The former 8) pick_new_async_head() new_task->ah = ah; cachemiss_loop() for (;;) {

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Willy Tarreau
Hi Ingo ! On Tue, Feb 13, 2007 at 03:20:10PM +0100, Ingo Molnar wrote: > I'm pleased to announce the first release of the "Syslet" kernel feature > and kernel subsystem, which provides generic asynchrous system call > support: > >http://redhat.com/~mingo/syslet-patches/ > > Syslets are

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Ingo Molnar wrote: > I'm pleased to announce the first release of the "Syslet" kernel feature > and kernel subsystem, which provides generic asynchrous system call > support: > [...] Ok, I had little to time review the code, but it has been a long working day, so bear

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Ingo Molnar wrote: > * Davide Libenzi wrote: > > > > If this is going to be a generic AIO subsystem: > > > > > > - Cancellation of peding request > > > > What about the busy_async_threads list becoming a hash/rb_tree indexed > > by syslet_atom ptr. A cancel would lookup

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Ingo Molnar wrote: > > * Davide Libenzi wrote: > > > > Open issues: > > > If this is going to be a generic AIO subsystem: > > > > - Cancellation of pending request > > How about implementing aio_cancel() as a NOP. Can anyone prove that the > kernel didnt actually

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ulrich Drepper
Ingo Molnar wrote: > really, what's the point behind aio_cancel()? - sequence aio_write() aio_cancel() aio_write() with both writes going to the same place must be predictably - think beyond files. Writes to sockets, ttys, they can block and cancel must abort them. Even for

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Olivier Galibert
On Tue, Feb 13, 2007 at 10:57:24PM +0100, Ingo Molnar wrote: > > * Davide Libenzi wrote: > > > > Open issues: > > > If this is going to be a generic AIO subsystem: > > > > - Cancellation of pending request > > How about implementing aio_cancel() as a NOP. Can anyone prove that the > kernel

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Andi Kleen
> ok. The TID+signal approach i mentioned in the other reply should work. Not sure if a signal is good for this. It might conflict with existing strange historical semantics. > If it's frequent enough we could make this an explicit > sys_async_cancel(TID) API. Ideally there should be a new

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Andi Kleen <[EMAIL PROTECTED]> wrote: > > ok, that should work fine already - exit in the user context gets > > That would be a little heavy handed. I wouldn't expect my GUI program > to quit itself on cancel. And requiring it to create a new thread just > to exit on cancel would be also

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Dmitry Torokhov
Hi Ingo, On Tuesday 13 February 2007 15:39, Ingo Molnar wrote: > > * Dmitry Torokhov <[EMAIL PROTECTED]> wrote: > > > > What are the semantics of async sys_async_wait and async sys_async ? > > > > Ohh. OpenVMS lives forever ;) Me likeee ;) > > hm, i dont know OpenVMS - but googled around

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Andi Kleen
On Tue, Feb 13, 2007 at 11:26:26PM +0100, Ingo Molnar wrote: > > * Andi Kleen <[EMAIL PROTECTED]> wrote: > > > > really, what's the point behind aio_cancel()? > > > > The main use case is when you open a file requester on a network file > > system where the server is down and you get tired of

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Andi Kleen <[EMAIL PROTECTED]> wrote: > > really, what's the point behind aio_cancel()? > > The main use case is when you open a file requester on a network file > system where the server is down and you get tired of waiting and press > "Cancel" it should abort the hanging IO immediately.

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Andi Kleen
Ingo Molnar <[EMAIL PROTECTED]> writes: > > really, what's the point behind aio_cancel()? The main use case is when you open a file requester on a network file system where the server is down and you get tired of waiting and press "Cancel" it should abort the hanging IO immediately. At least I

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > I have not received first mail with announcement yet, so I will place > > my thoughts here if you do not mind. > > An issue with sys_async_wait(): is is possible that events_left will > be setup too late so that all events are already ready

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > [...] it still has a problem - syscall blocks and the same thread thus > is not allowed to continue execution and fill the pipe - so what if > system issues thousands of requests and there are only tens of working > thread at most. [...] the

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Davide Libenzi wrote: > > If this is going to be a generic AIO subsystem: > > > > - Cancellation of peding request > > What about the busy_async_threads list becoming a hash/rb_tree indexed > by syslet_atom ptr. A cancel would lookup the thread and send a signal > (of course, signal

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Davide Libenzi wrote: > > Open issues: > If this is going to be a generic AIO subsystem: > > - Cancellation of pending request How about implementing aio_cancel() as a NOP. Can anyone prove that the kernel didnt actually attempt to cancel that IO? [but unfortunately failed at doing so,

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Davide Libenzi wrote: > If this is going to be a generic AIO subsystem: > > - Cancellation of peding request What about the busy_async_threads list becoming a hash/rb_tree indexed by syslet_atom ptr. A cancel would lookup the thread and send a signal (of course, signal

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Dmitry Torokhov <[EMAIL PROTECTED]> wrote: > > What are the semantics of async sys_async_wait and async sys_async ? > > Ohh. OpenVMS lives forever ;) Me likeee ;) hm, i dont know OpenVMS - but googled around a bit for 'VMS asynchronous' and it gave me this:

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Benjamin LaHaise <[EMAIL PROTECTED]> wrote: > [...] interaction with set_fs()... hm, this one should already work in the current version, because addr_limit is in thread_info and hence stays with the async context. Or can you see any hole in it? Ingo - To unsubscribe from this

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Linus Torvalds wrote: > if (in_async_context()) > return -EINVAL; > > or similar. We need that "async_context()" function anyway for the other > cases where we can't do other things concurrently, like changing the UID. Yes, that's definitely better.

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Ingo Molnar wrote: > As it might be obvious to some of you, the syslet subsystem takes many > ideas and experience from my Tux in-kernel webserver :) The syslet code > originates from a heavy rewrite of the Tux-atom and the Tux-cachemiss > infrastructure. > > Open issues:

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Evgeniy Polyakov
> I have not received first mail with announcement yet, so I will place > my thoughts here if you do not mind. An issue with sys_async_wait(): is is possible that events_left will be setup too late so that all events are already ready and thus sys_async_wait() can wait forever (or until next

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Evgeniy Polyakov
On Tue, Feb 13, 2007 at 05:56:42PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Benjamin LaHaise <[EMAIL PROTECTED]> wrote: > > > > > Open issues: > > > > > > Let me add some more > > > > Also: FPU state (especially important with the FPU and SSE memory copy > > variants), segment

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > Ie, we could just add to "do_fork()" (which is where all of the > vfork/clone/fork cases end up) a simple case like > > err = wait_async_context(); > if (err) > return err; > > or > > if (in_async_context()) >

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Benjamin LaHaise <[EMAIL PROTECTED]> wrote: > > > Open issues: > > > > Let me add some more > > Also: FPU state (especially important with the FPU and SSE memory copy > variants), segment register bases on x86-64, interaction with > set_fs()... agreed - i'll fix this. But i can see no big

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Andi Kleen <[EMAIL PROTECTED]> wrote: > > sys_exec and other security boundaries must be synchronous > > only and not allow async "spill over" (consider setuid async binary > > patching) > > He probably would need some generalization of Andrea's seccomp work. > Perhaps using bitmaps?

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Alan <[EMAIL PROTECTED]> wrote: > > A syslet is executed opportunistically: i.e. the syslet subsystem > > assumes that the syslet will not block, and it will switch to a > > cachemiss kernel thread from the scheduler. This means that even a > > How is scheduler fairness maintained ? and

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Linus Torvalds
On Tue, 13 Feb 2007, Andi Kleen wrote: > > sys_exec and other security boundaries must be synchronous only > > and not allow async "spill over" (consider setuid async binary patching) > > He probably would need some generalization of Andrea's seccomp work. > Perhaps using bitmaps? For

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread bert hubert
On Tue, Feb 13, 2007 at 09:58:48AM -0500, Benjamin LaHaise wrote: > not present is mandatory). I have looked into exactly this approach, and > it's only cheaper if the code is incomplete. Linux's native threads are > pretty damned good. Cheaper in time or in memory? Iow, would you be able to

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Dmitry Torokhov
On 2/13/07, Alan <[EMAIL PROTECTED]> wrote: > A syslet is executed opportunistically: i.e. the syslet subsystem > assumes that the syslet will not block, and it will switch to a > cachemiss kernel thread from the scheduler. This means that even a How is scheduler fairness maintained ? and what

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Andi Kleen
Alan <[EMAIL PROTECTED]> writes: Funny, it sounds like batch() on stereoids @) Ok with an async context it becomes somewhat more interesting. > sys_setuid/gid/etc need to be synchronous only and not occur > while other async syscalls are running in parallel to meet current kernel >

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Arjan van de Ven
On Tue, 2007-02-13 at 09:58 -0500, Benjamin LaHaise wrote: > On Tue, Feb 13, 2007 at 03:00:19PM +, Alan wrote: > > > Open issues: > > > > Let me add some more > > Also: FPU state (especially important with the FPU and SSE memory copy > variants) are these preserved over explicit system

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Benjamin LaHaise
On Tue, Feb 13, 2007 at 03:00:19PM +, Alan wrote: > > Open issues: > > Let me add some more Also: FPU state (especially important with the FPU and SSE memory copy variants), segment register bases on x86-64, interaction with set_fs()... There is no easy way of getting around the full

Re: [patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Alan
> A syslet is executed opportunistically: i.e. the syslet subsystem > assumes that the syslet will not block, and it will switch to a > cachemiss kernel thread from the scheduler. This means that even a How is scheduler fairness maintained ? and what is done for resource accounting here ? >

[patch 00/11] ANNOUNCE: "Syslets", generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
I'm pleased to announce the first release of the "Syslet" kernel feature and kernel subsystem, which provides generic asynchrous system call support: http://redhat.com/~mingo/syslet-patches/ Syslets are small, simple, lightweight programs (consisting of system-calls, 'atoms') that the

[patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
I'm pleased to announce the first release of the Syslet kernel feature and kernel subsystem, which provides generic asynchrous system call support: http://redhat.com/~mingo/syslet-patches/ Syslets are small, simple, lightweight programs (consisting of system-calls, 'atoms') that the kernel

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Alan
A syslet is executed opportunistically: i.e. the syslet subsystem assumes that the syslet will not block, and it will switch to a cachemiss kernel thread from the scheduler. This means that even a How is scheduler fairness maintained ? and what is done for resource accounting here ? that

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Benjamin LaHaise
On Tue, Feb 13, 2007 at 03:00:19PM +, Alan wrote: Open issues: Let me add some more Also: FPU state (especially important with the FPU and SSE memory copy variants), segment register bases on x86-64, interaction with set_fs()... There is no easy way of getting around the full thread

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Arjan van de Ven
On Tue, 2007-02-13 at 09:58 -0500, Benjamin LaHaise wrote: On Tue, Feb 13, 2007 at 03:00:19PM +, Alan wrote: Open issues: Let me add some more Also: FPU state (especially important with the FPU and SSE memory copy variants) are these preserved over explicit system calls? -- if

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Andi Kleen
Alan [EMAIL PROTECTED] writes: Funny, it sounds like batch() on stereoids @) Ok with an async context it becomes somewhat more interesting. sys_setuid/gid/etc need to be synchronous only and not occur while other async syscalls are running in parallel to meet current kernel

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Dmitry Torokhov
On 2/13/07, Alan [EMAIL PROTECTED] wrote: A syslet is executed opportunistically: i.e. the syslet subsystem assumes that the syslet will not block, and it will switch to a cachemiss kernel thread from the scheduler. This means that even a How is scheduler fairness maintained ? and what is

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread bert hubert
On Tue, Feb 13, 2007 at 09:58:48AM -0500, Benjamin LaHaise wrote: not present is mandatory). I have looked into exactly this approach, and it's only cheaper if the code is incomplete. Linux's native threads are pretty damned good. Cheaper in time or in memory? Iow, would you be able to

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Linus Torvalds
On Tue, 13 Feb 2007, Andi Kleen wrote: sys_exec and other security boundaries must be synchronous only and not allow async spill over (consider setuid async binary patching) He probably would need some generalization of Andrea's seccomp work. Perhaps using bitmaps? For paranoia I

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Alan [EMAIL PROTECTED] wrote: A syslet is executed opportunistically: i.e. the syslet subsystem assumes that the syslet will not block, and it will switch to a cachemiss kernel thread from the scheduler. This means that even a How is scheduler fairness maintained ? and what is done

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Andi Kleen [EMAIL PROTECTED] wrote: sys_exec and other security boundaries must be synchronous only and not allow async spill over (consider setuid async binary patching) He probably would need some generalization of Andrea's seccomp work. Perhaps using bitmaps? For paranoia I

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Benjamin LaHaise [EMAIL PROTECTED] wrote: Open issues: Let me add some more Also: FPU state (especially important with the FPU and SSE memory copy variants), segment register bases on x86-64, interaction with set_fs()... agreed - i'll fix this. But i can see no big conceptual

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Linus Torvalds [EMAIL PROTECTED] wrote: Ie, we could just add to do_fork() (which is where all of the vfork/clone/fork cases end up) a simple case like err = wait_async_context(); if (err) return err; or if (in_async_context())

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Evgeniy Polyakov
On Tue, Feb 13, 2007 at 05:56:42PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: * Benjamin LaHaise [EMAIL PROTECTED] wrote: Open issues: Let me add some more Also: FPU state (especially important with the FPU and SSE memory copy variants), segment register bases on x86-64,

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Evgeniy Polyakov
I have not received first mail with announcement yet, so I will place my thoughts here if you do not mind. An issue with sys_async_wait(): is is possible that events_left will be setup too late so that all events are already ready and thus sys_async_wait() can wait forever (or until next

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Ingo Molnar wrote: As it might be obvious to some of you, the syslet subsystem takes many ideas and experience from my Tux in-kernel webserver :) The syslet code originates from a heavy rewrite of the Tux-atom and the Tux-cachemiss infrastructure. Open issues: -

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Linus Torvalds wrote: if (in_async_context()) return -EINVAL; or similar. We need that async_context() function anyway for the other cases where we can't do other things concurrently, like changing the UID. Yes, that's definitely better. Let's have

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Benjamin LaHaise [EMAIL PROTECTED] wrote: [...] interaction with set_fs()... hm, this one should already work in the current version, because addr_limit is in thread_info and hence stays with the async context. Or can you see any hole in it? Ingo - To unsubscribe from this list:

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Dmitry Torokhov [EMAIL PROTECTED] wrote: What are the semantics of async sys_async_wait and async sys_async ? Ohh. OpenVMS lives forever ;) Me likeee ;) hm, i dont know OpenVMS - but googled around a bit for 'VMS asynchronous' and it gave me this:

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Davide Libenzi wrote: If this is going to be a generic AIO subsystem: - Cancellation of peding request What about the busy_async_threads list becoming a hash/rb_tree indexed by syslet_atom ptr. A cancel would lookup the thread and send a signal (of course, signal

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Davide Libenzi davidel@xmailserver.org wrote: Open issues: If this is going to be a generic AIO subsystem: - Cancellation of pending request How about implementing aio_cancel() as a NOP. Can anyone prove that the kernel didnt actually attempt to cancel that IO? [but unfortunately

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Davide Libenzi davidel@xmailserver.org wrote: If this is going to be a generic AIO subsystem: - Cancellation of peding request What about the busy_async_threads list becoming a hash/rb_tree indexed by syslet_atom ptr. A cancel would lookup the thread and send a signal (of course,

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Evgeniy Polyakov [EMAIL PROTECTED] wrote: [...] it still has a problem - syscall blocks and the same thread thus is not allowed to continue execution and fill the pipe - so what if system issues thousands of requests and there are only tens of working thread at most. [...] the same

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Evgeniy Polyakov [EMAIL PROTECTED] wrote: I have not received first mail with announcement yet, so I will place my thoughts here if you do not mind. An issue with sys_async_wait(): is is possible that events_left will be setup too late so that all events are already ready and thus

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Andi Kleen
Ingo Molnar [EMAIL PROTECTED] writes: really, what's the point behind aio_cancel()? The main use case is when you open a file requester on a network file system where the server is down and you get tired of waiting and press Cancel it should abort the hanging IO immediately. At least I would

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Andi Kleen [EMAIL PROTECTED] wrote: really, what's the point behind aio_cancel()? The main use case is when you open a file requester on a network file system where the server is down and you get tired of waiting and press Cancel it should abort the hanging IO immediately. ok, that

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Andi Kleen
On Tue, Feb 13, 2007 at 11:26:26PM +0100, Ingo Molnar wrote: * Andi Kleen [EMAIL PROTECTED] wrote: really, what's the point behind aio_cancel()? The main use case is when you open a file requester on a network file system where the server is down and you get tired of waiting and

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Dmitry Torokhov
Hi Ingo, On Tuesday 13 February 2007 15:39, Ingo Molnar wrote: * Dmitry Torokhov [EMAIL PROTECTED] wrote: What are the semantics of async sys_async_wait and async sys_async ? Ohh. OpenVMS lives forever ;) Me likeee ;) hm, i dont know OpenVMS - but googled around a bit for 'VMS

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ingo Molnar
* Andi Kleen [EMAIL PROTECTED] wrote: ok, that should work fine already - exit in the user context gets That would be a little heavy handed. I wouldn't expect my GUI program to quit itself on cancel. And requiring it to create a new thread just to exit on cancel would be also nasty.

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Andi Kleen
ok. The TID+signal approach i mentioned in the other reply should work. Not sure if a signal is good for this. It might conflict with existing strange historical semantics. If it's frequent enough we could make this an explicit sys_async_cancel(TID) API. Ideally there should be a new

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Olivier Galibert
On Tue, Feb 13, 2007 at 10:57:24PM +0100, Ingo Molnar wrote: * Davide Libenzi davidel@xmailserver.org wrote: Open issues: If this is going to be a generic AIO subsystem: - Cancellation of pending request How about implementing aio_cancel() as a NOP. Can anyone prove that the

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Ulrich Drepper
Ingo Molnar wrote: really, what's the point behind aio_cancel()? - sequence aio_write() aio_cancel() aio_write() with both writes going to the same place must be predictably - think beyond files. Writes to sockets, ttys, they can block and cancel must abort them. Even for

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Ingo Molnar wrote: * Davide Libenzi davidel@xmailserver.org wrote: Open issues: If this is going to be a generic AIO subsystem: - Cancellation of pending request How about implementing aio_cancel() as a NOP. Can anyone prove that the kernel didnt

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Ingo Molnar wrote: * Davide Libenzi davidel@xmailserver.org wrote: If this is going to be a generic AIO subsystem: - Cancellation of peding request What about the busy_async_threads list becoming a hash/rb_tree indexed by syslet_atom ptr. A cancel would

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Ingo Molnar wrote: I'm pleased to announce the first release of the Syslet kernel feature and kernel subsystem, which provides generic asynchrous system call support: [...] Ok, I had little to time review the code, but it has been a long working day, so bear with me

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Willy Tarreau
Hi Ingo ! On Tue, Feb 13, 2007 at 03:20:10PM +0100, Ingo Molnar wrote: I'm pleased to announce the first release of the Syslet kernel feature and kernel subsystem, which provides generic asynchrous system call support: http://redhat.com/~mingo/syslet-patches/ Syslets are small,

Re: [patch 00/11] ANNOUNCE: Syslets, generic asynchronous system call support

2007-02-13 Thread Davide Libenzi
On Tue, 13 Feb 2007, Davide Libenzi wrote: [...] So the sys_async_exec task is going to block. Now, am I being really tired, or the cachemiss fast return is simply not there? The former 8) pick_new_async_head() new_task-ah = ah; cachemiss_loop() for (;;) {