Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Tue, Dec 19, 2000 at 09:42:05AM +0100, Mikulas Patocka wrote: > Failing getblk would likely introduce filesystem corruption. Look at > getblk in 2.0 - when allocating new page fails it tries to reuse existing > clean buffers or wakes up bdflush and waits until it writes them. This is > the right thing to do. wrong. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Tue, 19 Dec 2000, Andrea Arcangeli wrote: > On Mon, Dec 18, 2000 at 10:57:44PM +0100, Mikulas Patocka wrote: > > You have small posibility that interrupt will eat up memory - interrupt in > > process that has PF_MEMALLOC. Patch: > > this is not the point of getblk, to fix the getblk deadlock the only way is to > implement a fail path in each caller and allow getblk to return NULL (as every > other memory allocation function can do). Failing getblk would likely introduce filesystem corruption. Look at getblk in 2.0 - when allocating new page fails it tries to reuse existing clean buffers or wakes up bdflush and waits until it writes them. This is the right thing to do. Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Tue, 19 Dec 2000, Andrea Arcangeli wrote: On Mon, Dec 18, 2000 at 10:57:44PM +0100, Mikulas Patocka wrote: You have small posibility that interrupt will eat up memory - interrupt in process that has PF_MEMALLOC. Patch: this is not the point of getblk, to fix the getblk deadlock the only way is to implement a fail path in each caller and allow getblk to return NULL (as every other memory allocation function can do). Failing getblk would likely introduce filesystem corruption. Look at getblk in 2.0 - when allocating new page fails it tries to reuse existing clean buffers or wakes up bdflush and waits until it writes them. This is the right thing to do. Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Tue, Dec 19, 2000 at 09:42:05AM +0100, Mikulas Patocka wrote: Failing getblk would likely introduce filesystem corruption. Look at getblk in 2.0 - when allocating new page fails it tries to reuse existing clean buffers or wakes up bdflush and waits until it writes them. This is the right thing to do. wrong. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Mon, Dec 18, 2000 at 10:57:44PM +0100, Mikulas Patocka wrote: > You have small posibility that interrupt will eat up memory - interrupt in > process that has PF_MEMALLOC. Patch: this is not the point of getblk, to fix the getblk deadlock the only way is to implement a fail path in each caller and allow getblk to return NULL (as every other memory allocation function can do). Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > Imagine that kpiod is slow. try_to_swap_out returns 1 and pretends it > > freed something but it didn't. It just passed request to kpiod. There are > > no pages to be freed by shrink_mmap. do_try_to_swap_out calls swap_out > > several times, then returns. And this repeats again and again. > > kpiod ceased to exist as of 2.2.19pre2 BTW. why didn't you fix SMP race in accessing pte? It's several months old and quite subtle bug. Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> Imagine that kpiod is slow. try_to_swap_out returns 1 and pretends it > freed something but it didn't. It just passed request to kpiod. There are > no pages to be freed by shrink_mmap. do_try_to_swap_out calls swap_out > several times, then returns. And this repeats again and again. kpiod ceased to exist as of 2.2.19pre2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Mon, 18 Dec 2000, Rik van Riel wrote: > On Mon, 18 Dec 2000, Andrea Arcangeli wrote: > > On Mon, Dec 18, 2000 at 06:29:24PM -0200, Rik van Riel wrote: > > > Wrong. Getblk won't deadlock, it will just sleep and another > > > > getblk will deadlock. > > OUCH. The only protection we have against this is the fact > that atomic allocations are not allowed to eat up all memory > in the system and that every thread can only have 1 getblk > operation at a time, right? You have small posibility that interrupt will eat up memory - interrupt in process that has PF_MEMALLOC. Patch: --- linux/mm/page_alloc.c_ Mon Dec 18 22:48:47 2000 +++ linux/mm/page_alloc.c Mon Dec 18 22:53:52 2000 @@ -516,7 +516,7 @@ /* XXX: is pages_min/4 a good amount to reserve for this? */ if (z->free_pages < z->pages_min / 4 && - !(current->flags & PF_MEMALLOC)) + (!(current->flags & PF_MEMALLOC) || in_interrupt())) continue; page = rmqueue(z, order); if (page) Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Mon, 18 Dec 2000, Rik van Riel wrote: > On Sat, 16 Dec 2000, Mikulas Patocka wrote: > > > > Not unless your driver is broken. > > > > ok_to_allocate: > > *** INTERRUPT > > spin_lock_irqsave(_alloc_lock, flags); > > /* if it's not a dma request, try non-dma first */ > > if (!(gfp_mask & __GFP_DMA)) > > RMQUEUE_TYPE(order, 0); > > RMQUEUE_TYPE(order, 1); > > spin_unlock_irqrestore(_alloc_lock, flags); > > > > nopage: > > return 0; > > } > > Now read the code carefully and see how allocations can > end up here ... and when they can't... GFP_ATOMIC allocations can eat all memory in 2.2. There are no free pages. Now process wants to allocate page with GFP_KERNEL or GFP_USER. It calls try_to_free_pages. try_to_free_pages succeeds and frees few pages. Interrupt is received and eats pages that were just freed. RMQUEUE fails. get_free_page returns zero. Process is shot. > > Deadlock in getblk, if memory is full of dirty file mapped pages. > > Wrong. Getblk won't deadlock, it will just sleep and another > thread will continue later on. Killing processes will (in 2.4) > only happen when you run out of swap ... 2.4 will simply have > its processes loop in alloc_pages() until memory is available. I'm talking about 2.2. getblk will sleep until some memory becomes available. And some memory can be available only if getblk succeeds. > > You actually do not need network flood to kill your box. Just imagine that > > kpiod is swapping files out too slowly, free memory is going lower and > > lower, every process screaming with "VM: do_try_to_free_pages failed" and > > the system is aproaching instant death. > > Umm? Can you explain how this could happen? Imagine that kpiod is slow. try_to_swap_out returns 1 and pretends it freed something but it didn't. It just passed request to kpiod. There are no pages to be freed by shrink_mmap. do_try_to_swap_out calls swap_out several times, then returns. And this repeats again and again. There are two possible ends: -swap_out unmaps everything and fails (all pages are pending on kpiod queue), try_to_free_pages fails, process is shot -as try_to_free_pages is pretending that it freed something, free memory is going lower and lower; finally it reaches zero - see above Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Mon, 18 Dec 2000, Andrea Arcangeli wrote: > On Mon, Dec 18, 2000 at 06:29:24PM -0200, Rik van Riel wrote: > > Wrong. Getblk won't deadlock, it will just sleep and another > > getblk will deadlock. OUCH. The only protection we have against this is the fact that atomic allocations are not allowed to eat up all memory in the system and that every thread can only have 1 getblk operation at a time, right? But even so, the deadlock is still theoretically possible and should probably be fixed, this sounds too much like a time bomb waiting to go off... ;( regards, Rik -- Hollywood goes for world dumbination, Trailer at 11. http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Mon, Dec 18, 2000 at 06:29:24PM -0200, Rik van Riel wrote: > Wrong. Getblk won't deadlock, it will just sleep and another getblk will deadlock. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > cat /mnt/www/www.kernel.org/index.html > > can you do ls /mnt/www/www.kernel.org/ as well? I'm interested, I came > to conclusion that web filesystem is not possible... (If you can't do Yes, if the server supports webDAV or something similar. > listings, it is not really filesystem; you could do > > cat /mnt/www/www.kernel.org_index.html as well, and that's easy to > do.) > > > and the CorbaFS userspace server takes care of loading the webpage and > > returning it to the kernel client. And these new filesystems don't > > take up any extra space in the kernel, since they all talk to the same > > CorbaFS kernel module! Not to mention being able to implement the > > filesystem in any language you like, debug the implementation in > > userspace, etc. > > codafs can do pretty much the same. Yes, but codaFS is specific to filesystems. kORBit, of course, can do much much more, in a very uniform way. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Sat, 16 Dec 2000, Mikulas Patocka wrote: > > Not unless your driver is broken. > > ok_to_allocate: > *** INTERRUPT > spin_lock_irqsave(_alloc_lock, flags); > /* if it's not a dma request, try non-dma first */ > if (!(gfp_mask & __GFP_DMA)) > RMQUEUE_TYPE(order, 0); > RMQUEUE_TYPE(order, 1); > spin_unlock_irqrestore(_alloc_lock, flags); > > nopage: > return 0; > } Now read the code carefully and see how allocations can end up here ... and when they can't... > When interrupt comes here and eats page just freed by try_to_free_pages(), > GFP_KERNEL allocation will fail => The kernel goes crazy, shoots > processes, returns -ENOMEM to calls, maybe damages its structures. > Deadlock in getblk, if memory is full of dirty file mapped pages. Wrong. Getblk won't deadlock, it will just sleep and another thread will continue later on. Killing processes will (in 2.4) only happen when you run out of swap ... 2.4 will simply have its processes loop in alloc_pages() until memory is available. The "maybe damages its structures" is a sure indication of you having a very vivid imagination ;) > You actually do not need network flood to kill your box. Just imagine that > kpiod is swapping files out too slowly, free memory is going lower and > lower, every process screaming with "VM: do_try_to_free_pages failed" and > the system is aproaching instant death. Umm? Can you explain how this could happen? > Besides, __get_free_pages just encourages people to write broken > drivers because it tries to hide allocation bugs. If there would > be something like > > if (current->flags & PF_MEMALLOC && !in_interrupt()) > panic("swapper fscked up!"); We have something a little bit like this in 2.4, though it's just a printk and it doesn't print a backtrace. Now that you mention it, though, printing a backtrace may be a nice debugging option for missed higher-order allocations ;) (but only useful if you get unreasonable amounts of them) regards, Rik -- Hollywood goes for world dumbination, Trailer at 11. http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi! > The cool thing is that the CorbaFS userspace server can implement any > kind of filesystem you want, as long as it follows the CorbaFS > interface! The current implementation exports the filesystem on the > host machine that it is running on, similar to NFS. But we also have > ideas for FTP or web filesystems, for example. Imagine being able to > mount the web CorbaFS onto /mnt/www and do a > > cat /mnt/www/www.kernel.org/index.html can you do ls /mnt/www/www.kernel.org/ as well? I'm interested, I came to conclusion that web filesystem is not possible... (If you can't do listings, it is not really filesystem; you could do cat /mnt/www/www.kernel.org_index.html as well, and that's easy to do.) > and the CorbaFS userspace server takes care of loading the webpage and > returning it to the kernel client. And these new filesystems don't > take up any extra space in the kernel, since they all talk to the same > CorbaFS kernel module! Not to mention being able to implement the > filesystem in any language you like, debug the implementation in > userspace, etc. codafs can do pretty much the same. Pavel -- I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Mikulas Patocka wrote: > > Oh, and try to eat atomic memory by ping -f kORBit-ized box. > > When linux is out of atomic memory, it will die anyway. Only if you subscribe to the "we don't need to handle exceptions or check return values" programmers guild...i.e. lazy error prone coders. I tend to run out of memory a lot and Linux is handling things pretty well. I must say, I haven't had a memory pressure kill my machine in oh...six months? ...when the new VM went in and there were bugs to be worked out. Programmers who write like this..or should I say "scripters"...are the bane of good clean code. They're the ones that write bloated apps, the ones where their cgi leaves 4,000,000 junk cookies behind...in one directory, that chews up 40 megs of memory in their .pl shopping cart so a customer can order one widget... -d begin:vcard n:Ford;David x-mozilla-html:TRUE url:www.blue-labs.org adr:;; version:2.1 email;internet:[EMAIL PROTECTED] title:Blue Labs Developer note;quoted-printable:GPG key: http:[EMAIL PROTECTED]=0D=0A x-mozilla-cpt:;9952 fn:David Ford end:vcard
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Mikulas Patocka wrote: Oh, and try to eat atomic memory by ping -f kORBit-ized box. When linux is out of atomic memory, it will die anyway. Only if you subscribe to the "we don't need to handle exceptions or check return values" programmers guild...i.e. lazy error prone coders. I tend to run out of memory a lot and Linux is handling things pretty well. I must say, I haven't had a memory pressure kill my machine in oh...six months? ...when the new VM went in and there were bugs to be worked out. Programmers who write like this..or should I say "scripters"...are the bane of good clean code. They're the ones that write bloated apps, the ones where their cgi leaves 4,000,000 junk cookies behind...in one directory, that chews up 40 megs of memory in their .pl shopping cart so a customer can order one widget... /rant -d begin:vcard n:Ford;David x-mozilla-html:TRUE url:www.blue-labs.org adr:;; version:2.1 email;internet:[EMAIL PROTECTED] title:Blue Labs Developer note;quoted-printable:GPG key: http:[EMAIL PROTECTED]=0D=0A x-mozilla-cpt:;9952 fn:David Ford end:vcard
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi! The cool thing is that the CorbaFS userspace server can implement any kind of filesystem you want, as long as it follows the CorbaFS interface! The current implementation exports the filesystem on the host machine that it is running on, similar to NFS. But we also have ideas for FTP or web filesystems, for example. Imagine being able to mount the web CorbaFS onto /mnt/www and do a cat /mnt/www/www.kernel.org/index.html can you do ls /mnt/www/www.kernel.org/ as well? I'm interested, I came to conclusion that web filesystem is not possible... (If you can't do listings, it is not really filesystem; you could do cat /mnt/www/www.kernel.org_index.html as well, and that's easy to do.) and the CorbaFS userspace server takes care of loading the webpage and returning it to the kernel client. And these new filesystems don't take up any extra space in the kernel, since they all talk to the same CorbaFS kernel module! Not to mention being able to implement the filesystem in any language you like, debug the implementation in userspace, etc. codafs can do pretty much the same. Pavel -- I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Sat, 16 Dec 2000, Mikulas Patocka wrote: Not unless your driver is broken. ok_to_allocate: *** INTERRUPT spin_lock_irqsave(page_alloc_lock, flags); /* if it's not a dma request, try non-dma first */ if (!(gfp_mask __GFP_DMA)) RMQUEUE_TYPE(order, 0); RMQUEUE_TYPE(order, 1); spin_unlock_irqrestore(page_alloc_lock, flags); nopage: return 0; } Now read the code carefully and see how allocations can end up here ... and when they can't... When interrupt comes here and eats page just freed by try_to_free_pages(), GFP_KERNEL allocation will fail = The kernel goes crazy, shoots processes, returns -ENOMEM to calls, maybe damages its structures. Deadlock in getblk, if memory is full of dirty file mapped pages. Wrong. Getblk won't deadlock, it will just sleep and another thread will continue later on. Killing processes will (in 2.4) only happen when you run out of swap ... 2.4 will simply have its processes loop in alloc_pages() until memory is available. The "maybe damages its structures" is a sure indication of you having a very vivid imagination ;) You actually do not need network flood to kill your box. Just imagine that kpiod is swapping files out too slowly, free memory is going lower and lower, every process screaming with "VM: do_try_to_free_pages failed" and the system is aproaching instant death. Umm? Can you explain how this could happen? Besides, __get_free_pages just encourages people to write broken drivers because it tries to hide allocation bugs. If there would be something like if (current-flags PF_MEMALLOC !in_interrupt()) panic("swapper fscked up!"); We have something a little bit like this in 2.4, though it's just a printk and it doesn't print a backtrace. Now that you mention it, though, printing a backtrace may be a nice debugging option for missed higher-order allocations ;) (but only useful if you get unreasonable amounts of them) regards, Rik -- Hollywood goes for world dumbination, Trailer at 11. http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
cat /mnt/www/www.kernel.org/index.html can you do ls /mnt/www/www.kernel.org/ as well? I'm interested, I came to conclusion that web filesystem is not possible... (If you can't do Yes, if the server supports webDAV or something similar. listings, it is not really filesystem; you could do cat /mnt/www/www.kernel.org_index.html as well, and that's easy to do.) and the CorbaFS userspace server takes care of loading the webpage and returning it to the kernel client. And these new filesystems don't take up any extra space in the kernel, since they all talk to the same CorbaFS kernel module! Not to mention being able to implement the filesystem in any language you like, debug the implementation in userspace, etc. codafs can do pretty much the same. Yes, but codaFS is specific to filesystems. kORBit, of course, can do much much more, in a very uniform way. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Mon, 18 Dec 2000, Andrea Arcangeli wrote: On Mon, Dec 18, 2000 at 06:29:24PM -0200, Rik van Riel wrote: Wrong. Getblk won't deadlock, it will just sleep and another getblk will deadlock. OUCH. The only protection we have against this is the fact that atomic allocations are not allowed to eat up all memory in the system and that every thread can only have 1 getblk operation at a time, right? But even so, the deadlock is still theoretically possible and should probably be fixed, this sounds too much like a time bomb waiting to go off... ;( regards, Rik -- Hollywood goes for world dumbination, Trailer at 11. http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Mon, 18 Dec 2000, Rik van Riel wrote: On Sat, 16 Dec 2000, Mikulas Patocka wrote: Not unless your driver is broken. ok_to_allocate: *** INTERRUPT spin_lock_irqsave(page_alloc_lock, flags); /* if it's not a dma request, try non-dma first */ if (!(gfp_mask __GFP_DMA)) RMQUEUE_TYPE(order, 0); RMQUEUE_TYPE(order, 1); spin_unlock_irqrestore(page_alloc_lock, flags); nopage: return 0; } Now read the code carefully and see how allocations can end up here ... and when they can't... GFP_ATOMIC allocations can eat all memory in 2.2. There are no free pages. Now process wants to allocate page with GFP_KERNEL or GFP_USER. It calls try_to_free_pages. try_to_free_pages succeeds and frees few pages. Interrupt is received and eats pages that were just freed. RMQUEUE fails. get_free_page returns zero. Process is shot. Deadlock in getblk, if memory is full of dirty file mapped pages. Wrong. Getblk won't deadlock, it will just sleep and another thread will continue later on. Killing processes will (in 2.4) only happen when you run out of swap ... 2.4 will simply have its processes loop in alloc_pages() until memory is available. I'm talking about 2.2. getblk will sleep until some memory becomes available. And some memory can be available only if getblk succeeds. You actually do not need network flood to kill your box. Just imagine that kpiod is swapping files out too slowly, free memory is going lower and lower, every process screaming with "VM: do_try_to_free_pages failed" and the system is aproaching instant death. Umm? Can you explain how this could happen? Imagine that kpiod is slow. try_to_swap_out returns 1 and pretends it freed something but it didn't. It just passed request to kpiod. There are no pages to be freed by shrink_mmap. do_try_to_swap_out calls swap_out several times, then returns. And this repeats again and again. There are two possible ends: -swap_out unmaps everything and fails (all pages are pending on kpiod queue), try_to_free_pages fails, process is shot -as try_to_free_pages is pretending that it freed something, free memory is going lower and lower; finally it reaches zero - see above Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Imagine that kpiod is slow. try_to_swap_out returns 1 and pretends it freed something but it didn't. It just passed request to kpiod. There are no pages to be freed by shrink_mmap. do_try_to_swap_out calls swap_out several times, then returns. And this repeats again and again. kpiod ceased to exist as of 2.2.19pre2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Imagine that kpiod is slow. try_to_swap_out returns 1 and pretends it freed something but it didn't. It just passed request to kpiod. There are no pages to be freed by shrink_mmap. do_try_to_swap_out calls swap_out several times, then returns. And this repeats again and again. kpiod ceased to exist as of 2.2.19pre2 BTW. why didn't you fix SMP race in accessing pte? It's several months old and quite subtle bug. Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Mon, Dec 18, 2000 at 10:57:44PM +0100, Mikulas Patocka wrote: You have small posibility that interrupt will eat up memory - interrupt in process that has PF_MEMALLOC. Patch: this is not the point of getblk, to fix the getblk deadlock the only way is to implement a fail path in each caller and allow getblk to return NULL (as every other memory allocation function can do). Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > I guess that when you mmap large files over nfs and write to them, you get > > similar problems. > > > > > Oh, and try to eat atomic memory by ping -f kORBit-ized box. > > > > When linux is out of atomic memory, it will die anyway. > > Not unless your driver is broken. ok_to_allocate: *** INTERRUPT spin_lock_irqsave(_alloc_lock, flags); /* if it's not a dma request, try non-dma first */ if (!(gfp_mask & __GFP_DMA)) RMQUEUE_TYPE(order, 0); RMQUEUE_TYPE(order, 1); spin_unlock_irqrestore(_alloc_lock, flags); nopage: return 0; } When interrupt comes here and eats page just freed by try_to_free_pages(), GFP_KERNEL allocation will fail => The kernel goes crazy, shoots processes, returns -ENOMEM to calls, maybe damages its structures. Deadlock in getblk, if memory is full of dirty file mapped pages. You actually do not need network flood to kill your box. Just imagine that kpiod is swapping files out too slowly, free memory is going lower and lower, every process screaming with "VM: do_try_to_free_pages failed" and the system is aproaching instant death. Besides, __get_free_pages just encourages people to write broken drivers because it tries to hide allocation bugs. If there would be something like if (current->flags & PF_MEMALLOC && !in_interrupt()) panic("swapper fscked up!"); you would see how many parts of system are "broken". Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> I guess that when you mmap large files over nfs and write to them, you get > similar problems. > > > Oh, and try to eat atomic memory by ping -f kORBit-ized box. > > When linux is out of atomic memory, it will die anyway. Not unless your driver is broken. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > > > For one of our demos, we ran a file server on a remote linux box (that we > > > > just had a user account on), mounted it on a kORBit'ized box, and ran > > > > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > > > > syscalls. If nothing else, it's pretty nifty what you can do in little > > > > code... > > > > > > Cool! > > > > > > However, can you do one test for me? Do _heavy_ writes on kORBit-ized > > > box. That might show you some problems. > > > > I guess that when you mmap large files over nfs and write to them, you get > > similar problems. > > > > > Oh, and try to eat atomic memory by ping -f kORBit-ized box. > > > > When linux is out of atomic memory, it will die anyway. > > Why should it die? Because it is written badly :-( > It is quite easy to make machine run out of atomic > memory: just bomb it with lots of packets. It should recover, eventually Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi! > > > For one of our demos, we ran a file server on a remote linux box (that we > > > just had a user account on), mounted it on a kORBit'ized box, and ran > > > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > > > syscalls. If nothing else, it's pretty nifty what you can do in little > > > code... > > > > Cool! > > > > However, can you do one test for me? Do _heavy_ writes on kORBit-ized > > box. That might show you some problems. > > I guess that when you mmap large files over nfs and write to them, you get > similar problems. > > > Oh, and try to eat atomic memory by ping -f kORBit-ized box. > > When linux is out of atomic memory, it will die anyway. Why should it die? It is quite easy to make machine run out of atomic memory: just bomb it with lots of packets. It should recover, eventually > > Mikulas > > -- The best software in life is free (not shareware)! Pavel GCM d? s-: !g p?:+ au- a--@ w+ v- C++@ UL+++ L++ N++ E++ W--- M- Y- R+ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi. > > For one of our demos, we ran a file server on a remote linux box (that we > > just had a user account on), mounted it on a kORBit'ized box, and ran > > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > > syscalls. If nothing else, it's pretty nifty what you can do in little > > code... > > Cool! > > However, can you do one test for me? Do _heavy_ writes on kORBit-ized > box. That might show you some problems. I guess that when you mmap large files over nfs and write to them, you get similar problems. > Oh, and try to eat atomic memory by ping -f kORBit-ized box. When linux is out of atomic memory, it will die anyway. Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > For one of our demos, we ran a file server on a remote linux box (that we > > just had a user account on), mounted it on a kORBit'ized box, and ran > > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > > syscalls. If nothing else, it's pretty nifty what you can do in little > > code... > > Cool! > > However, can you do one test for me? Do _heavy_ writes on kORBit-ized > box. That might show you some problems. Oh, and try to eat atomic > memory by ping -f kORBit-ized box. I'll give that a try when I get a chance. :) > I've always wanted to do this: redirect /dev/dsp from one machine to > another. (Like, I have development machine and old 386. I want all > programs on devel machine use soundcard from 386. Can you do that?) Yes. Definately. There are probably other ways of doing that... but one of the things we implemented was a "generic" character device... and we tested it by having a chardev server that basically reads from a "local" (to the server) character device, and forward it over CORBA. So this is already implemented! :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi! > > It was just an example. Basically, you'd be able to do in with just > > about any language that has ORBit bindings. > > > > Ben Ford wrote: > > > Why would you *ever* want to write a device driver in perl??? > > > > Precisely... but also, there could be a case where perl would make > sense. Consider an FTP filesystem. There performance is not dictated by > the speed of the language, it's limited by bandwidth. It could make sense > to write your almighty FTPfs like this: > > 1. Prototype it in Perl, get all the bugs out. > 2. Rewrite in C in userspace, get all the bugs out. > 3. recompile/relink in kernel space with no source modifications > 4. ship product. :) Bad example, as (with codafs), you can do this safely & nicely without korbit. See http://uservfs.sourceforge.net/ Pavel -- I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Pavel Machek wrote: > Hi! > > > For one of our demos, we ran a file server on a remote linux box (that we > > just had a user account on), mounted it on a kORBit'ized box, and ran > > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > > syscalls. If nothing else, it's pretty nifty what you can do in little > > code... > > Cool! > > However, can you do one test for me? Do _heavy_ writes on kORBit-ized > box. That might show you some problems. Oh, and try to eat atomic > memory by ping -f kORBit-ized box. > > I've always wanted to do this: redirect /dev/dsp from one machine to > another. (Like, I have development machine and old 386. I want all > programs on devel machine use soundcard from 386. Can you do that?) > > Pavel > In Plan9 you can. /joh - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi! > For one of our demos, we ran a file server on a remote linux box (that we > just had a user account on), mounted it on a kORBit'ized box, and ran > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > syscalls. If nothing else, it's pretty nifty what you can do in little > code... Cool! However, can you do one test for me? Do _heavy_ writes on kORBit-ized box. That might show you some problems. Oh, and try to eat atomic memory by ping -f kORBit-ized box. I've always wanted to do this: redirect /dev/dsp from one machine to another. (Like, I have development machine and old 386. I want all programs on devel machine use soundcard from 386. Can you do that?) Pavel -- I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi! For one of our demos, we ran a file server on a remote linux box (that we just had a user account on), mounted it on a kORBit'ized box, and ran programs on SPARC Solaris that accessed the kORBit'ized linux box's file syscalls. If nothing else, it's pretty nifty what you can do in little code... Cool! However, can you do one test for me? Do _heavy_ writes on kORBit-ized box. That might show you some problems. Oh, and try to eat atomic memory by ping -f kORBit-ized box. I've always wanted to do this: redirect /dev/dsp from one machine to another. (Like, I have development machine and old 386. I want all programs on devel machine use soundcard from 386. Can you do that?) Pavel -- I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Pavel Machek wrote: Hi! For one of our demos, we ran a file server on a remote linux box (that we just had a user account on), mounted it on a kORBit'ized box, and ran programs on SPARC Solaris that accessed the kORBit'ized linux box's file syscalls. If nothing else, it's pretty nifty what you can do in little code... Cool! However, can you do one test for me? Do _heavy_ writes on kORBit-ized box. That might show you some problems. Oh, and try to eat atomic memory by ping -f kORBit-ized box. I've always wanted to do this: redirect /dev/dsp from one machine to another. (Like, I have development machine and old 386. I want all programs on devel machine use soundcard from 386. Can you do that?) Pavel In Plan9 you can. /joh - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi! It was just an example. Basically, you'd be able to do in with just about any language that has ORBit bindings. Ben Ford wrote: Why would you *ever* want to write a device driver in perl??? Precisely... but also, there could be a case where perl would make sense. Consider an FTP filesystem. There performance is not dictated by the speed of the language, it's limited by bandwidth. It could make sense to write your almighty FTPfs like this: 1. Prototype it in Perl, get all the bugs out. 2. Rewrite in C in userspace, get all the bugs out. 3. recompile/relink in kernel space with no source modifications 4. ship product. :) Bad example, as (with codafs), you can do this safely nicely without korbit. See http://uservfs.sourceforge.net/ Pavel -- I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
For one of our demos, we ran a file server on a remote linux box (that we just had a user account on), mounted it on a kORBit'ized box, and ran programs on SPARC Solaris that accessed the kORBit'ized linux box's file syscalls. If nothing else, it's pretty nifty what you can do in little code... Cool! However, can you do one test for me? Do _heavy_ writes on kORBit-ized box. That might show you some problems. Oh, and try to eat atomic memory by ping -f kORBit-ized box. I'll give that a try when I get a chance. :) I've always wanted to do this: redirect /dev/dsp from one machine to another. (Like, I have development machine and old 386. I want all programs on devel machine use soundcard from 386. Can you do that?) Yes. Definately. There are probably other ways of doing that... but one of the things we implemented was a "generic" character device... and we tested it by having a chardev server that basically reads from a "local" (to the server) character device, and forward it over CORBA. So this is already implemented! :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi. For one of our demos, we ran a file server on a remote linux box (that we just had a user account on), mounted it on a kORBit'ized box, and ran programs on SPARC Solaris that accessed the kORBit'ized linux box's file syscalls. If nothing else, it's pretty nifty what you can do in little code... Cool! However, can you do one test for me? Do _heavy_ writes on kORBit-ized box. That might show you some problems. I guess that when you mmap large files over nfs and write to them, you get similar problems. Oh, and try to eat atomic memory by ping -f kORBit-ized box. When linux is out of atomic memory, it will die anyway. Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Hi! For one of our demos, we ran a file server on a remote linux box (that we just had a user account on), mounted it on a kORBit'ized box, and ran programs on SPARC Solaris that accessed the kORBit'ized linux box's file syscalls. If nothing else, it's pretty nifty what you can do in little code... Cool! However, can you do one test for me? Do _heavy_ writes on kORBit-ized box. That might show you some problems. I guess that when you mmap large files over nfs and write to them, you get similar problems. Oh, and try to eat atomic memory by ping -f kORBit-ized box. When linux is out of atomic memory, it will die anyway. Why should it die? It is quite easy to make machine run out of atomic memory: just bomb it with lots of packets. It should recover, eventually Mikulas -- The best software in life is free (not shareware)! Pavel GCM d? s-: !g p?:+ au- a--@ w+ v- C++@ UL+++ L++ N++ E++ W--- M- Y- R+ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
I guess that when you mmap large files over nfs and write to them, you get similar problems. Oh, and try to eat atomic memory by ping -f kORBit-ized box. When linux is out of atomic memory, it will die anyway. Not unless your driver is broken. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
I guess that when you mmap large files over nfs and write to them, you get similar problems. Oh, and try to eat atomic memory by ping -f kORBit-ized box. When linux is out of atomic memory, it will die anyway. Not unless your driver is broken. ok_to_allocate: *** INTERRUPT spin_lock_irqsave(page_alloc_lock, flags); /* if it's not a dma request, try non-dma first */ if (!(gfp_mask __GFP_DMA)) RMQUEUE_TYPE(order, 0); RMQUEUE_TYPE(order, 1); spin_unlock_irqrestore(page_alloc_lock, flags); nopage: return 0; } When interrupt comes here and eats page just freed by try_to_free_pages(), GFP_KERNEL allocation will fail = The kernel goes crazy, shoots processes, returns -ENOMEM to calls, maybe damages its structures. Deadlock in getblk, if memory is full of dirty file mapped pages. You actually do not need network flood to kill your box. Just imagine that kpiod is swapping files out too slowly, free memory is going lower and lower, every process screaming with "VM: do_try_to_free_pages failed" and the system is aproaching instant death. Besides, __get_free_pages just encourages people to write broken drivers because it tries to hide allocation bugs. If there would be something like if (current-flags PF_MEMALLOC !in_interrupt()) panic("swapper fscked up!"); you would see how many parts of system are "broken". Mikulas - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit (and ioctl must die!)
Alexander Viro <[EMAIL PROTECTED]> writes: > ioctl() is avoidable. Proof: Plan 9. They don't _have_ that system call. > It doesn't mean that we should (or could) remove it. It _does_ mean that > new APIs do not need it. *I* sure wish we could. From the standpoint of trying to trace system calls, it's a big stinking black hole. All of the other syscalls (I think) have pretty well defined semantics in terms of what they do to a process' memory space, but the semantics of ioctl are "may read or write any memory whatsoever, and if you want to know what, well, it sucks to be you". Even NT does this better, if I'm interpreting this correctly: http://msdn.microsoft.com/library/psdk/winbase/devio_9quk.htm --Mike -- [O]ne of the features of the Internet [...] is that small groups of people can greatly disturb large organizations. --Charles C. Mann - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Thu, 14 Dec 2000, Chris Lattner wrote: > > Yes, I did. What I don't understand is how kernel mechanism for marshalling > > would make your life easier wrt changes. > > I gave a very simple example of how an interface could be designed and > then later extended without breaking any user space programs. Please > reread the example and tell me precisely what doesn't make sense, and I > would be glad to explain it better. What does it win compared to adding new file and allowing new programs use it while the old ones keep using the old one? That's precisely the same situation, isn't it? > > So what's to stop you from letting readdir() do the work? WTF do we need > > one more API for returning the list of objects? > > Urm.. that was the point. You don't add a new API (it's built into corba, > along with introspection). Also, like I JUST MENTIONED, readdir will have > no way of knowing what is a network device in the quagmire of stuff > floating around in /dev. (ls *net* does actually call readdir ya > know. Perhaps you should read up more on unix... [sorry couldn't help > it. ;]) Cool. We don't have network devices represented as fs. We could have, but nobody had really cared. Guess what, I know that. It's _not_ a good thing. What you propose is expanding the area of brokenness. > structure of. Imaging this. You have a bytestream flowing out of some > pipe/device/socket/file/whatever and you pipe it to a > "Decode" program. The decode program spits this out: Is it separated on per-object streams? > Heh... I'm not trying to replace od or cat or tee or pipes. What I'm > trying to do is give you a larger toolkit that gives you MORE > power. Imagine you want to get the first float value (of each struct) of > the above stream. Imagine a functional scripting language where you could > do this (haskell syntax stolen gratitously): > > cat /dev/floatymouse | funcwith 'map fst' No problems with that. But notice that you did introduce a new device here. > > No, it isn't. The difference being: letting driver to define a filesystem > > and mounting/exporting it/whatever means that you get to use _all_ _normal_ > > _data-agnostic_ _tools_. And believe me, it matters. If you don't understand > > that - no offense, but you don't understand UNIX. > > Ouch. > > /me digs around and puts on thick skin > > Okay, fine. You want to talk unix. You proclaim that this beautiful > infalliable API is simple open/close/read/write calls on stanardized > devices, right? What about ioctl? What about the HUGE class of crap that ioctl() is avoidable. Proof: Plan 9. They don't _have_ that system call. It doesn't mean that we should (or could) remove it. It _does_ mean that new APIs do not need it. And I'm quite serious. ioctl() is used in too many programs to kill it off (compatibility reasons). No arguments here. However, looking back at these cases... All of them could be done without ioctl(). Moreover, it had been done. Says something about the need of adding new ioctls, doesn't it? Instead of having a metric buttload of device nodes in /dev you have drivers export filesystems. You can mount them anywhere you want. These filesystems can contain both devices and control files. Writing commands to control file gives you a replacement of ioctl(). It actually works quite nicely. As long as you separate the data streams for different objects and give them some names, so they can be intercepted, etc. - fine with me. There's nothing wrong with having a library for marshalling. There's nothing wrong with having it in the kernel too, BTW. However, _that_ is completely independent from any form of transport. All you need is "export a piece of namespace to another box". - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> But alan, that's the beautiful thing. Given a CORBA object, you can > understand its structure without knowing exactly what the contents > are. You can effectively derive it's prototype just by inspecting it. Oh dear this isnt going in is it. Look I know the prototype of every single lisp transaction - its a list. Period. I know how to manipulate any data I have and print it, because its a list. I know how to extend it, because its .. a list. If I need to put content identification in, well guess what - thats a list ((my_name "Hello") (his_name "Foo")) and XML is simply lisp done wrong. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > > Oh, great. So we don't have to care about formatting changes. We just > > > have to care about the data changes. IOW, we are shielded from the > > > results of changes that should never happen in the first place. And the > > > benefit being...? > > > > What the hell are you talking about? Did you even read my example? I was > > giving an example of extending an API, adding new functionality to it. > > Yes, I did. What I don't understand is how kernel mechanism for marshalling > would make your life easier wrt changes. I gave a very simple example of how an interface could be designed and then later extended without breaking any user space programs. Please reread the example and tell me precisely what doesn't make sense, and I would be glad to explain it better. > > > Notice also that I can say ls /dev/*mouse* and get some idea of the files > > > there. I can't do that for your interfaces. > > And you know what? I can't do ls /dev/*net* and get all the network > > devices either. Actually, one of the very cool things about CORBA is that > > you can BROWSE/LIST/SEARCH all objects currently instantiated. Being > > able to browse all in kernel objects would be very cool. > So what's to stop you from letting readdir() do the work? WTF do we need > one more API for returning the list of objects? Urm.. that was the point. You don't add a new API (it's built into corba, along with introspection). Also, like I JUST MENTIONED, readdir will have no way of knowing what is a network device in the quagmire of stuff floating around in /dev. (ls *net* does actually call readdir ya know. Perhaps you should read up more on unix... [sorry couldn't help it. ;]) > > > The point being: if your program spends efforts on marshalling it would > > > better _do_ something with the obtained data. And then we are back to > > > the square 1. > > Uh huh, go ahead and read the example I sent to you. > Did that. So they you understand that no interface was broken even though more data is now available... > > > Returning to your example, I could not tee(1) the stream into file for > > > later analysis. Not unless I write a special-case program for intercepting > > > that stuff. I don't see why it is a good thing. > > On the contrary, it would be pretty easy to do something like that with > > CORBA. No you wouldn't be able to use tee, but why would you want to tee > > a binary data file? The only reason that tee works in this situation is > For the same reasons why I use tar, gzip, whatever. I don't _want_ to > invent a new utility every time when Joe Doe adds a new piece of interface. > Data is data is data. I can uuencode it and send to somebody who would > care to analize the bloody thing. Do you mean that I need to write a > special tool for that? For _every_ member of every interface somebody > decided to add? I don't think so. I really don't. That's fine, I don't blame you. Ya know what? I even agree with you (image that? hehe :). You seem to forget that a CORBA object is exactly the same thing: a stream of bytes. Actually it does better than that. It's a stream of bytes that you can introspect on and determine the structure of. Imaging this. You have a bytestream flowing out of some pipe/device/socket/file/whatever and you pipe it to a "Decode" program. The decode program spits this out: sequence < struct { float float }> This example is meant to illustrate that _in a general way_ you can decode the structure of the bytestream. (hence my claims about CORBA adding structure to a standard bytestream)... this is quite a bit more useful than trying to figure out what 'cat' is trying to tell you about binary data (especially if it's floating point numbers). > > line... your analysis program would have to have special purpose code to > > parse the file. EVERY consumer of "mouse data" would have to parse the > Or I would look at the size. Or I would say od and look at the result. Don't > tell me what to do with the data, when I'll need to parse it I will. And > if you expect me to bother with writing more stuff when generic tools would > work fine - too bad, I've been there, done that and I'm not coming back. Heh... I'm not trying to replace od or cat or tee or pipes. What I'm trying to do is give you a larger toolkit that gives you MORE power. Imagine you want to get the first float value (of each struct) of the above stream. Imagine a functional scripting language where you could do this (haskell syntax stolen gratitously): cat /dev/floatymouse | funcwith 'map fst' Which you could then point to the "decode" type program mentioned above, and you would now get: sequence > BTW, you cared about size of /dev, didn't you? /usr/bin choke-full of tee-foo, > tee-bar, yodda, yodda would be better? You wouldn't have to replace the standard utilities. Sorry to rain on your parade. > > file. That seems pretty silly to me. > Difference between good program and bad one: the
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > > There is a large perception of CORBA being slow, but for the most part it > > > is unjustified. > > Really? I have that same perception but I can't claim that I've measured it. > On the other hand, I have measured the overhead of straight UDP, TCP, and > Sun RPC ping/pong tests and you can find the code for that in any version > of lmbench. It should be a 5 minute task for someone who groks corba to > do the same thing using the same framework. If someone wants to do it, > I'll guide them through the lmbench stuff. It's pretty trivial, start Urm... thanks for the offer... but you misunderstand me if you think that I'm claiming that kORBit is the ideal/fast implementation that everyone has been looking for. There is still much to be done. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > Of course. Which is why CORBA is about putting STRUCTURE in that stream > > of random bytes coming over the wire. Why should I have to rewrite my > > marshalling and demarshalling code every time I want to write a > > server. read and write are fine. But sometimes I want a > > structure. Sometimes, my structures aren't laid out like C struct's > > either. What then? What if I want to send an "object" to you? > > Then I need to understand the object anyway. And Corba objects are horribly > over complex. Any lisp hacker will tell you there is only one type: a list. But alan, that's the beautiful thing. Given a CORBA object, you can understand its structure without knowing exactly what the contents are. You can effectively derive it's prototype just by inspecting it. The only difference between lisp and a CORBA object in this respect is that each item in the list is typed so that you have even more info about what to do with it. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
Fredrik Vraalsen wrote: > The cool thing is that the CorbaFS userspace server can implement any > kind of filesystem you want, as long as it follows the CorbaFS > interface! Sorry, it's yet another one. Or does it do something different? (YAO hasn't stopped me working on userspace filesystems either :-) > The current implementation exports the filesystem on the > host machine that it is running on, similar to NFS. But we also have > ideas for FTP or web filesystems, for example. Imagine being able to > mount the web CorbaFS onto /mnt/www and do a > > cat /mnt/www/www.kernel.org/index.html > > and the CorbaFS userspace server takes care of loading the webpage and > returning it to the kernel client. And these new filesystems don't > take up any extra space in the kernel, since they all talk to the same > CorbaFS kernel module! Not to mention being able to implement the > filesystem in any language you like, debug the implementation in > userspace, etc. A bit like CodaFS and Perlfs. Except, being CORBA, you can run the userspace server remotely? ;-) -- Jamie - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > There is a large perception of CORBA being slow, but for the most part it > > is unjustified. I believe that the act of _designing_ a completely new > CORBA is slow compared to some of the other solutions. The question I was > trying to ask is whether you should put something smaller and faster into the > kernel space and leave CORBA in userland. What I'm trying to show is that CORBA itself is not slow. CORBA can be thought of as an IDEA, and the current implementations are suboptimal. For example, no CORBA implementation has been tweaked to run well in-kernel. Because of this, people will say that CORBA sucks and is slow, but that's not true. I will agree that IIOP (the standardized corba communication mechanism) is much slower and more complex than we would want for a generalized user->kernel communication mechanism. The nice thing about CORBA is that you can design your own transport to optimize things. Thus 99% of the time you could use a heavily optimized, very simple transport that doesn't do much. This is what you are asking for, and it make perfect sense. What CORBA buys you is the ability to use this streamlined interface without giving up full generality... > It's complex, it has security > implications surely it belongs talking something simple and fast to the kernel. First off, there are a lot of complex systems in the kernel. :) Second, there are not security implications above and beyond any normal kernel hacking. The reason that kORBit has security implications right now is that there _IS NO SECURITY_. Imagine implementing sys_read with no checks of what uid is running, and you get the idea. Actually security in CORBA can be done BETTER in kernel space than in user space, but I digress... > If you look at microkernels they talk a much simpler faster rpc protocol. Yes but for the most part they lose all of the advantages of CORBA as well. For example, every microkernel that I'm aware of can only talk to remote servers/clients that are running the same microkernel on the remote machine. There are no provisions for byte swapping if the endianness is incorrect or for having structured bytestreams (so that you can tell things about the raw bytes coming over the line without understanding them all). -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On 14 Dec 2000, Michael Livshin wrote: > this might be because the Bell Labs folks don't like RPC in general > when network latencies become involved? I'm guessing. > > 'cause CORBA is still pretty much objectified RPC, as far as I know. > I don't think you want to abstract the network out just like that when > dealing with kernels. OK, since everyone seems to want to argue about this on orbit-list for some reason, my $0.02: CORBA/IIOP is saner than SunRPC in a lot of ways, and it's not too horribly more complicated to implement a barebones ORB than a barebones SunRPC impl. Comments about CORBA being too slow are nonsense - it's just that a few crackheads have tried to stick ORBit in the kernel as a global IPC mechanism, which it really is not suitable for. CORBA has its place, and a GIOP mapping to a kernel-friendly IPC mechanism (instead of TCP/IP) would certainly make it more useful in the kernel, but generic mechanisms such as CORBA cannot by definition be as fast as IPC mechanisms optimized for a specific task. People like Al Viro, who haven't written GNOME programs of any size and certainly don't have mounds of in-depth knowledge of it, should probably shut up about about its API & design. :) TTFN, -- Elliot No new ideas for my .sig, and Alan told me my old one was an urban myth, so just ignore this. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
* Rik van Riel | | On Wed, 13 Dec 2000, Chris Lattner wrote: | | > 1. kORBit adds about 150k of code to the 2.4t10 kernel. | > 2. kNFS adds about 100k of code to the 2.4t10 kernel. | > 3. kORBit can do everything kNFS does, plus a WHOLE lot more: For example | >implement an NFS like server that uses SSL to send files and | >requests... so it is really actually "secure". | | So can you implement a kNFS server in kORBit that takes | less than 50kB of RAM? Otherwise it's still a contributor | to bloat and this argument won't work ;) Well, kORBit itself is bigger than kNFS, but like Chris said, it can do much more. :) For testing kORBit we wrote a new filesystem CorbaFS. The CorbaFS client is a kernel module that basically forwards the Linux VFS calls to the userspace CorbaFS server. According to lsmod, the CorbaFS module takes up only 11KB of RAM. Keep in mind that CorbaFS is currently only a proof-of-concept of the kernel calling into userspace through kORBit (it is a read-only filesystem at the moment, for example). But it should give you some idea. The cool thing is that the CorbaFS userspace server can implement any kind of filesystem you want, as long as it follows the CorbaFS interface! The current implementation exports the filesystem on the host machine that it is running on, similar to NFS. But we also have ideas for FTP or web filesystems, for example. Imagine being able to mount the web CorbaFS onto /mnt/www and do a cat /mnt/www/www.kernel.org/index.html and the CorbaFS userspace server takes care of loading the webpage and returning it to the kernel client. And these new filesystems don't take up any extra space in the kernel, since they all talk to the same CorbaFS kernel module! Not to mention being able to implement the filesystem in any language you like, debug the implementation in userspace, etc. -- Fredrik Vraalsen - Research Assistant, Pablo research group Department of Computer Science, University of Illinois at U-C - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> >that is not transparently hidden to the user. Why can't I just open > >/dev/net0 and get the first network device? Because we have so many > >inconsistent, poorly design, inextensible interfaces laying around, thats > >why. > This is a bad example, but a (perhaps?) good point. It seems it should be > possible to implement an unlimited number of TCP/IP devices, each Part of my argument is that although every kernel interface boils down to something as simple as ioctl (where you have lots of api's multiplexed onto one data sink, int 0x80 is another example, kORBit is another), some ways of doing this multiplexing are better than others. I look at it as a hierarchy: 1. CORBA/kORBit 2. int 80/lcall 3. ioctl Each level is more structured than those below it. Each interface is also "cleaner" than those below it. I don't think that anyone would argue that we should replace int 80 and friends with an interface in the spirit of ioctl (even though it would be functionally identical). > representing a connection, and each connected, disconnected, etc. by > ioctls... Of course, I could be way wrong about this ioctl's... they come up again. Everytime that I look at the beautiful, clean, abstract interface that Unix exposes... it makes me happy. But unfortunately, if you look closer, you realize that the Unix API isn't really clean or nice at all... there is this dumping ground for odds and ends that don't fit into the standard model. So when you show someone the standard model, it looks clean and pretty... but someone always forgets to mention ioctl. I would claim that ioctl is one of the biggest reasons that kORBit needs to exist. See below. > >I can't do ls /dev/*net* and get all the network > >devices either. Actually, one of the very cool things about CORBA is that > Different network devices are _very_ different... You can't broadcast on a > modem, for instance, nor can you use promiscuous mode on a modem... You > also can't do byte-by-byte raw mode on an ethernet... Precisely. But they do all have very common interfaces. For example, the standard network interface would probably have "transmit/receive block" and "getstatus" commands. This does not mean that they cannot implement other interfaces, however. Imagine this situation for writing your packet sniffer: 1. You open up /dev/net0 and get the Network interface 2. You query the network interface to see if it has an implementation of the promiscuous interface. 3. If not, you bail, because it doesn't support it. 4. If, you go ahead and use it. Contrast that to: 1. Survey all the different interfaces that are known (at development time of course) to have promiscuous interfaces. 2. Find out which ones are active. 3. Depending on what kind of interface it is, load a library that can understand the byte format comming off the device. 4. Use that library. The problems with the second (currently used) approach is that the APPLICATION writer has to keep up with new hardware developments and new interfaces. They have to understand and code parsers for the bytestreams coming from each device, which (as you mentioned) are all different, because they all support different (although overlapping) extensions. All of these calls that currently get dumped into ioctl (because they are not important enough to warrant an API function at the top level) suddenly become well structure and well designed interfaces. For example, if your device doesn't support the terminal handling ioctls the system has to basically figure that out and report errors on it. With CORBA/kORBit, you would just not implement that interface... so if someone asked for it, they would get a null pointer to the interface, which they check for and realize that it's impossible to do terminal stuff on. isatty (an example of an IOCTL wrapper) suddenly becomes implicit and the API is cleaner... I have no problem with the "unix way", but I do think that it can be augmented in some ways... -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > 1. kORBit adds about 150k of code to the 2.4t10 kernel. > > 2. kNFS adds about 100k of code to the 2.4t10 kernel. > So can you implement a kNFS server in kORBit that takes > less than 50kB of RAM? Otherwise it's still a contributor > to bloat and this argument won't work ;) Actually the kORBitNFS server would have to take -50K of code to break even. :) The point was that kORBit lets you do a lot more... so hopefully that 50k of generality gives you something. :) > I guess it's time to stop the flaming and to see what can > be achieved using kORBit. The people who favour kORBit should > IMHO be left alone and given the opportunity to show what can > be achieved with kORBit ... if they don't achieve anything, > the nay-sayers can always claim their "victory"; if something > useful comes out the kORBit people can claim usefulness. Agreed! -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Wed, 13 Dec 2000, Chris Lattner wrote: > 1. kORBit adds about 150k of code to the 2.4t10 kernel. > 2. kNFS adds about 100k of code to the 2.4t10 kernel. > 3. kORBit can do everything kNFS does, plus a WHOLE lot more: For example >implement an NFS like server that uses SSL to send files and >requests... so it is really actually "secure". So can you implement a kNFS server in kORBit that takes less than 50kB of RAM? Otherwise it's still a contributor to bloat and this argument won't work ;) I guess it's time to stop the flaming and to see what can be achieved using kORBit. The people who favour kORBit should IMHO be left alone and given the opportunity to show what can be achieved with kORBit ... if they don't achieve anything, the nay-sayers can always claim their "victory"; if something useful comes out the kORBit people can claim usefulness. regards, Rik -- Hollywood goes for world dumbination, Trailer at 11. http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Alexander Viro wrote: > > Err... how about this: Give me two or three kORBit syscalls and I can get > > rid of all the other 100+ syscalls! :) > > Like it ioctl() does it? Number of entry points is _not_ an issue. Diversity > of the API is. Technically, kernel has 1 (_o_n_e_) entry point as far as > userland is concerned. int 0x80 on x86. Can't beat that, can you? If there's one thing that could be nicer, ioctl() would be it. ioctl() works ok in C and terribly in everything else. CORBA works well in scripting languages and Java, and sucks in C compared with a simple ioctl(). Is there a way to marry the best sides of each? Is 9P that way (I don't know much about it)? -- Jamie - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
I surely hope that this thread wont end here. It's extremely important to look at this since we're heading towards distributed resources, where humans dont work on a server but towards other people through servers. /josef - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Chris Lattner <[EMAIL PROTECTED]> writes: > > p9fs exists. I didn't see these patches since August, but probably I can poke > > Roman into porting it to the current tree. 9P is quite simple and unlike > > CORBA it had been designed for taking kernel stuff to userland. Besides, > > authors definitely understand UNIX... > > One thing that you might want to mention Alexander: 9P is not a general > communications protocol. In fact, it doesn't work very well across the > internet at all. To get decent performance, the Plan9 group (which, is a > very cool group. :) has to specify a new protocol that competes with TCP > on the level of complexity (IL: http://plan9.bell-labs.com/sys/doc/il/il.html) this might be because the Bell Labs folks don't like RPC in general when network latencies become involved? I'm guessing. 'cause CORBA is still pretty much objectified RPC, as far as I know. I don't think you want to abstract the network out just like that when dealing with kernels. -- Entropy isn't what it used to be. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
[Alan DID not say this:] > > There is a large perception of CORBA being slow, but for the most part it > > is unjustified. Really? I have that same perception but I can't claim that I've measured it. On the other hand, I have measured the overhead of straight UDP, TCP, and Sun RPC ping/pong tests and you can find the code for that in any version of lmbench. It should be a 5 minute task for someone who groks corba to do the same thing using the same framework. If someone wants to do it, I'll guide them through the lmbench stuff. It's pretty trivial, start with this as a guide: /* * tcp_xact.c - simple TCP transaction latency test * * Three programs in one - * server usage: tcp_xact -s * client usage: tcp_xact hostname * shutdown: tcp_xact -hostname * * Copyright (c) 1994 Larry McVoy. Distributed under the FSF GPL with * additional restriction that results may published only if * (1) the benchmark is unmodified, and * (2) the version in the sccsid below is included in the report. * Support for this development by Sun Microsystems is gratefully acknowledged. */ char*id = "$Id$\n"; #include "bench.h" voidclient_main(int ac, char **av); voiddoserver(int sock); voiddoclient(int sock); voidserver_main(int ac, char **av); voiddoserver(int sock); int main(int ac, char **av) { if (ac != 2) { fprintf(stderr, "Usage: %s -s OR %s [-]serverhost\n", av[0], av[0]); exit(1); } if (!strcmp(av[1], "-s")) { if (fork() == 0) { server_main(ac, av); } exit(0); } else { client_main(ac, av); } return(0); } void client_main(int ac, char **av) { int sock; char*server; charbuf[100]; if (ac != 2) { fprintf(stderr, "usage: %s host\n", av[0]); exit(1); } server = av[1][0] == '-' ? [1][1] : av[1]; sock = tcp_connect(server, TCP_XACT, SOCKOPT_NONE); /* * Stop server code. */ if (av[1][0] == '-') { close(sock); exit(0); } BENCH(doclient(sock), MEDIUM); sprintf(buf, "TCP latency using %s", av[1]); micro(buf, get_n()); exit(0); /* NOTREACHED */ } void doclient(int sock) { charc; write(sock, , 1); read(sock, , 1); } void child() { wait(0); signal(SIGCHLD, child); } void server_main(int ac, char **av) { int newsock, sock; if (ac != 2) { fprintf(stderr, "usage: %s -s\n", av[0]); exit(1); } GO_AWAY; signal(SIGCHLD, child); sock = tcp_server(TCP_XACT, SOCKOPT_NONE); for (;;) { newsock = tcp_accept(sock, SOCKOPT_NONE); switch (fork()) { case -1: perror("fork"); break; case 0: doserver(newsock); exit(0); default: close(newsock); break; } } /* NOTREACHED */ } void doserver(int sock) { charc; int n = 0; while (read(sock, , 1) == 1) { write(sock, , 1); n++; } /* * A connection with no data means shut down. */ if (n == 0) { tcp_done(TCP_XACT); kill(getppid(), SIGTERM); exit(0); } } -- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> There is a large perception of CORBA being slow, but for the most part it > is unjustified. I believe that the act of _designing_ a completely new > protocol, standardizing it, and making it actually work would be a huge > process that would basically reinvent CORBA (obviously some of the design > decisions could be made differently, but all the same issues would have be > dealt with). CORBA is slow compared to some of the other solutions. The question I was trying to ask is whether you should put something smaller and faster into the kernel space and leave CORBA in userland. It's complex, it has security implications surely it belongs talking something simple and fast to the kernel. If you look at microkernels they talk a much simpler faster rpc protocol. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Thu, Dec 14, 2000 at 10:23:15AM +0100, josef höök wrote: > Plan9 aint unix/posix though it has its Ape environment. > What we do need to look at is a good implementation for distributed resources. > The ideal would bee getting 9P and IL into linux. Think of having thousand of small > > linux boxes dedicated to either run as a CPU server or a Fileserver or whatever > service there is. In the who cares department: this idea was one of the main reasons I founded BitMover, it's what we wanted to do before we got sidetracked by BitKeeper. So you have my vote, and if anyone does this, I'll be pissed as hell because I wanted to do it, but happy that it's getting done. If you look at lmbench, it's basically designed to measure all the crud that you would need to make this sort of thing go fast. So I've been thinking about this for at least 6 years. Sigh. -- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Alexander Viro spake thus: > Maybe... I definitely agree that 14 is below the limit, but 30... Hell knows, > from what I see on the box I'm using right now it seems to fall into several > cathegories: > * Very-Long-And-Verbose-Named-HOWTO.html > * manpages for X and Tcl functions with obscenely long names > * *.deb and corresponding *.diff.gz and *.dsc > * var/state/apt/lists/* > * ghostscript maps * Metallica - Master of Puppets.mp3 Oystein ;) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Chris Lattner wrote: > > > Err shame on you, don't forget about lcall and exceptions, and interrupts, > > > and... That is technically more than _o_n_e_ "entry point". :) Oh wait, > > > what about sysenter/exit too? :) > > OK, you got me on lcall (however, that's iBCS-only, IIRC), but the rest... > > what the hell does userland to interrupts? OK, make it 2 - pagefault > > can be arguably used in that way. > > The reason that I considered exceptions and interrupts is that often, > exceptions get reflected as signals to the running processes (SIGSEGV, > SIGFPE, SIGILL, others?), and interrupts can wake up processes (from > sys_poll among others). I was considering more of the user->kernel and > kernel->user transitions... anyways, that's really besides the point. :) > > > > this wonderful design we get all kinds of stuff like sys_oldumount vs > > > sys_umount and others... > > Check how often anything uses the majority of that stuff... > > Correct, it's for backwards compatibility with old programs (for example > libc5 uses a lot of those "old" syscalls). > > > > > Yes, standard RPC mechanism would be nice. No, CORBA is not a good > > candidate - > > too baroque and actually known to lead to extremely > > tasteless APIs being > > implemented over it. Yes, I mean GNOME. So > > > Check 9P and compare. Really. Section 5 of Plan 9 manpages. Available on > > plan-9.bell-labs.com/sys/man/ > > That's fine. Since the server is down (or the URL is bad), can you please > give me an example of how 9P is better than CORBA? I freely admit to not > knowing much about 9P... how much do you know about CORBA (aside from > your opinion that GNOME uses it, and therefore it is bad. ;)? > > > > without breaking backwards compatibility). Please don't tell me that OOP > > > is bad... or else we will have the eviscerate the VFS layer from the > > > kernel (amount other subsystems)... :) > > > OOP is a nice tool. However, it's a tool that has incredible potential of > > shooting one's foot. It's wonderful if you have sane set of methods. And > > that's a _big_ if. "Easily extensible" is not an absolutely good thing - > > C++ wankers all over the world are busily proving it every day. Heck, they > > make a living out of that. IOW, the problem with interface changes is _not_ > > in converting the old code. It's in choosing the right changes. And that > > part of the game can't be simplified. > > Oif. That's like telling someone that C is evil because it has for loops, > and for loops can be used to write nasty code. "just write in > assembler" he says. :) I would claim that someone could write a bad > program (or shoot themselves in the foot) with any turing complete > language. C++ definately give you more rope to do that with, but used > wisely, it can also be nice. The trick is to just not have to work with > other peoples C++ code. :) Hey, did I mention that kORBit and all its > extensions are written in C? :) > > > > Like I mentioned in a previous email, CORBA does not preclude 9P. What > > > it does buy you though, is compatibility with LOTS of preexisting CORBA > > > tools. How much development infrastructure is there for 9P? I thought > > > so. :) > > > All UNIX userland on the client side. lib9p on the server side (23Kb of sparse > > C). Examples of use in servers - see the aforementioned site. > > Err... yeah, so you're effectively mapping UNIX/POSIX across 9P. That's > not very creative, and you could do the same thing with CORBA. I ask > again, "How much development infrastructure is there for 9P?". If you say > "just use unix", then what is the point of 9P at all? (on linux). Linux > already has most of posix (and some would claim all of the "good > stuff" in posix.). Plan9 aint unix/posix though it has its Ape environment. What we do need to look at is a good implementation for distributed resources. The ideal would bee getting 9P and IL into linux. Think of having thousand of small linux boxes dedicated to either run as a CPU server or a Fileserver or whatever service there is. kOrbit is good idea in theory (havent looked at the code yet) though the drawback with corba is that is client/server oriented.The advantage of 9P/IL is that it's a small protocol for distributing services, objects, devices on a file basis. Different computers can act as a server or client transparent to the users. 9P is still a client server protocol in a manner of users and services ( files ) instead of the "old" thinking of pc's and servers. /josef > > > > > For one of our demos, we ran a file server on a remote linux box (that we > > > just had a user account on), mounted it on a kORBit'ized box, and ran > > > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > > > syscalls. If nothing else, it's pretty nifty what you can do in little > > > code... > > Duh. And what's new about that? > > The "new" part is that our servers were < 100 lines of code each. Compare > that to kNFS. :) > >
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Chris Lattner wrote: Err shame on you, don't forget about lcall and exceptions, and interrupts, and... That is technically more than _o_n_e_ "entry point". :) Oh wait, what about sysenter/exit too? :) OK, you got me on lcall (however, that's iBCS-only, IIRC), but the rest... what the hell does userland to interrupts? thinks OK, make it 2 - pagefault can be arguably used in that way. The reason that I considered exceptions and interrupts is that often, exceptions get reflected as signals to the running processes (SIGSEGV, SIGFPE, SIGILL, others?), and interrupts can wake up processes (from sys_poll among others). I was considering more of the user-kernel and kernel-user transitions... anyways, that's really besides the point. :) this wonderful design we get all kinds of stuff like sys_oldumount vs sys_umount and others... Check how often anything uses the majority of that stuff... Correct, it's for backwards compatibility with old programs (for example libc5 uses a lot of those "old" syscalls). Yes, standard RPC mechanism would be nice. No, CORBA is not a good candidate - too baroque and actually known to lead to extremely tasteless APIs being implemented over it. Yes, I mean GNOME. So Check 9P and compare. Really. Section 5 of Plan 9 manpages. Available on plan-9.bell-labs.com/sys/man/ That's fine. Since the server is down (or the URL is bad), can you please give me an example of how 9P is better than CORBA? I freely admit to not knowing much about 9P... how much do you know about CORBA (aside from your opinion that GNOME uses it, and therefore it is bad. ;)? without breaking backwards compatibility). Please don't tell me that OOP is bad... or else we will have the eviscerate the VFS layer from the kernel (amount other subsystems)... :) OOP is a nice tool. However, it's a tool that has incredible potential of shooting one's foot. It's wonderful if you have sane set of methods. And that's a _big_ if. "Easily extensible" is not an absolutely good thing - C++ wankers all over the world are busily proving it every day. Heck, they make a living out of that. IOW, the problem with interface changes is _not_ in converting the old code. It's in choosing the right changes. And that part of the game can't be simplified. Oif. That's like telling someone that C is evil because it has for loops, and for loops can be used to write nasty code. "just write in assembler" he says. :) I would claim that someone could write a bad program (or shoot themselves in the foot) with any turing complete language. C++ definately give you more rope to do that with, but used wisely, it can also be nice. The trick is to just not have to work with other peoples C++ code. :) Hey, did I mention that kORBit and all its extensions are written in C? :) Like I mentioned in a previous email, CORBA does not preclude 9P. What it does buy you though, is compatibility with LOTS of preexisting CORBA tools. How much development infrastructure is there for 9P? I thought so. :) All UNIX userland on the client side. lib9p on the server side (23Kb of sparse C). Examples of use in servers - see the aforementioned site. Err... yeah, so you're effectively mapping UNIX/POSIX across 9P. That's not very creative, and you could do the same thing with CORBA. I ask again, "How much development infrastructure is there for 9P?". If you say "just use unix", then what is the point of 9P at all? (on linux). Linux already has most of posix (and some would claim all of the "good stuff" in posix.). Plan9 aint unix/posix though it has its Ape environment. What we do need to look at is a good implementation for distributed resources. The ideal would bee getting 9P and IL into linux. Think of having thousand of small linux boxes dedicated to either run as a CPU server or a Fileserver or whatever service there is. kOrbit is good idea in theory (havent looked at the code yet) though the drawback with corba is that is client/server oriented.The advantage of 9P/IL is that it's a small protocol for distributing services, objects, devices on a file basis. Different computers can act as a server or client transparent to the users. 9P is still a client server protocol in a manner of users and services ( files ) instead of the "old" thinking of pc's and servers. /josef For one of our demos, we ran a file server on a remote linux box (that we just had a user account on), mounted it on a kORBit'ized box, and ran programs on SPARC Solaris that accessed the kORBit'ized linux box's file syscalls. If nothing else, it's pretty nifty what you can do in little code... Duh. And what's new about that? The "new" part is that our servers were 100 lines of code each. Compare that to kNFS. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Alexander Viro spake thus: Maybe... I definitely agree that 14 is below the limit, but 30... Hell knows, from what I see on the box I'm using right now it seems to fall into several cathegories: * Very-Long-And-Verbose-Named-HOWTO.html * manpages for X and Tcl functions with obscenely long names * *.deb and corresponding *.diff.gz and *.dsc * var/state/apt/lists/* * ghostscript maps * Metallica - Master of Puppets.mp3 Oystein ;) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Thu, Dec 14, 2000 at 10:23:15AM +0100, josef höök wrote: Plan9 aint unix/posix though it has its Ape environment. What we do need to look at is a good implementation for distributed resources. The ideal would bee getting 9P and IL into linux. Think of having thousand of small linux boxes dedicated to either run as a CPU server or a Fileserver or whatever service there is. In the who cares department: this idea was one of the main reasons I founded BitMover, it's what we wanted to do before we got sidetracked by BitKeeper. So you have my vote, and if anyone does this, I'll be pissed as hell because I wanted to do it, but happy that it's getting done. If you look at lmbench, it's basically designed to measure all the crud that you would need to make this sort of thing go fast. So I've been thinking about this for at least 6 years. Sigh. -- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
There is a large perception of CORBA being slow, but for the most part it is unjustified. I believe that the act of _designing_ a completely new protocol, standardizing it, and making it actually work would be a huge process that would basically reinvent CORBA (obviously some of the design decisions could be made differently, but all the same issues would have be dealt with). CORBA is slow compared to some of the other solutions. The question I was trying to ask is whether you should put something smaller and faster into the kernel space and leave CORBA in userland. It's complex, it has security implications surely it belongs talking something simple and fast to the kernel. If you look at microkernels they talk a much simpler faster rpc protocol. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
[Alan DID not say this:] There is a large perception of CORBA being slow, but for the most part it is unjustified. Really? I have that same perception but I can't claim that I've measured it. On the other hand, I have measured the overhead of straight UDP, TCP, and Sun RPC ping/pong tests and you can find the code for that in any version of lmbench. It should be a 5 minute task for someone who groks corba to do the same thing using the same framework. If someone wants to do it, I'll guide them through the lmbench stuff. It's pretty trivial, start with this as a guide: /* * tcp_xact.c - simple TCP transaction latency test * * Three programs in one - * server usage: tcp_xact -s * client usage: tcp_xact hostname * shutdown: tcp_xact -hostname * * Copyright (c) 1994 Larry McVoy. Distributed under the FSF GPL with * additional restriction that results may published only if * (1) the benchmark is unmodified, and * (2) the version in the sccsid below is included in the report. * Support for this development by Sun Microsystems is gratefully acknowledged. */ char*id = "$Id$\n"; #include "bench.h" voidclient_main(int ac, char **av); voiddoserver(int sock); voiddoclient(int sock); voidserver_main(int ac, char **av); voiddoserver(int sock); int main(int ac, char **av) { if (ac != 2) { fprintf(stderr, "Usage: %s -s OR %s [-]serverhost\n", av[0], av[0]); exit(1); } if (!strcmp(av[1], "-s")) { if (fork() == 0) { server_main(ac, av); } exit(0); } else { client_main(ac, av); } return(0); } void client_main(int ac, char **av) { int sock; char*server; charbuf[100]; if (ac != 2) { fprintf(stderr, "usage: %s host\n", av[0]); exit(1); } server = av[1][0] == '-' ? av[1][1] : av[1]; sock = tcp_connect(server, TCP_XACT, SOCKOPT_NONE); /* * Stop server code. */ if (av[1][0] == '-') { close(sock); exit(0); } BENCH(doclient(sock), MEDIUM); sprintf(buf, "TCP latency using %s", av[1]); micro(buf, get_n()); exit(0); /* NOTREACHED */ } void doclient(int sock) { charc; write(sock, c, 1); read(sock, c, 1); } void child() { wait(0); signal(SIGCHLD, child); } void server_main(int ac, char **av) { int newsock, sock; if (ac != 2) { fprintf(stderr, "usage: %s -s\n", av[0]); exit(1); } GO_AWAY; signal(SIGCHLD, child); sock = tcp_server(TCP_XACT, SOCKOPT_NONE); for (;;) { newsock = tcp_accept(sock, SOCKOPT_NONE); switch (fork()) { case -1: perror("fork"); break; case 0: doserver(newsock); exit(0); default: close(newsock); break; } } /* NOTREACHED */ } void doserver(int sock) { charc; int n = 0; while (read(sock, c, 1) == 1) { write(sock, c, 1); n++; } /* * A connection with no data means shut down. */ if (n == 0) { tcp_done(TCP_XACT); kill(getppid(), SIGTERM); exit(0); } } -- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Chris Lattner [EMAIL PROTECTED] writes: p9fs exists. I didn't see these patches since August, but probably I can poke Roman into porting it to the current tree. 9P is quite simple and unlike CORBA it had been designed for taking kernel stuff to userland. Besides, authors definitely understand UNIX... One thing that you might want to mention Alexander: 9P is not a general communications protocol. In fact, it doesn't work very well across the internet at all. To get decent performance, the Plan9 group (which, is a very cool group. :) has to specify a new protocol that competes with TCP on the level of complexity (IL: http://plan9.bell-labs.com/sys/doc/il/il.html) this might be because the Bell Labs folks don't like RPC in general when network latencies become involved? I'm guessing. 'cause CORBA is still pretty much objectified RPC, as far as I know. I don't think you want to abstract the network out just like that when dealing with kernels. -- Entropy isn't what it used to be. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
I surely hope that this thread wont end here. It's extremely important to look at this since we're heading towards distributed resources, where humans dont work on a server but towards other people through servers. /josef - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
Alexander Viro wrote: Err... how about this: Give me two or three kORBit syscalls and I can get rid of all the other 100+ syscalls! :) Like it ioctl() does it? Number of entry points is _not_ an issue. Diversity of the API is. Technically, kernel has 1 (_o_n_e_) entry point as far as userland is concerned. int 0x80 on x86. Can't beat that, can you? If there's one thing that could be nicer, ioctl() would be it. ioctl() works ok in C and terribly in everything else. CORBA works well in scripting languages and Java, and sucks in C compared with a simple ioctl(). Is there a way to marry the best sides of each? Is 9P that way (I don't know much about it)? -- Jamie - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Wed, 13 Dec 2000, Chris Lattner wrote: 1. kORBit adds about 150k of code to the 2.4t10 kernel. 2. kNFS adds about 100k of code to the 2.4t10 kernel. 3. kORBit can do everything kNFS does, plus a WHOLE lot more: For example implement an NFS like server that uses SSL to send files and requests... so it is really actually "secure". So can you implement a kNFS server in kORBit that takes less than 50kB of RAM? Otherwise it's still a contributor to bloat and this argument won't work ;) I guess it's time to stop the flaming and to see what can be achieved using kORBit. The people who favour kORBit should IMHO be left alone and given the opportunity to show what can be achieved with kORBit ... if they don't achieve anything, the nay-sayers can always claim their "victory"; if something useful comes out the kORBit people can claim usefulness. regards, Rik -- Hollywood goes for world dumbination, Trailer at 11. http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
1. kORBit adds about 150k of code to the 2.4t10 kernel. 2. kNFS adds about 100k of code to the 2.4t10 kernel. So can you implement a kNFS server in kORBit that takes less than 50kB of RAM? Otherwise it's still a contributor to bloat and this argument won't work ;) Actually the kORBitNFS server would have to take -50K of code to break even. :) The point was that kORBit lets you do a lot more... so hopefully that 50k of generality gives you something. :) I guess it's time to stop the flaming and to see what can be achieved using kORBit. The people who favour kORBit should IMHO be left alone and given the opportunity to show what can be achieved with kORBit ... if they don't achieve anything, the nay-sayers can always claim their "victory"; if something useful comes out the kORBit people can claim usefulness. Agreed! -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
that is not transparently hidden to the user. Why can't I just open /dev/net0 and get the first network device? Because we have so many inconsistent, poorly design, inextensible interfaces laying around, thats why. This is a bad example, but a (perhaps?) good point. It seems it should be possible to implement an unlimited number of TCP/IP devices, each Part of my argument is that although every kernel interface boils down to something as simple as ioctl (where you have lots of api's multiplexed onto one data sink, int 0x80 is another example, kORBit is another), some ways of doing this multiplexing are better than others. I look at it as a hierarchy: 1. CORBA/kORBit 2. int 80/lcall 3. ioctl Each level is more structured than those below it. Each interface is also "cleaner" than those below it. I don't think that anyone would argue that we should replace int 80 and friends with an interface in the spirit of ioctl (even though it would be functionally identical). representing a connection, and each connected, disconnected, etc. by ioctls... Of course, I could be way wrong about this ioctl's... they come up again. Everytime that I look at the beautiful, clean, abstract interface that Unix exposes... it makes me happy. But unfortunately, if you look closer, you realize that the Unix API isn't really clean or nice at all... there is this dumping ground for odds and ends that don't fit into the standard model. So when you show someone the standard model, it looks clean and pretty... but someone always forgets to mention ioctl. I would claim that ioctl is one of the biggest reasons that kORBit needs to exist. See below. I can't do ls /dev/*net* and get all the network devices either. Actually, one of the very cool things about CORBA is that Different network devices are _very_ different... You can't broadcast on a modem, for instance, nor can you use promiscuous mode on a modem... You also can't do byte-by-byte raw mode on an ethernet... Precisely. But they do all have very common interfaces. For example, the standard network interface would probably have "transmit/receive block" and "getstatus" commands. This does not mean that they cannot implement other interfaces, however. Imagine this situation for writing your packet sniffer: 1. You open up /dev/net0 and get the Network interface 2. You query the network interface to see if it has an implementation of the promiscuous interface. 3. If not, you bail, because it doesn't support it. 4. If, you go ahead and use it. Contrast that to: 1. Survey all the different interfaces that are known (at development time of course) to have promiscuous interfaces. 2. Find out which ones are active. 3. Depending on what kind of interface it is, load a library that can understand the byte format comming off the device. 4. Use that library. The problems with the second (currently used) approach is that the APPLICATION writer has to keep up with new hardware developments and new interfaces. They have to understand and code parsers for the bytestreams coming from each device, which (as you mentioned) are all different, because they all support different (although overlapping) extensions. All of these calls that currently get dumped into ioctl (because they are not important enough to warrant an API function at the top level) suddenly become well structure and well designed interfaces. For example, if your device doesn't support the terminal handling ioctls the system has to basically figure that out and report errors on it. With CORBA/kORBit, you would just not implement that interface... so if someone asked for it, they would get a null pointer to the interface, which they check for and realize that it's impossible to do terminal stuff on. isatty (an example of an IOCTL wrapper) suddenly becomes implicit and the API is cleaner... I have no problem with the "unix way", but I do think that it can be augmented in some ways... -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
* Rik van Riel | | On Wed, 13 Dec 2000, Chris Lattner wrote: | | 1. kORBit adds about 150k of code to the 2.4t10 kernel. | 2. kNFS adds about 100k of code to the 2.4t10 kernel. | 3. kORBit can do everything kNFS does, plus a WHOLE lot more: For example | implement an NFS like server that uses SSL to send files and | requests... so it is really actually "secure". | | So can you implement a kNFS server in kORBit that takes | less than 50kB of RAM? Otherwise it's still a contributor | to bloat and this argument won't work ;) Well, kORBit itself is bigger than kNFS, but like Chris said, it can do much more. :) For testing kORBit we wrote a new filesystem CorbaFS. The CorbaFS client is a kernel module that basically forwards the Linux VFS calls to the userspace CorbaFS server. According to lsmod, the CorbaFS module takes up only 11KB of RAM. Keep in mind that CorbaFS is currently only a proof-of-concept of the kernel calling into userspace through kORBit (it is a read-only filesystem at the moment, for example). But it should give you some idea. The cool thing is that the CorbaFS userspace server can implement any kind of filesystem you want, as long as it follows the CorbaFS interface! The current implementation exports the filesystem on the host machine that it is running on, similar to NFS. But we also have ideas for FTP or web filesystems, for example. Imagine being able to mount the web CorbaFS onto /mnt/www and do a cat /mnt/www/www.kernel.org/index.html and the CorbaFS userspace server takes care of loading the webpage and returning it to the kernel client. And these new filesystems don't take up any extra space in the kernel, since they all talk to the same CorbaFS kernel module! Not to mention being able to implement the filesystem in any language you like, debug the implementation in userspace, etc. -- Fredrik Vraalsen - Research Assistant, Pablo research group Department of Computer Science, University of Illinois at U-C - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On 14 Dec 2000, Michael Livshin wrote: this might be because the Bell Labs folks don't like RPC in general when network latencies become involved? I'm guessing. 'cause CORBA is still pretty much objectified RPC, as far as I know. I don't think you want to abstract the network out just like that when dealing with kernels. OK, since everyone seems to want to argue about this on orbit-list for some reason, my $0.02: CORBA/IIOP is saner than SunRPC in a lot of ways, and it's not too horribly more complicated to implement a barebones ORB than a barebones SunRPC impl. Comments about CORBA being too slow are nonsense - it's just that a few crackheads have tried to stick ORBit in the kernel as a global IPC mechanism, which it really is not suitable for. CORBA has its place, and a GIOP mapping to a kernel-friendly IPC mechanism (instead of TCP/IP) would certainly make it more useful in the kernel, but generic mechanisms such as CORBA cannot by definition be as fast as IPC mechanisms optimized for a specific task. People like Al Viro, who haven't written GNOME programs of any size and certainly don't have mounds of in-depth knowledge of it, should probably shut up about about its API design. :) TTFN, -- Elliot No new ideas for my .sig, and Alan told me my old one was an urban myth, so just ignore this. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
There is a large perception of CORBA being slow, but for the most part it is unjustified. I believe that the act of _designing_ a completely new CORBA is slow compared to some of the other solutions. The question I was trying to ask is whether you should put something smaller and faster into the kernel space and leave CORBA in userland. What I'm trying to show is that CORBA itself is not slow. CORBA can be thought of as an IDEA, and the current implementations are suboptimal. For example, no CORBA implementation has been tweaked to run well in-kernel. Because of this, people will say that CORBA sucks and is slow, but that's not true. I will agree that IIOP (the standardized corba communication mechanism) is much slower and more complex than we would want for a generalized user-kernel communication mechanism. The nice thing about CORBA is that you can design your own transport to optimize things. Thus 99% of the time you could use a heavily optimized, very simple transport that doesn't do much. This is what you are asking for, and it make perfect sense. What CORBA buys you is the ability to use this streamlined interface without giving up full generality... It's complex, it has security implications surely it belongs talking something simple and fast to the kernel. First off, there are a lot of complex systems in the kernel. :) Second, there are not security implications above and beyond any normal kernel hacking. The reason that kORBit has security implications right now is that there _IS NO SECURITY_. Imagine implementing sys_read with no checks of what uid is running, and you get the idea. Actually security in CORBA can be done BETTER in kernel space than in user space, but I digress... If you look at microkernels they talk a much simpler faster rpc protocol. Yes but for the most part they lose all of the advantages of CORBA as well. For example, every microkernel that I'm aware of can only talk to remote servers/clients that are running the same microkernel on the remote machine. There are no provisions for byte swapping if the endianness is incorrect or for having structured bytestreams (so that you can tell things about the raw bytes coming over the line without understanding them all). -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
Fredrik Vraalsen wrote: The cool thing is that the CorbaFS userspace server can implement any kind of filesystem you want, as long as it follows the CorbaFS interface! Sorry, it's yet another one. Or does it do something different? (YAO hasn't stopped me working on userspace filesystems either :-) The current implementation exports the filesystem on the host machine that it is running on, similar to NFS. But we also have ideas for FTP or web filesystems, for example. Imagine being able to mount the web CorbaFS onto /mnt/www and do a cat /mnt/www/www.kernel.org/index.html and the CorbaFS userspace server takes care of loading the webpage and returning it to the kernel client. And these new filesystems don't take up any extra space in the kernel, since they all talk to the same CorbaFS kernel module! Not to mention being able to implement the filesystem in any language you like, debug the implementation in userspace, etc. A bit like CodaFS and Perlfs. Except, being CORBA, you can run the userspace server remotely? ;-) evil grin -- Jamie - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
Of course. Which is why CORBA is about putting STRUCTURE in that stream of random bytes coming over the wire. Why should I have to rewrite my marshalling and demarshalling code every time I want to write a server. read and write are fine. But sometimes I want a structure. Sometimes, my structures aren't laid out like C struct's either. What then? What if I want to send an "object" to you? Then I need to understand the object anyway. And Corba objects are horribly over complex. Any lisp hacker will tell you there is only one type: a list. But alan, that's the beautiful thing. Given a CORBA object, you can understand its structure without knowing exactly what the contents are. You can effectively derive it's prototype just by inspecting it. The only difference between lisp and a CORBA object in this respect is that each item in the list is typed so that you have even more info about what to do with it. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
There is a large perception of CORBA being slow, but for the most part it is unjustified. Really? I have that same perception but I can't claim that I've measured it. On the other hand, I have measured the overhead of straight UDP, TCP, and Sun RPC ping/pong tests and you can find the code for that in any version of lmbench. It should be a 5 minute task for someone who groks corba to do the same thing using the same framework. If someone wants to do it, I'll guide them through the lmbench stuff. It's pretty trivial, start Urm... thanks for the offer... but you misunderstand me if you think that I'm claiming that kORBit is the ideal/fast implementation that everyone has been looking for. There is still much to be done. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
Oh, great. So we don't have to care about formatting changes. We just have to care about the data changes. IOW, we are shielded from the results of changes that should never happen in the first place. And the benefit being...? What the hell are you talking about? Did you even read my example? I was giving an example of extending an API, adding new functionality to it. Yes, I did. What I don't understand is how kernel mechanism for marshalling would make your life easier wrt changes. I gave a very simple example of how an interface could be designed and then later extended without breaking any user space programs. Please reread the example and tell me precisely what doesn't make sense, and I would be glad to explain it better. Notice also that I can say ls /dev/*mouse* and get some idea of the files there. I can't do that for your interfaces. And you know what? I can't do ls /dev/*net* and get all the network devices either. Actually, one of the very cool things about CORBA is that you can BROWSE/LIST/SEARCH all objects currently instantiated. Being able to browse all in kernel objects would be very cool. So what's to stop you from letting readdir() do the work? WTF do we need one more API for returning the list of objects? Urm.. that was the point. You don't add a new API (it's built into corba, along with introspection). Also, like I JUST MENTIONED, readdir will have no way of knowing what is a network device in the quagmire of stuff floating around in /dev. (ls *net* does actually call readdir ya know. Perhaps you should read up more on unix... [sorry couldn't help it. ;]) The point being: if your program spends efforts on marshalling it would better _do_ something with the obtained data. And then we are back to the square 1. Uh huh, go ahead and read the example I sent to you. Did that. So they you understand that no interface was broken even though more data is now available... Returning to your example, I could not tee(1) the stream into file for later analysis. Not unless I write a special-case program for intercepting that stuff. I don't see why it is a good thing. On the contrary, it would be pretty easy to do something like that with CORBA. No you wouldn't be able to use tee, but why would you want to tee a binary data file? The only reason that tee works in this situation is For the same reasons why I use tar, gzip, whatever. I don't _want_ to invent a new utility every time when Joe Doe adds a new piece of interface. Data is data is data. I can uuencode it and send to somebody who would care to analize the bloody thing. Do you mean that I need to write a special tool for that? For _every_ member of every interface somebody decided to add? I don't think so. I really don't. That's fine, I don't blame you. Ya know what? I even agree with you (image that? hehe :). You seem to forget that a CORBA object is exactly the same thing: a stream of bytes. Actually it does better than that. It's a stream of bytes that you can introspect on and determine the structure of. Imaging this. You have a bytestream flowing out of some pipe/device/socket/file/whatever and you pipe it to a "Decode" program. The decode program spits this out: sequence struct { float float } This example is meant to illustrate that _in a general way_ you can decode the structure of the bytestream. (hence my claims about CORBA adding structure to a standard bytestream)... this is quite a bit more useful than trying to figure out what 'cat' is trying to tell you about binary data (especially if it's floating point numbers). line... your analysis program would have to have special purpose code to parse the file. EVERY consumer of "mouse data" would have to parse the Or I would look at the size. Or I would say od and look at the result. Don't tell me what to do with the data, when I'll need to parse it I will. And if you expect me to bother with writing more stuff when generic tools would work fine - too bad, I've been there, done that and I'm not coming back. Heh... I'm not trying to replace od or cat or tee or pipes. What I'm trying to do is give you a larger toolkit that gives you MORE power. Imagine you want to get the first float value (of each struct) of the above stream. Imagine a functional scripting language where you could do this (haskell syntax stolen gratitously): cat /dev/floatymouse | funcwith 'map fst' Which you could then point to the "decode" type program mentioned above, and you would now get: sequencefloat BTW, you cared about size of /dev, didn't you? /usr/bin choke-full of tee-foo, tee-bar, yodda, yodda would be better? You wouldn't have to replace the standard utilities. Sorry to rain on your parade. file. That seems pretty silly to me. Difference between good program and bad one: the former gets used in ways its authors never thought about... *cough* see above.
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
But alan, that's the beautiful thing. Given a CORBA object, you can understand its structure without knowing exactly what the contents are. You can effectively derive it's prototype just by inspecting it. Oh dear this isnt going in is it. Look I know the prototype of every single lisp transaction - its a list. Period. I know how to manipulate any data I have and print it, because its a list. I know how to extend it, because its .. a list. If I need to put content identification in, well guess what - thats a list ((my_name "Hello") (his_name "Foo")) and XML is simply lisp done wrong. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Thu, 14 Dec 2000, Chris Lattner wrote: Yes, I did. What I don't understand is how kernel mechanism for marshalling would make your life easier wrt changes. I gave a very simple example of how an interface could be designed and then later extended without breaking any user space programs. Please reread the example and tell me precisely what doesn't make sense, and I would be glad to explain it better. What does it win compared to adding new file and allowing new programs use it while the old ones keep using the old one? That's precisely the same situation, isn't it? So what's to stop you from letting readdir() do the work? WTF do we need one more API for returning the list of objects? Urm.. that was the point. You don't add a new API (it's built into corba, along with introspection). Also, like I JUST MENTIONED, readdir will have no way of knowing what is a network device in the quagmire of stuff floating around in /dev. (ls *net* does actually call readdir ya know. Perhaps you should read up more on unix... [sorry couldn't help it. ;]) Cool. We don't have network devices represented as fs. We could have, but nobody had really cared. Guess what, I know that. It's _not_ a good thing. What you propose is expanding the area of brokenness. structure of. Imaging this. You have a bytestream flowing out of some pipe/device/socket/file/whatever and you pipe it to a "Decode" program. The decode program spits this out: Is it separated on per-object streams? Heh... I'm not trying to replace od or cat or tee or pipes. What I'm trying to do is give you a larger toolkit that gives you MORE power. Imagine you want to get the first float value (of each struct) of the above stream. Imagine a functional scripting language where you could do this (haskell syntax stolen gratitously): cat /dev/floatymouse | funcwith 'map fst' No problems with that. But notice that you did introduce a new device here. No, it isn't. The difference being: letting driver to define a filesystem and mounting/exporting it/whatever means that you get to use _all_ _normal_ _data-agnostic_ _tools_. And believe me, it matters. If you don't understand that - no offense, but you don't understand UNIX. Ouch. /me digs around and puts on thick skin Okay, fine. You want to talk unix. You proclaim that this beautiful infalliable API is simple open/close/read/write calls on stanardized devices, right? What about ioctl? What about the HUGE class of crap that ioctl() is avoidable. Proof: Plan 9. They don't _have_ that system call. It doesn't mean that we should (or could) remove it. It _does_ mean that new APIs do not need it. And I'm quite serious. ioctl() is used in too many programs to kill it off (compatibility reasons). No arguments here. However, looking back at these cases... All of them could be done without ioctl(). Moreover, it had been done. Says something about the need of adding new ioctls, doesn't it? Instead of having a metric buttload of device nodes in /dev you have drivers export filesystems. You can mount them anywhere you want. These filesystems can contain both devices and control files. Writing commands to control file gives you a replacement of ioctl(). It actually works quite nicely. As long as you separate the data streams for different objects and give them some names, so they can be intercepted, etc. - fine with me. There's nothing wrong with having a library for marshalling. There's nothing wrong with having it in the kernel too, BTW. However, _that_ is completely independent from any form of transport. All you need is "export a piece of namespace to another box". - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit (and ioctl must die!)
Alexander Viro [EMAIL PROTECTED] writes: ioctl() is avoidable. Proof: Plan 9. They don't _have_ that system call. It doesn't mean that we should (or could) remove it. It _does_ mean that new APIs do not need it. *I* sure wish we could. From the standpoint of trying to trace system calls, it's a big stinking black hole. All of the other syscalls (I think) have pretty well defined semantics in terms of what they do to a process' memory space, but the semantics of ioctl are "may read or write any memory whatsoever, and if you want to know what, well, it sucks to be you". Even NT does this better, if I'm interpreting this correctly: http://msdn.microsoft.com/library/psdk/winbase/devio_9quk.htm --Mike -- [O]ne of the features of the Internet [...] is that small groups of people can greatly disturb large organizations. --Charles C. Mann - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Thu, 14 Dec 2000, josef [iso-8859-1] höök wrote: > Chip Salzenberg wrote: > > > According to Alexander Viro: > > > 9P is quite simple and unlike CORBA it had been designed for taking > > > kernel stuff to userland. Besides, authors definitely understand > > > UNIX... > > > > As nice as 9P is, it'll need some tweaks to work with Linux. > > For example, it limits filenames to 30 characters; that's not OK. > > -- > > Chip Salzenberg- a.k.a. -<[EMAIL PROTECTED]> > >"Give me immortality, or give me death!" // Firesign Theatre > > > > Another thing in mind that if we would want to use 9P we would also need to > port IL . It can live atop of TCP, it can live atop of AF_UNIX. IL would be nice for more than one reason, though... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Thu, 14 Dec 2000, Chris Lattner wrote: > > Oh, great. So we don't have to care about formatting changes. We just > > have to care about the data changes. IOW, we are shielded from the > > results of changes that should never happen in the first place. And the > > benefit being...? > > What the hell are you talking about? Did you even read my example? I was > giving an example of extending an API, adding new functionality to it. Yes, I did. What I don't understand is how kernel mechanism for marshalling would make your life easier wrt changes. > > Notice that we could equally well add /dev/floatymouse and everything that > > worked with old API would keep working. The only programs that would need > > to know about floats would be ones that would... need to know about floats > > since they want to work with them. > > Wonderful solution. This is exactly what you would _have_ to do. Are you > aware of why devfs came into existence? It is because there are already > so damn many device nodes in /dev. Lets keep adding more. And? The only _wrong_ thing here is storing that stuff on a physical disk. Let the driver export the filesystem and union-mount it on /dev. End of story. Or mount it anywhere else, whatever place makes you happy. It's not like we were eating majors/minors that way... > > Notice also that I can say ls /dev/*mouse* and get some idea of the files > > there. I can't do that for your interfaces. > > And you know what? I can't do ls /dev/*net* and get all the network > devices either. Actually, one of the very cool things about CORBA is that > you can BROWSE/LIST/SEARCH all objects currently instantiated. Being > able to browse all in kernel objects would be very cool. So what's to stop you from letting readdir() do the work? WTF do we need one more API for returning the list of objects? > > The point being: if your program spends efforts on marshalling it would > > better _do_ something with the obtained data. And then we are back to > > the square 1. > > Uh huh, go ahead and read the example I sent to you. Did that. > > Returning to your example, I could not tee(1) the stream into file for > > later analysis. Not unless I write a special-case program for intercepting > > that stuff. I don't see why it is a good thing. > > On the contrary, it would be pretty easy to do something like that with > CORBA. No you wouldn't be able to use tee, but why would you want to tee > a binary data file? The only reason that tee works in this situation is For the same reasons why I use tar, gzip, whatever. I don't _want_ to invent a new utility every time when Joe Doe adds a new piece of interface. Data is data is data. I can uuencode it and send to somebody who would care to analize the bloody thing. Do you mean that I need to write a special tool for that? For _every_ member of every interface somebody decided to add? I don't think so. I really don't. > because it's agnostic to the format of the data coming off the Exactly. > line... your analysis program would have to have special purpose code to > parse the file. EVERY consumer of "mouse data" would have to parse the Or I would look at the size. Or I would say od and look at the result. Don't tell me what to do with the data, when I'll need to parse it I will. And if you expect me to bother with writing more stuff when generic tools would work fine - too bad, I've been there, done that and I'm not coming back. BTW, you cared about size of /dev, didn't you? /usr/bin choke-full of tee-foo, tee-bar, yodda, yodda would be better? > file. That seems pretty silly to me. Difference between good program and bad one: the former gets used in ways its authors never thought about... > > I also don't see where the need in new system calls (or ioctls, same shit) > > comes from. Notice that your way is much closer to new system call than > > read()/write() of the right stuff. > > A new syscall was one example. It would be very simple to implement this > by making _yet_another_ device node in /dev and issue reads and writes to > it. That's more of a syntactic issue than a symantic one. No, it isn't. The difference being: letting driver to define a filesystem and mounting/exporting it/whatever means that you get to use _all_ _normal_ _data-agnostic_ _tools_. And believe me, it matters. If you don't understand that - no offense, but you don't understand UNIX. To get this stuff working without excessive PITA you'll need to teach userland about the new mechanism. Shells, mailers, archivers, backup, you name it. It might be an interesting OS, but it sure as hell wouldn't be UNIX and what's worse, it doesn't exist. Mix of that CORBA_OS and UNIX would be extremely nasty. I have a serious suspicion that CORBA_OS in itself would be a wonderful illustration to "Why Pascal is Not My Favorite Programming Language", but that's another story (BTW, that paper may make an interesting reading - check
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > NO. You want leagacy program to "just get" rounded ints, and new programs > > to get the "full precision" of the floating point #'s. > What rounded ints? Rounded to zero? To nearest integer? To plus or minus > infinity? Does program have something to say here? The exact same thing that older mouse drivers did. In this particular case, it doesn't matter anyways, because it's a simple example. If you REALLY care about rounding those coordinates a certain way, it's trivial to add an extra component to the interface. > ... > My mouse example > ... > Oh, great. So we don't have to care about formatting changes. We just > have to care about the data changes. IOW, we are shielded from the > results of changes that should never happen in the first place. And the > benefit being...? What the hell are you talking about? Did you even read my example? I was giving an example of extending an API, adding new functionality to it. > Notice that we could equally well add /dev/floatymouse and everything that > worked with old API would keep working. The only programs that would need > to know about floats would be ones that would... need to know about floats > since they want to work with them. Wonderful solution. This is exactly what you would _have_ to do. Are you aware of why devfs came into existence? It is because there are already so damn many device nodes in /dev. Lets keep adding more. Why is it (for example) that on my machine (mandrake 7 install, no devfs), I automatically get inodes for ISDN & PPP!?! Well, because ISDN has a slightly different interface, and you're talking to a different driver that is not transparently hidden to the user. Why can't I just open /dev/net0 and get the first network device? Because we have so many inconsistent, poorly design, inextensible interfaces laying around, thats why. > Notice also that I can say ls /dev/*mouse* and get some idea of the files > there. I can't do that for your interfaces. And you know what? I can't do ls /dev/*net* and get all the network devices either. Actually, one of the very cool things about CORBA is that you can BROWSE/LIST/SEARCH all objects currently instantiated. Being able to browse all in kernel objects would be very cool. > The point being: if your program spends efforts on marshalling it would > better _do_ something with the obtained data. And then we are back to > the square 1. Uh huh, go ahead and read the example I sent to you. > Returning to your example, I could not tee(1) the stream into file for > later analysis. Not unless I write a special-case program for intercepting > that stuff. I don't see why it is a good thing. On the contrary, it would be pretty easy to do something like that with CORBA. No you wouldn't be able to use tee, but why would you want to tee a binary data file? The only reason that tee works in this situation is because it's agnostic to the format of the data coming off the line... your analysis program would have to have special purpose code to parse the file. EVERY consumer of "mouse data" would have to parse the file. That seems pretty silly to me. > I also don't see where the need in new system calls (or ioctls, same shit) > comes from. Notice that your way is much closer to new system call than > read()/write() of the right stuff. A new syscall was one example. It would be very simple to implement this by making _yet_another_ device node in /dev and issue reads and writes to it. That's more of a syntactic issue than a symantic one. > As for the proc/meminfo... What would you do to a userland programmer who > had defined a structure like that? Let's see: way too large, ugly as hell, > many fields are almost guaranteed to become meaningless at later point... > It was not designed, it got accreted. And that's very mild description - > judging by results it might as well be s/ac/ex/. There is no provision in CORBA that prevents the creation of ugly interfaces. There is no provision in C that prevents them either, as witnessed by this example. At least with CORBA, I can extend the interface to actually return meaningful results while still maintaining backwards compatibility with programs that don't actually look at that field (as is done now by sticking the 0 in there). The trick is having enough foresight to design robust interfaces. I'm not saying that I'm an expert at that, and obviously nobody can be. What I _am_ saying is that CORBA/kORBit gives you flexibility in how you do interfaces, and it lets you "future proof" them (obviously only within reason). Adding a new device node everytime an API gets changed is like defining a different API for every hard disk that exists. Seems kinda silly. -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Wed, 13 Dec 2000, Chris Lattner wrote: > > OK, now I'm completely confused. > > * which complex data structures do you want to export from the kernel > > in non-opaque way? > > * which of those structures are guaranteed to remain unchanged? > > * if you have userland-to-userland RPC in mind - why put anything > > marshalling-related into the kernel? > > Okay, I think I did my best to completely confuse you. :) #1: CORBA > objects _are_ opaque. #2 is irrelevant due to #1. #3: userland->userland > is not the interesting part. We want kernel->userland or user->kernel, as > the common (ie fast) case, but we also want to do client->client or > kernel->kernel without anything breaking. > > ints and received a bunch of doubles... You either need to decide on > > rounding (and it's a non-obvious question) or you need to change quite a > > bit of code in your program. It goes way past the demarshalling, no > > matter whether you use CORBA, 9P or printf/scanf. > > NO. You want leagacy program to "just get" rounded ints, and new programs > to get the "full precision" of the floating point #'s. What rounded ints? Rounded to zero? To nearest integer? To plus or minus infinity? Does program have something to say here? > Original interface for the mouse: > > interface mouse { > void getPosition(out long X, out long Y); > }; > > So if you get the mouse interface in CORBA (which is an opaque object), > you get back something that you can make requests to, and, for example, > get the coordinate the mouse is currently at. Okay, that's fine, 15 years > later and after much legacy software has been developed, company X > develops a new high precions floating point mouse. Well crap, don't want > to change the interface. Now instead of getting that, you get the > "mouse2" or "floaty mouse" interface (for sake of argument suppose that > we actually did want to use FP arithmetic in the kernel, that's an > artifact of a bad example, not a bad point :): > > interface floatymouse : extends mouse { > void getFloatyPosition(out float X, out float Y); > } > > Now people that get the "/dev/mouse" interface get a floatymouse. Does > this break all that leagacy mouse wielding code? No, because we used > inheritance, the floatymouse is a superset of the mouse, and the original > interface still works. The server/kernel side of the floaty mouse just > chops off the decimal places when you use the getPosition method. Oh, great. So we don't have to care about formatting changes. We just have to care about the data changes. IOW, we are shielded from the results of changes that should never happen in the first place. And the benefit being...? Notice that we could equally well add /dev/floatymouse and everything that worked with old API would keep working. The only programs that would need to know about floats would be ones that would... need to know about floats since they want to work with them. Notice also that I can say ls /dev/*mouse* and get some idea of the files there. I can't do that for your interfaces. The point being: if your program spends efforts on marshalling it would better _do_ something with the obtained data. And then we are back to the square 1. Returning to your example, I could not tee(1) the stream into file for later analysis. Not unless I write a special-case program for intercepting that stuff. I don't see why it is a good thing. I also don't see where the need in new system calls (or ioctls, same shit) comes from. Notice that your way is much closer to new system call than read()/write() of the right stuff. As for the proc/meminfo... What would you do to a userland programmer who had defined a structure like that? Let's see: way too large, ugly as hell, many fields are almost guaranteed to become meaningless at later point... It was not designed, it got accreted. And that's very mild description - judging by results it might as well be s/ac/ex/. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
At 12:15 AM 12/14/2000 -0500, you wrote: >Hmm... Cutoff seems to sit somewhere around 45 - above that there are only >apt-get droppings and they definitely are over the top. Dunno, you may be >right, but looks like I never had a need to create anything that long. It's always good to be able to include at least two date-time pairs in a filename... especially for scientific stuff. -- This message has been brought to you by the letter alpha and the number pi. Open Source: Think locally; act globally. David Feuer [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Wed, 13 Dec 2000, Chip Salzenberg wrote: > As long as names are to be created, or at least understood, by humans, > there will be some limit on *usable* length. In my experience, 255 is > above that limit, but 30 is below it. And I cut my teeth on a system > that had exactly that length limitation (UNOS). Maybe... I definitely agree that 14 is below the limit, but 30... Hell knows, from what I see on the box I'm using right now it seems to fall into several cathegories: * Very-Long-And-Verbose-Named-HOWTO.html * manpages for X and Tcl functions with obscenely long names * *.deb and corresponding *.diff.gz and *.dsc * var/state/apt/lists/* * ghostscript maps Hmm... Cutoff seems to sit somewhere around 45 - above that there are only apt-get droppings and they definitely are over the top. Dunno, you may be right, but looks like I never had a need to create anything that long. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> OK, now I'm completely confused. > * which complex data structures do you want to export from the kernel > in non-opaque way? > * which of those structures are guaranteed to remain unchanged? > * if you have userland-to-userland RPC in mind - why put anything > marshalling-related into the kernel? Okay, I think I did my best to completely confuse you. :) #1: CORBA objects _are_ opaque. #2 is irrelevant due to #1. #3: userland->userland is not the interesting part. We want kernel->userland or user->kernel, as the common (ie fast) case, but we also want to do client->client or kernel->kernel without anything breaking. > > /dev/mouse for example. Combine that with the problem that /dev/mouse > > might change format in the future (okay stupid example, but you get the > > idea) to use floating point coordinates, and things certainly get hairy. > HUH? OK, suppose it had happened. Do you really expect that you will not > need to change your applications? I mean, if you expected a bunch of YES. You should not have to change your apps at all. > ints and received a bunch of doubles... You either need to decide on > rounding (and it's a non-obvious question) or you need to change quite a > bit of code in your program. It goes way past the demarshalling, no > matter whether you use CORBA, 9P or printf/scanf. NO. You want leagacy program to "just get" rounded ints, and new programs to get the "full precision" of the floating point #'s. > OK, suppose you have a CORBA-based system and mouse drivers' API had been > changed - they really want to return floating point coordinates. How will > CORBA help you? Aside of making your programs scream aloud, that is. Okay, maybe this isn't a terrible example afterall. :) Consider this theoretical API for a mouse, that I'm certainly not saying is wonderful or perfect, just consider it to be an example (on the fly), and no it doesn't let you poll for mouse activity, this is just off the cuff: Original interface for the mouse: interface mouse { void getPosition(out long X, out long Y); }; So if you get the mouse interface in CORBA (which is an opaque object), you get back something that you can make requests to, and, for example, get the coordinate the mouse is currently at. Okay, that's fine, 15 years later and after much legacy software has been developed, company X develops a new high precions floating point mouse. Well crap, don't want to change the interface. Now instead of getting that, you get the "mouse2" or "floaty mouse" interface (for sake of argument suppose that we actually did want to use FP arithmetic in the kernel, that's an artifact of a bad example, not a bad point :): interface floatymouse : extends mouse { void getFloatyPosition(out float X, out float Y); } Now people that get the "/dev/mouse" interface get a floatymouse. Does this break all that leagacy mouse wielding code? No, because we used inheritance, the floatymouse is a superset of the mouse, and the original interface still works. The server/kernel side of the floaty mouse just chops off the decimal places when you use the getPosition method. This is the kind of stuff that I'm talking about. The concept that a mouse is more than a byte stream... that there is actual structure that must be understood before a datastream can be used. Yes, I am very aware that there are other ways of doing this, but CORBA is a very general & powerful system that is also quite mature (unlike kORBit. ;) I hope this helps clarify what the heck I'm talking about. :) -Chris btw, just imagine how much cleaner the mount interface could be... ;) http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
According to Alexander Viro: > On Wed, 13 Dec 2000, Chip Salzenberg wrote: > > According to Alexander Viro: > > > On Wed, 13 Dec 2000, Chip Salzenberg wrote: > > > > According to Alexander Viro: > > > > > 9P is quite simple and unlike CORBA it had been designed for taking > > > > > kernel stuff to userland. Besides, authors definitely understand > > > > > UNIX... > > > > > > > > As nice as 9P is, it'll need some tweaks to work with Linux. > > > > For example, it limits filenames to 30 characters; that's not OK. > > > > > > For RPC-style uses? Why? > > > > For the same reason C compilers recognize more than eight significant > > characters in externals, even though ANSI doesn't require them to. > > s/30/255/ and you've got a big problem with ext2... As long as names are to be created, or at least understood, by humans, there will be some limit on *usable* length. In my experience, 255 is above that limit, but 30 is below it. And I cut my teeth on a system that had exactly that length limitation (UNOS). -- Chip Salzenberg- a.k.a. -<[EMAIL PROTECTED]> "Give me immortality, or give me death!" // Firesign Theatre - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Wed, 13 Dec 2000, Chip Salzenberg wrote: > According to Alexander Viro: > > On Wed, 13 Dec 2000, Chip Salzenberg wrote: > > > According to Alexander Viro: > > > > 9P is quite simple and unlike CORBA it had been designed for taking > > > > kernel stuff to userland. Besides, authors definitely understand > > > > UNIX... > > > > > > As nice as 9P is, it'll need some tweaks to work with Linux. > > > For example, it limits filenames to 30 characters; that's not OK. > > > > For RPC-style uses? Why? > > For the same reason C compilers recognize more than eight significant > characters in externals, even though ANSI doesn't require them to. s/30/255/ and you've got a big problem with ext2... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Wed, 13 Dec 2000, Chris Lattner wrote: > Which is exactly why it doesn't work well for many applications. The > problem is this: how do you get from a byte stream to a structured data > stream? There are many answers: > > 1. Keep your data structures so simple, that it's obvious. Not a good > choice. :) > 2. Defined interfaces to the bytestream for every interface that you > define. > 3. Define a standard for inflating/deflating "things" into > bytestreams. Oh wait, that's what corba does. :) OK, now I'm completely confused. * which complex data structures do you want to export from the kernel in non-opaque way? * which of those structures are guaranteed to remain unchanged? * if you have userland-to-userland RPC in mind - why put anything marshalling-related into the kernel? > The 9P way of doing things is not fundementally new, they just applied the > idea that "everything is a file" more broadly. What annoys me is that it > is not immediately obvious how to "demarshall" the data that comes out of > /dev/mouse for example. Combine that with the problem that /dev/mouse > might change format in the future (okay stupid example, but you get the > idea) to use floating point coordinates, and things certainly get hairy. HUH? OK, suppose it had happened. Do you really expect that you will not need to change your applications? I mean, if you expected a bunch of ints and received a bunch of doubles... You either need to decide on rounding (and it's a non-obvious question) or you need to change quite a bit of code in your program. It goes way past the demarshalling, no matter whether you use CORBA, 9P or printf/scanf. OK, suppose you have a CORBA-based system and mouse drivers' API had been changed - they really want to return floating point coordinates. How will CORBA help you? Aside of making your programs scream aloud, that is. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Wed, 13 Dec 2000, Chip Salzenberg wrote: > According to Alexander Viro: > > 9P is quite simple and unlike CORBA it had been designed for taking > > kernel stuff to userland. Besides, authors definitely understand > > UNIX... > > As nice as 9P is, it'll need some tweaks to work with Linux. > For example, it limits filenames to 30 characters; that's not OK. For RPC-style uses? Why? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Wed, 13 Dec 2000, Chris Lattner wrote: > > > > either. Oops, wasn't interoperability an important part of the Linux > > > kernel design? Didn't we want to use and follow and define _real_ > > > standards? > > Erm... 9P stub exists for Linux. It exists for FreeBSD. I suspect that > > it exists for other *BSD too - never checked that. > > Okay, so there are _stubs_ for these platforms. How many languages are > there bindings for? Grr... Let's define the terms, OK? What is available: kernel code that represents the client side of RPC as a filesystem. Userland clients do not know (or care) about the mechanisms involved. > > You mentioned OOP, didn't you? Encapsulation is a good thing and what you > > are talking about is "layering violations made Real Easy(tm)". > > I simply don't see why _that_ is a good goal. > > I completely fail to see how I'm breaking encapsulation or making layering > violations easy. Why I'm talking about is the fact that CORBA doesn't > care how you ship bytes around: it's a higher level protocol that cares > about shipping "things" around. Bytes are old school, "stuff" is the wave > of the future. ;) heh. And files with structure are things of dreadful past. BTDT. You really need to... work with an OS that would have and enforce "structured files" to appreciate the beauty of ASCII streams. However, that's a different story. What I _really_ don't understand is the need to export anything structured from kernel to userland. IOW, I would really like to see a description of use of your mechanism. If it's something along the lines of "let's take a network card driver, implement it in userland and preserve the current API" - see the comment about layering violations. You've taken an internal API and exposed it to userland in all gory details. See also your own comment about internal APIs being not convenient for such operations. If it's something else - I wonder what kind of objects you are talking about and why opaque stuff (== file descriptors) would not be sufficient. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
According to Alexander Viro: > 9P is quite simple and unlike CORBA it had been designed for taking > kernel stuff to userland. Besides, authors definitely understand > UNIX... As nice as 9P is, it'll need some tweaks to work with Linux. For example, it limits filenames to 30 characters; that's not OK. -- Chip Salzenberg- a.k.a. -<[EMAIL PROTECTED]> "Give me immortality, or give me death!" // Firesign Theatre - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > > plan-9.bell-labs.com/sys/man/ > Arrgh. s/plan-9/plan9/. My apologies. Cool, thanks, will read. :) > IDGI. What 9P gives is an RPC mechanism that uses normal (as in "named streams > of characters") representation on the client side and very light-weight > library on the server side. It looks like you are trying to do a mechanism > that would export arbitrary _internal_ kernel APIs. I really don't see > what you would want it for. Which is exactly why it doesn't work well for many applications. The problem is this: how do you get from a byte stream to a structured data stream? There are many answers: 1. Keep your data structures so simple, that it's obvious. Not a good choice. :) 2. Defined interfaces to the bytestream for every interface that you define. 3. Define a standard for inflating/deflating "things" into bytestreams. Oh wait, that's what corba does. :) The point of the matter is that CORBA gives you a nice, clean, well supported method of not having to write marshalling/demarshalling code every time you want to use an interface. Not only that, it lets you define "fast paths" for special cases (ie, with user level corba, if you are communicating between two objects in the same processes, you can do direct calls and do no marshalling). These fast paths do not interfere with the generality of the system at all. The 9P way of doing things is not fundementally new, they just applied the idea that "everything is a file" more broadly. What annoys me is that it is not immediately obvious how to "demarshall" the data that comes out of /dev/mouse for example. Combine that with the problem that /dev/mouse might change format in the future (okay stupid example, but you get the idea) to use floating point coordinates, and things certainly get hairy. Why reinvent the wheel countless times? -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
On Wed, 13 Dec 2000, Chris Lattner wrote: > > plan-9.bell-labs.com/sys/man/ Arrgh. s/plan-9/plan9/. My apologies. > Err... yeah, so you're effectively mapping UNIX/POSIX across 9P. That's > not very creative, and you could do the same thing with CORBA. I ask > again, "How much development infrastructure is there for 9P?". If you say > "just use unix", then what is the point of 9P at all? (on linux). Linux > already has most of posix (and some would claim all of the "good > stuff" in posix.). IDGI. What 9P gives is an RPC mechanism that uses normal (as in "named streams of characters") representation on the client side and very light-weight library on the server side. It looks like you are trying to do a mechanism that would export arbitrary _internal_ kernel APIs. I really don't see what you would want it for. Notice that exporting a kernel API is possible right now and you don't need _any_ RPC for that. Check procfs/devfs/shmfs/etc. RPC comes into the game when you want to export the piece of namespace to other box or when userland server wants to export its stuff into namespace. IOW, you don't export internal interfaces via RPC, you publish them as a filesystem and then use RPC if you want to access it from another box. Where it looks like filesystem, again. _Another_ issue is with the stuff like plumber when you want to talk with userland program via filesystem, but you seem to be talking about something that looks like the first kind of situation... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Korbit-cvs] Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > either. Oops, wasn't interoperability an important part of the Linux > > kernel design? Didn't we want to use and follow and define _real_ > > standards? > Erm... 9P stub exists for Linux. It exists for FreeBSD. I suspect that > it exists for other *BSD too - never checked that. Okay, so there are _stubs_ for these platforms. How many languages are there bindings for? What about Win32 (gross I know, but at least for now, it is important)? What about OS/2 what about Amiga and palm? CORBA is very popular for certain things... even inferno isn't nearly as big... > > Heh, that's interesting. _getting_ the data isn't hard in 9P, actually > > using it or displaying it in a meaningful way commonly requires something > > more than cat. :) > Same for any other RPC mechanism. Of course. Which is why CORBA is about putting STRUCTURE in that stream of random bytes coming over the wire. Why should I have to rewrite my marshalling and demarshalling code every time I want to write a server. read and write are fine. But sometimes I want a structure. Sometimes, my structures aren't laid out like C struct's either. What then? What if I want to send an "object" to you? > > nothing enforces it. 9P works wonderfully for Plan9 because they had the > > luxery of redefining/rewriting the whole OS, and the whole interaction > > with user space processes. We, unfortunately, don't have that > > possibility. :) > Notice that they _don't_ export random internal APIs to userland. And neither do we. Beautiful isn't it? :) > > Please don't get me wrong. I'm not opposed to other ideas, and 9P may in > > fact turn out to be a very nice protocol that would be able to support > > much kORBit level functionality). I do maintain that by writing a custom > > user->kernel marshalling library, you could obtain better peformance than > > 9P though, because you could take advantage of lots of machine specific > > optimizations. Hell you could even pass things in register if you'd have > > <= 4 args. :) > > You mentioned OOP, didn't you? Encapsulation is a good thing and what you > are talking about is "layering violations made Real Easy(tm)". > I simply don't see why _that_ is a good goal. I completely fail to see how I'm breaking encapsulation or making layering violations easy. Why I'm talking about is the fact that CORBA doesn't care how you ship bytes around: it's a higher level protocol that cares about shipping "things" around. Bytes are old school, "stuff" is the wave of the future. ;) heh. -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ANNOUNCE: Linux Kernel ORB: kORBit
> > Err shame on you, don't forget about lcall and exceptions, and interrupts, > > and... That is technically more than _o_n_e_ "entry point". :) Oh wait, > > what about sysenter/exit too? :) > OK, you got me on lcall (however, that's iBCS-only, IIRC), but the rest... > what the hell does userland to interrupts? OK, make it 2 - pagefault > can be arguably used in that way. The reason that I considered exceptions and interrupts is that often, exceptions get reflected as signals to the running processes (SIGSEGV, SIGFPE, SIGILL, others?), and interrupts can wake up processes (from sys_poll among others). I was considering more of the user->kernel and kernel->user transitions... anyways, that's really besides the point. :) > > this wonderful design we get all kinds of stuff like sys_oldumount vs > > sys_umount and others... > Check how often anything uses the majority of that stuff... Correct, it's for backwards compatibility with old programs (for example libc5 uses a lot of those "old" syscalls). > > > Yes, standard RPC mechanism would be nice. No, CORBA is not a good > candidate - > > too baroque and actually known to lead to extremely > tasteless APIs being > > implemented over it. Yes, I mean GNOME. So > Check 9P and compare. Really. Section 5 of Plan 9 manpages. Available on > plan-9.bell-labs.com/sys/man/ That's fine. Since the server is down (or the URL is bad), can you please give me an example of how 9P is better than CORBA? I freely admit to not knowing much about 9P... how much do you know about CORBA (aside from your opinion that GNOME uses it, and therefore it is bad. ;)? > > without breaking backwards compatibility). Please don't tell me that OOP > > is bad... or else we will have the eviscerate the VFS layer from the > > kernel (amount other subsystems)... :) > OOP is a nice tool. However, it's a tool that has incredible potential of > shooting one's foot. It's wonderful if you have sane set of methods. And > that's a _big_ if. "Easily extensible" is not an absolutely good thing - > C++ wankers all over the world are busily proving it every day. Heck, they > make a living out of that. IOW, the problem with interface changes is _not_ > in converting the old code. It's in choosing the right changes. And that > part of the game can't be simplified. Oif. That's like telling someone that C is evil because it has for loops, and for loops can be used to write nasty code. "just write in assembler" he says. :) I would claim that someone could write a bad program (or shoot themselves in the foot) with any turing complete language. C++ definately give you more rope to do that with, but used wisely, it can also be nice. The trick is to just not have to work with other peoples C++ code. :) Hey, did I mention that kORBit and all its extensions are written in C? :) > > Like I mentioned in a previous email, CORBA does not preclude 9P. What > > it does buy you though, is compatibility with LOTS of preexisting CORBA > > tools. How much development infrastructure is there for 9P? I thought > > so. :) > All UNIX userland on the client side. lib9p on the server side (23Kb of sparse > C). Examples of use in servers - see the aforementioned site. Err... yeah, so you're effectively mapping UNIX/POSIX across 9P. That's not very creative, and you could do the same thing with CORBA. I ask again, "How much development infrastructure is there for 9P?". If you say "just use unix", then what is the point of 9P at all? (on linux). Linux already has most of posix (and some would claim all of the "good stuff" in posix.). > > For one of our demos, we ran a file server on a remote linux box (that we > > just had a user account on), mounted it on a kORBit'ized box, and ran > > programs on SPARC Solaris that accessed the kORBit'ized linux box's file > > syscalls. If nothing else, it's pretty nifty what you can do in little > > code... > Duh. And what's new about that? The "new" part is that our servers were < 100 lines of code each. Compare that to kNFS. :) -Chris http://www.nondot.org/~sabre/os/ http://www.nondot.org/MagicStats/ http://korbit.sourceforge.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/