Thanks for the detailed input, Norman. I note your successes and
respond to your concerns; real world feedback is most interesting and
valuable for me.
On 23/01/15 12:41, Norman Feske wrote:
> There are still some concerns that call for optimizations. First, for
> our use case for re-using merely a specific kernel subsystem (like a
> file system driver), the supporting infrastructure required around the
> subsystem is quite large so that our Rump-based file-system component
> contains a lot of "support code" in addition to the "feature code".
> E.g., we have to carry with it substantial parts of the NetBSD kernel
> such as the VFS or the block cache (we would really love to handle the
> block cache by a separate component instead).
The problem there is that file system implementations are very
intertwined with the vfs and block/page cache interface semantics. For
example, blocks must be written in correct order to retain file system
consistency, and some blocks must not be written before other are.
Furthermore, there's no global right way to do it, and e.g. ext2 and
journalled FFS have slightly different kinks.
Notably, though, the very early versions of rump kernels ca 2007 (called
"RUMP" back then) did run with just a shim version of block/page cache.
This was because rump kernels were only used for running kernel file
system drivers in userspace servers. Since caching was already done in
the host kernel, it was unnecessary in "RUMP". Things worked,
superficially, but it wasn't correct in the sense that a crash could
curdle the file system.
Simply, block/page caches (and furthermore some bits of the VM) are a
part of the current implementations of the file system drivers -- you
even noted something like this in your description of the Linux file
system porting efforts. What we should be able to do fairly easily,
though, is to figure out how to make those layers work as write-through
as possible, so you could have control of the actual caching in a
separate component at/under the block device layer. If that approach
could solve a reasonable chunk of the issue for you, maybe we can
discuss it more in a separate thread or e.g. at FOSDEM.
> Second, since the Rump
> kernel spawns one host thread for each NetBSD kernel thread, the
> component's footprint with respect to the usage of threads is much
> higher than it could be. But those things do not at all diminish the
> value that Rump kernels provide to us.
There's no mandate to create a host thread per se. You need to create a
separately schedulable entity with a stack and thread-local storage, but
if you choose to implement that multiplexed on top of a single host
thread, that's fine. Due to the way the NetBSD kernel is structured,
they do need to be independently runnable, i.e. t1 must be able to run
even if t2 is blocking. Otherwise you'll run into deadlocks ... very
non-obvious deadlocks ... everywhere. Notably, the non-kernel threads
accessing the rump kernel must be under the same scheduler architecture
as the kernel threads. Otherwise, say a non-kernel thread calls the
rump kernel, and blocks. Also assume that unblocking that thread
depends on a kernel thread running. The scheduler must have knowledge
that it needs to run the kernel thread. The same story applies when the
kernel thread has finished running and the user thread should run again.
A thread itself should be reasonably cheap (~10kB memory including max.
stack consumption). So, the 10-or-so kernel threads will set you back
~100k, a non-trivial amount if you run hundreds of rump kernel servers,
but probably not the major cost. If you can be more specific about what
the "thread footprint" is, I can at least think of how to reduce it to
an acceptable level.
Anyway, the concerns that Genode brings are interesting, because due to
the componentized nature, they are different from the concerns that
unikernels bring. While it is true that the codebase isn't infinitely
flexible, and you cannot swap out any component in the stack as you
please (*), I argue that with good engineering choices we should be able
to land somewhere around "acceptable".
*) and even if you could, the interfaces between components are so
complex that it's extremely difficult to produce a correct
implementation ("been there, done that (and failed)"). I guess that's
what happens with 30 years of history in the fashion of "hey I need this
bit in that interface to do this thing". It's easy to do it once when
implementing the interface, and sort of easy to use that interface, but
it's highly unlikely that anyone will produce another "bug compatible"
implementation of that interface.
> When it comes to ext4, we still need to look elsewhere because ext4 is
> not supported by the NetBSD kernel. Hence, for now, we are mostly using
> ext2.
ext4 (and ZFS) would be nice to have in the fs portfolio indeed. The
good news is that if they become supported on NetBSD, they'll be
automatically and instantly supported by rump kernels. The bad news is
that they, like most drivers today, are so non-trivial that it's
unlikely that anyone will just sit down and decide to hack things up.
- antti
------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet
_______________________________________________
rumpkernel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/rumpkernel-users
