On 07/01/15 07:48, Hajime Tazaki wrote: > > hi, > > I'm working on userspace network stack of Linux, called > NUSE. > > as I found NUSE and rumpkernel has a lot of commons, I > believe sharing development effort would be a nice way to > survive (not only for ourselves I believe). > > ultimate goal for this integration is to provide all > required interface of rumpkernel for Linux kernel,
That would be very nice indeed! Especially scheduling was very difficult to figure out and equally difficult to implement, so I warmly encourage you to reuse that code where possible and those principles where the code doesn't directly apply. (nb. scheduling in a rump kernel is still missing support to pin the virtual core to be the same as the physical core, maybe we can collaborate on that part later?) > [snip] > it just works fine !!! (on FC19 64bits) > > # in fact, it took 1 month to reach here, not just works > btw. ;) > though the current code needs really small tricks (a few > LoC) to src-netbsd, rumpserver looks fine with other guest > kernel (i.e., linux). > > temporary codes are available: > > https://github.com/libos-nuse/net-next-nuse/tree/syscall-proxy > https://github.com/libos-nuse/src-netbsd/tree/buildrump-src-nuse Can you post diffs for the changes under discussion? I might be able to guess by looking at the repo, but it's always easier to not have to guess. > - is src-netbsd (branch buildrump-src) the appropriate > repository for this purpose ? It's supposed to be a mirror of NetBSD sources. Of course that applies only to repo.rumpkernel.org/src-netbsd, and I cannot tell other people what they can or cannot provide in their forks. That said, the current proper method of getting something into src-netbsd is to submit it to NetBSD and subsequently update the mirror. If you think you have something that looks ready to upstream, we should just discuss + integrate that part. > I only use part of src-netbsd (librumpuser, librumphijack, > librumpclient, and sys/rump/librump/rumpkern with a couple > of .h files. sys/rump part should be implemented with > Linux-specific syscall implementation: for future work). it > would be nice that I can propose a standalone toolkit for > such an integration. Are you saying that the toolkit should consist of librump{user,hijack,client} and rumpuser.h? That's not a bad idea for something to investigate. Note that we have already been considering moving the development of the posix librumpuser currently in src-netbsd to an external repo. Rather than having patches in src-netbsd, maybe you can create a new repo and copy the things you need. I'm not quite sure how the work would flow in that case, so needs a bit of thinking. Too many repos ;) > - failure in hijacking > > in hijack.c, system calls are intercepted by LD_PRELOAD, as > NUSE does. in Linux glibc, some of the functions are using > strong alias (I guess: need to investigate more) which we > cannot hijack by LD_PRELOAD. for example, if_nametoindex(3) > uses '__socket' symbol, which results in inline system calls (we > can't replace it by our own?). does NetBSD glibc have such a > system call ? if yes, does librumphijack take a special care? What's an "inline system call"? I tried to read the glibc sources, but not sure what that means, looks like __socket is a symbol just like anything else, except that it's the libc internal name. On NetBSD, librumphijack hijacks some internal libc symbols, e.g. it hijacks _sys_write, not write. That just works. I didn't do very much testing with glibc when I was on that some years ago, so it's possible that things will just not work. > if not then, is ptrace(2) only a solution for this ? Well, ptrace() is different. The reason I chose LD_PRELOAD is that it keeps the policy in the client, which seemed like the right solution, and actually still does. ptrace() is a better option for a usermode OS type of scenario where the kernel wants to keep control over its clients. If you want to know more about ptrace in a similar scenario, I'd suggest looking at what ViewOS is doing with ptrace and utrace. [quoting Justin's mail to the thread] > The best option is probably to compile a new libc which replaces the > syscalls with library calls. That is what the rumprun-* code does, and > that guarantees you will catch all the calls. Heh, well, um, it did take some years of basic hacking to reach a stage where that became feasible... I think getting hijack limping along will open a significant number of development avenues, even if hijacking is not perfect. A full libc route might not open up avenues that quickly in the beginning, even if it will be more perfect method in the end. > - rump local, or remote, or embedded local/hijack ? > > current NUSE process model (without rumpserver) is like > 'rump local' mode with hijacked calls. is there any possible > combination like this (hijacked rump local mode) in rumpkernel ? Yes, it should be possible (and even sensible in high-performance cases), but nobody just got around to doing it. Note that rumprun-posix also provide a sort of "local hijacking", except that it doesn't do hijacking (hope that makes sense ;). For a very long time (until ~2-3 years ago), rump kernels were not about high performance at all. Then people started asking about things like using the TCP/IP stack for high-performance userspace packet processing. Things which were designed/written before that have not all caught up yet. Also: https://github.com/rumpkernel/buildrump.sh/issues/63 > have a nice day. > > # I'm going to talk about this stuff in a Linux specific > conference, netdev0.1. I plan to address the rump server > integration in my talk. FYI. > https://www.netdev01.org/news/18 Cool, let's try to get things to a good shape by then! - antti ------------------------------------------------------------------------------ Dive into the World of Parallel Programming! The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net _______________________________________________ rumpkernel-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/rumpkernel-users
