Antti, At Wed, 07 Jan 2015 18:02:47 +0000, Antti Kantee wrote: > > On 07/01/15 07:48, Hajime Tazaki wrote: > > > > hi, > > > > I'm working on userspace network stack of Linux, called > > NUSE. > > > > as I found NUSE and rumpkernel has a lot of commons, I > > believe sharing development effort would be a nice way to > > survive (not only for ourselves I believe). > > > > ultimate goal for this integration is to provide all > > required interface of rumpkernel for Linux kernel, > > That would be very nice indeed! > > Especially scheduling was very difficult to figure out and equally > difficult to implement, so I warmly encourage you to reuse that code > where possible and those principles where the code doesn't directly > apply. (nb. scheduling in a rump kernel is still missing support to pin > the virtual core to be the same as the physical core, maybe we can > collaborate on that part later?)
the collaboration is always fun. my current implementation of hyp.hyp_schedule() is NOP, which I haven't investigated what I (i.e. Linux) should care. probably later on, I will start to work on this more deeply: since system calls used by configuration command (i.e., netlink and ioctl) to the kernel works fine so far, I don't face any issues at this moment. more code reading and hacking (and playing) required. > > though the current code needs really small tricks (a few > > LoC) to src-netbsd, rumpserver looks fine with other guest > > kernel (i.e., linux). > > > > temporary codes are available: > > > > https://github.com/libos-nuse/net-next-nuse/tree/syscall-proxy > > https://github.com/libos-nuse/src-netbsd/tree/buildrump-src-nuse > > Can you post diffs for the changes under discussion? I might be able to > guess by looking at the repo, but it's always easier to not have to guess. for the NUSE part, diff is here. https://github.com/libos-nuse/net-next-nuse/commit/0a33b1a141133e2e33eb97f1cdd3ea3d5696188d for the src-netbsd part, https://github.com/libos-nuse/src-netbsd/compare/rumpkernel:buildrump-src...9961fd6528e2bb056cbca53c02bb9e1a763e333e?diff=unified&name=9961fd6528e2bb056cbca53c02bb9e1a763e333e or https://gist.github.com/thehajime/e6c32b681ca371b0a96a # if this forum prefers to paste raw (inline) patchset, I will follow your policy. > > - is src-netbsd (branch buildrump-src) the appropriate > > repository for this purpose ? > > It's supposed to be a mirror of NetBSD sources. Of course that applies > only to repo.rumpkernel.org/src-netbsd, and I cannot tell other people > what they can or cannot provide in their forks. That said, the current > proper method of getting something into src-netbsd is to submit it to > NetBSD and subsequently update the mirror. If you think you have > something that looks ready to upstream, we should just discuss + > integrate that part. right now, I have no reason to upstream my code (of src-netbsd changes) because it's really a gimmick. see my above diffs of src-netbsd, which uses thread local storage (horrible..) to store spc_client instance. some changes are only adding symbolic links for include path so, that would be removed if I can improve my Makefile... > > I only use part of src-netbsd (librumpuser, librumphijack, > > librumpclient, and sys/rump/librump/rumpkern with a couple > > of .h files. sys/rump part should be implemented with > > Linux-specific syscall implementation: for future work). it > > would be nice that I can propose a standalone toolkit for > > such an integration. > > Are you saying that the toolkit should consist of > librump{user,hijack,client} and rumpuser.h? That's not a bad idea for > something to investigate. something like that. if someone tries to apply the rumpkernel idea with their own OSs (as I'm doing), such toolkit will be helpful. > Note that we have already been considering moving the development of the > posix librumpuser currently in src-netbsd to an external repo. okay. > Rather than having patches in src-netbsd, maybe you can create a new > repo and copy the things you need. I'm not quite sure how the work > would flow in that case, so needs a bit of thinking. Too many repos ;) i concur ;) > > - failure in hijacking > > > > in hijack.c, system calls are intercepted by LD_PRELOAD, as > > NUSE does. in Linux glibc, some of the functions are using > > strong alias (I guess: need to investigate more) which we > > cannot hijack by LD_PRELOAD. for example, if_nametoindex(3) > > uses '__socket' symbol, which results in inline system calls (we > > can't replace it by our own?). does NetBSD glibc have such a > > system call ? if yes, does librumphijack take a special care? > > What's an "inline system call"? I tried to read the glibc sources, but > not sure what that means, looks like __socket is a symbol just like > anything else, except that it's the libc internal name. I guess Justin said all. for more information. https://sourceware.org/glibc/wiki/SyscallWrappers https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/x86_64/sysdep.h;h=3dbd7d2be4e0ca261592e7e15993ffaebb61b9bd;hb=HEAD#l234 > On NetBSD, librumphijack hijacks some internal libc symbols, e.g. it > hijacks _sys_write, not write. That just works. I didn't do very much > testing with glibc when I was on that some years ago, so it's possible > that things will just not work. I envy you ;) > > if not then, is ptrace(2) only a solution for this ? > > Well, ptrace() is different. The reason I chose LD_PRELOAD is that it > keeps the policy in the client, which seemed like the right solution, > and actually still does. ptrace() is a better option for a usermode OS > type of scenario where the kernel wants to keep control over its clients. make sense, and I like it (about who is the king). > If you want to know more about ptrace in a similar scenario, I'd suggest > looking at what ViewOS is doing with ptrace and utrace. thanks. I already looked a bit on UML, another ptrace user. > [quoting Justin's mail to the thread] > > The best option is probably to compile a new libc which replaces the > > syscalls with library calls. That is what the rumprun-* code does, and > > that guarantees you will catch all the calls. > > Heh, well, um, it did take some years of basic hacking to reach a stage > where that became feasible... I think getting hijack limping along will > open a significant number of development avenues, even if hijacking is > not perfect. A full libc route might not open up avenues that quickly > in the beginning, even if it will be more perfect method in the end. well, having several options to tackle the problem is always nice. I will postpone to conclude this for a while. > > - rump local, or remote, or embedded local/hijack ? > > > > current NUSE process model (without rumpserver) is like > > 'rump local' mode with hijacked calls. is there any possible > > combination like this (hijacked rump local mode) in rumpkernel ? > > Yes, it should be possible (and even sensible in high-performance > cases), but nobody just got around to doing it. Note that rumprun-posix > also provide a sort of "local hijacking", except that it doesn't do > hijacking (hope that makes sense ;). I'm a bit puzzled ;) I will look at code of rumprun-posix. objcopy ? (now I glanced the code) > For a very long time (until ~2-3 years ago), rump kernels were not about > high performance at all. Then people started asking about things like > using the TCP/IP stack for high-performance userspace packet processing. > Things which were designed/written before that have not all caught up yet. > > Also: > https://github.com/rumpkernel/buildrump.sh/issues/63 thanks for the info. now I'm relieved I'm not only the guy ;) > > have a nice day. > > > > # I'm going to talk about this stuff in a Linux specific > > conference, netdev0.1. I plan to address the rump server > > integration in my talk. FYI. > > https://www.netdev01.org/news/18 > > Cool, let's try to get things to a good shape by then! will let you know once I've come up with. -- Hajime ------------------------------------------------------------------------------ Dive into the World of Parallel Programming! The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net _______________________________________________ rumpkernel-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/rumpkernel-users
