On Tue, Jun 17, 2014 at 3:16 PM, Antti Kantee <[email protected]> wrote: > Hi, > > So we started thinking about what rump kernel build scripts should > really look like. This mail consists more or less of unfinished > brainstorming we had with Justin on #rumpkernel, but since the topic is > difficult to fully grasp, I'll post an intermediate summary. If someone > wants to engage on difficult thinking on the subject, I'll be delighted > to hear your thoughts, no matter how unpolished. > > To understand how the situation we need to fix came about, remember that > buildrump.sh started more or less as an easy way to document the answer > to the back-then popular email question I got: "is it possible to run > the TCP/IP stack in Linux userspace". This approach immediately put a > skew on buildrump.sh: it was meant to address running rump kernels in > userspace. > > Things started piling up: we got for example the ability to run rump > kernel directly on top of the Xen hypervisor, Genode, and are looking to > expand into various IoT-type bare-metal systems. Therefore, basing the > central piece of the build-technology on "how to run in Linux userspace" > is starting to be more than a little outdated. We're seeing the > problems for example in rumprun and rumpuser-xen in the form of awkward > build scripts. > > There are three different, mostly independent layers of building blocks. > I'll list them first, and then give a few examples to clarify. > > 1: the rump kernel itself, and tools for building the kernel components. > this layer will only include the kernel-side components with no > hypercall components. after a fashion, it's more or less dependent only > on the ABI that the (cross)compiler targets. this is more or less > currently handled by buildrump.sh -k > > 2: full application stack support. this means compiling the NetBSD libc > and assorted other userspace components on top of a rump kernel. > > 3: platform specific details. here we are talking about for example the > hypercall implementation and how to build applications for the platform > -- it differs quite a lot for example between userspace and Xen. Also > relevant is how to hide the platform namespace from the application part > of the rump kernel stack (cf. what rumprun does) > > 4: the application itself > > So, for example, the running the current buildrump.sh is more or less > like 1+3+4: it builds the kernel components and userspace bits. > > rumpuser-xen is a bit like 1+2+3+4: it uses buildrump.sh to build the > kernel components and using the tools of buildrump.sh, the libs and apps > (2) are built. The process for building the Xen image is private to "3". > > So the purpose of the revamp is to produce building blocks that conform > to the above scheme. It will also hopefully help with the packaging > tragedy. Ideally, I'd like to get this done within the coming few > weeks, but that's probably a too tight schedule. > > I'd ask for thoughts, but I don't have that many myself ;)
Some somewhat /dev/random thoughts: 1. splitting out 1 would mean you could do sane things like build kernel with -nostdinc so you can be sure there is no leakage from host - you can build -k like that now, but rumpuser has host deps 2. buildnb.sh from rumprun is really hard to make reuse a build from buildrump as we have seen recently. It seems to make sense to basically merge much of it into buildrump.sh. This has downsides (managing the source, as having a nice small source for buildrump is good, while I can see buildnb having a pretty big source. Plus rumprun still has patches for now. It might be easier to add the kernel sources into the userspace sources, so you could choose either the kernel only source tree or the kernel+userspace tree) 3. I am about to start working on my hypercall implementation thats non pthread based, which is a sort of hybrid between xen and posix. So there will be another general purpose version. I really dont want to make yet another repo with a linked copy of buildrump... 4. I would quite like to take the subrepo copy out of ljsyscall and just test against external versions, should do that anyway. 5. I guess if rumpuser was split out, its master copy could be outside NetBSD, with NetBSD just having an import (still in base as needed for tests). This would make some more sense in a way as it is not in any way NetBSD specific, and (was looking at this today) despite the comments at the top of rumpuser_port.h about maybe using the generic NetBSD compat, that is not really feasible, as it uses way more stuff, rather than just minimal Posixy stuff (eg might want to merge my aio block dev implementation). 6.The IO hypercalls are an issue, I wonder if they could be made more generic. I had some thoughts on this a while back, will look for them. Justin ------------------------------------------------------------------------------ HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing & Easy Data Exploration http://p.sf.net/sfu/hpccsystems _______________________________________________ rumpkernel-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/rumpkernel-users
