Antti and fellow rump kernel-ists, Thanks for the quick response! I'm working with Robbie on this.
> The Rumprun unikernel is not intended as a framework, except in the sense > of one for running applications -- posix or raw syscalls -- on top of it. > Rumprun is intended to support applications which use __thread, so it must > be supported, no way around that. > > It sounds to me like you're porting Rumprun, not rump kernels. (besides, I > don't think rump kernels are "ported" so much as "used" -- if you have to > port rump kernels, it means that they failed as a framework) > We are OS developers and have a system (composite.seas.gwu.edu) we've been working on for quite some time. We'd like to execute Rump Kernels (RK) in that system. The environment in which we want RKs to execute is user-level, and uses the system calls of our system to provide many of the services necessary within the rumpuser interface. Our initial plan was to simply implement a library that exports the rumpuser interface. However, as we went through the bmk rumprun unikernel, we found that we could simply reuse quite a bit of that code, while porting some key functionality (context switch, memory image allocation, interrupts, etc...). The fact that the rumprun environment was already somewhat general, supporting both bmk and Xen, also factored into our decision to add a new "architecture" to it, rather than use a clean-slate approach. The system is pretty far into the boot process on Composite, and we've been working on this for a while already. Rump kernels do not use thread local storage unless you ask them to > (CURLWP=__thread). Technically, a rump kernel should never use __thread, > but using TLS for curlwp gives a very measurable performance boost, so I > bent the rules there with an option. > > Rumprun does not explicitly use %gs, though it does use __thread. The > compiler generates the accesses to %gs (on x86, at any rate). If you want > __thread to do something else, you also have the option of patching the > compiler. It's not very convenient, I'll grant you that, but not wanting > to use the mainstream practise never is. > We agree with all of this. In our system, saving and restoring segmentation registers on protection domain switches has a large performance impact (think: fast, frequent IPC), and most of the system uses acceptably fast __thread replacements already. Ideally, we'd like to be able to to redefine the equivalent of the __tls_get_addr function that (conceptually) does the TLS lookup as described in ELF handling for Thread Local Storage ( http://www.akkadia.org/drepper/tls.pdf). I was hoping that something like this function is a library linked with each application, and that we could replace it (e.g. like the C++ runtime, and the OMP library, both of which we've replaced in the past). In both of these cases, the compiler generates function calls to the library which we could provide. I was hoping for the same in the TLS implementation. Looking at the output assembly, the %gs code is inlined into the call-sites, which is certainly a hint that TLS is not supported in the same way as the C++ runtime or OMP, but I was hoping that we simply the result of intelligent link-time-optimization. Maintaining our own version of the compiler is not an option. > You're of course welcome to further expand your motivations and > intentions, and maybe then we can work something out. > Hopefully this provides some context. However, if there is no pluggable library-supported TLS, then it seems like the resolution is to 1. support the use and modification of %gs, or 2. port each part of the system, and each application to use our own TLS facilities. Is this an accurate assessment? Best, Gabe On Sat, Nov 7, 2015 at 8:35 AM, Antti Kantee <[email protected]> wrote: > On 06/11/15 19:35, Robert Gifford wrote: > >> Good Afternoon, >> >> We are in the process of porting RK to a new host platform and have run >> into a few problems with thread local storage and segmentation in >> general.We do not want to use segmentation in our system and have thus far >> been reimplementing anything in RK that access the gs register. >> >> Our modifications thus far have included removing thread local storage >> within >> libbmk_core/sched.c and librumprun_base/_lwp.c >> Our most recent issues have been due to errno access within src-netbsd. >> > > Those files belong to the Rumprun unikernel, not rump kernels. > > Before we move forward, we were curious to know if there were build options >> to disable segmentation in RK? As of now, we can still continue to move >> forward if this is not a possible, however it is always nice to save a bit >> of time. >> > > Let's clarify a few things, mainly the difference between rump kernels and > the Rumprun unikernel. > > Rump kernels are intended as a framework. That intent means that if > someone has problems in adapting rump kernels to their needs, we'll try to > make it simpler for them. > > The Rumprun unikernel is not intended as a framework, except in the sense > of one for running applications -- posix or raw syscalls -- on top of it. > Rumprun is intended to support applications which use __thread, so it must > be supported, no way around that. > > It sounds to me like you're porting Rumprun, not rump kernels. (besides, I > don't think rump kernels are "ported" so much as "used" -- if you have to > port rump kernels, it means that they failed as a framework) > > Rump kernels do not use thread local storage unless you ask them to > (CURLWP=__thread). Technically, a rump kernel should never use __thread, > but using TLS for curlwp gives a very measurable performance boost, so I > bent the rules there with an option. > > Rumprun does not explicitly use %gs, though it does use __thread. The > compiler generates the accesses to %gs (on x86, at any rate). If you want > __thread to do something else, you also have the option of patching the > compiler. It's not very convenient, I'll grant you that, but not wanting > to use the mainstream practise never is. > > You're of course welcome to further expand your motivations and > intentions, and maybe then we can work something out. > >
