On Thursday, 10.09.2015 at 01:58, Antti Kantee wrote:
> Hi,
>
> For a while I've been talking about how the rumpbake-rumprun axis is not
> quite right. I think it's now clear, at least to me, that the problem stems
> from the launch tool being modelled after Docker. It was a smart decision
> at the time, but the problem we've discovered since is that it was made with
> too little case data (Xen xl toolchain only). You start from somewhere
> reasonable-looking, you learn, and now we've learned that we have too many
> targets completely unlike Docker. I'd even go so far as to state that
> "Docker-like" is the special case. What goes wrong is that the "rumprun"
> tool assumes it can both create a configuration and launch the guest. The
> former is always true (though embedding the config into something launchable
> is sometimes awkward), the latter is not.
While I agree that the current situation is not quite right, I don't
think that "modelling the rumprun tool after Docker" was a bad idea. My
main motivation for doing so when I started working on the rumprun tool
was ease of use during the development and testing cycle for unikernels.
I believe that the #1 target audience for the rumprun tool are
developers, working and deploying locally on a Xen/KVM system. This is
not the same as the target audience for the rumprun unikernel in
general, or rumprun unikernel packaged applications.
I think that the current situation comes from an increased scope, where
a one-size-fits-all rumprun tool is no longer the right/whole answer.
Therefore, I think that we should a) decrease the scope of the rumprun
launch tool, focusing on the original target audience, while at the same
time b) document and improve the configuration specification and entire
boot process of a rumprun unikernel so that others can either manually
deploy rumprun unikernels as they please or implement/integrate with
their own tools.
> So let's try to figure out how to fix the situation. So what do we actually
> want?
>
> 1: a distributable format which does not require the toolchain to launch
> (what rumpbake currently gives you)
> 2: a mechanism to configure the runtime behaviour of the distributable
> format (what the rumprun tool currently does)
> 3: a mechanism to easily launch the result of 1+2 *where available* (what
> the rumprun tool does for xen+kvm+qemu)
> 4: a mechanism to "specialize" the distributed format, e.g. "baked in root
> files" or even including "2" (to enable running without block storage)
>
> So then to the adjustment proposals.
>
> I don't think there's any reason to touch "1". We need some further
> adjustments for component management a la rumpbake.conf, but that's mostly
> orthogonal.
Agreed, mostly. I was going to write about where I think our
"distributable format" should head, but that's more about packaging
rumprun unikernel applications which is a separate discussion.
> Ok, that was easy, let's move to "2", config creation. There are actually 2
> different configs, the Rumprun runtime config and the config of whatever you
> launch on. We can't always control the latter from software, consider e.g.
> the case where you're launching on an embedded system, so this bit is only
> about the Rumprun config. We now use a json config. I'm thinking that the
> best option is to not provide a config generating tool at all, simply polish
> the json format spec and be happy with it.
>
> Then, "3" or launching. Now, there needs to be a congruence between your
> config from "2" is and what you launch on. The current rumprun tool sort of
> attempts to help you there, but with anything except a trivial setup you
> need to know what you're doing anyway, e.g. "-I ,,'-net tap,ifname=tap0"
> constructs and so forth. So the tool is not *really* catching you because
> it cannot read your mind. Do we need a quasi-abstracting tool? I'm going
> to say "no". The real eye-opener here for me was working on EC2. We simply
> have no way to sensibly abstract the 3 billion toggles available via EC2.
> Even for trivial systems like xl.conf (when compared to EC2), if you know
> xl.conf you'll just have to learn a second syntax to do what you already
> know. If you don't know xl.conf, we can provide an example or two. Not
> sure if we should provide some case-specific helper scripts, though nothing
> which pretends that everything is the same, hides power and throws off
> people who'd know how to use the relevant backend tools.
I'm going to split my reply into separate sections targetting what I
think are different audiences for the rumprun unikernel:
1) Rumprun developers (us), testers, application porters and generally
anyone who just wants to "try things out" on their laptop/handy server:
(For the sake of clarity, I'm going to dub the current rumprun tool
'rumplaunch', since the name is overloaded by the both the tool and the
unikernel project as a whole. Feel free to pick a better name.)
As I wrote above, I think that rumplaunch should stay but its scope
should be reduced to development, testing and deployment on a local
Xen/KVM/(bhyve, etc in future?) system. No such tool currently exists
and we need one simply to develop the rumprun unikernel!
If we reduce its scope and target audience, we can also improve the
abstractions provided:
- Provide a "template" network configuration for KVM/QEMU which users
can install. Such a configuration could be as simple as "create a
rumplaunch0 bridged network, run dnsmasq on it and NAT to the
outside world".
- With such a configuration in place, the *default* network setup for
unikernels run with rumplaunch becomes "just do DHCP". Bam, the
whole "-I ,,'-net tap,ifname=tap0" nonsense just went away.
If you want to do custom stuff that the rumplaunch tool doesn't provide
for, you can always do it manually using the relevant backend tools (xl,
qemu).
2) People who want to run rumprun unikernels on classic "full VM
hypervisors" today. This includes both desktop hypervisors such as
VirtualBox, VMWare Workstation and cloud hypervisors such as EC2, GCE or
Azure:
We should provide a 'rumpbundle' tool (again, tenative name) which
(ec2ivol problems notwithstanding) takes a rumprun unikernel, possibly
including application data and minimal configuration and spits out a
full machine image suitable for loading. Nothing else, i.e. we do not
attempt to actually perform the "import-register-run" dance for cloud
hypervisors.
This is more or less what the current "rumprun iso" and "rumprun ec2"
functionality already does.
3) Everyone else. This includes embedded systems, people for who
rumplaunch is not enough, rumprun based Xen stubdom, etc:
We provide documentation on:
- the unikernel-side JSON configuration spec.
- exactly how the configuration data is passed to the unikernel (the
current ROOTFSCFG=, or on the bootloader-supplied command line, Xen
PV command line, ...).
This should be enough for people to integrate rumprun unikernels any way
they see fit.
> After some thinking, the solution to "4" is obvious. They are simply
> components. Want a non-default /etc? Use a different component (see now
> why rumpbake.conf needs more flexibility/power?). Want to include the
> config too? Again, a component. (We probably need some tools to generate
> the components, but that's an entirely different discussion)
Agreed. So the work I did some time ago on baked-in-files, were it to
spit out a linkable component containing a MFS filesystem, would that be
a good start?
> Thoughts? Take your time, this is not a rush job, but let's say would be
> nice to have the rototill done by or latest in October.
I think that might be a bit optimistic, given that I spent about
three days just thinking and drafting this email :-/
-mato
