On 16/06/15 13:36, Martin Lucina wrote:
On Tuesday, 16.06.2015 at 10:35, Antti Kantee wrote:
I have no attachment to this particular *implementation*, in fact I'd be
the first to throw it out and replace with something like SquashFS.
I read between the lines that you don't really even want the tarball
method (correct me if I'm interpreting too much).
I don't particularly like the tarball method, but it was the least effort
for me to get working.
The tarball method has another advantage in being able to trivially support
compression (just add zlib, which you are going to have included in the
stack for most things anyhow).
Compression is not particularly useful if the first thing you do is blow
it up in memory (*). It doesn't even save binary transport costs, since
you should just compress the entire binary for transport, not only the
fs part.
*) granted, if the page allocator could handle multiple chunks, we could
then take the region of memory where the compressed part was into use.
But playing with the page allocator is a whole other story, and I'm not
sure if convoluting it for a single use case is smart.
Wouldn't a zero-effort driver implementation have been easier than an
adhoc tarball implementation, i.e. use MFS to mount an in-memory FFS
image? We'd need to bundle newfs and fs-utils, but we want to do that
*anyway* for other purposes. It seems like a double win over adding
something you don't really want.
I don't follow -- MFS is a separate filesystem from FFS. When we looked at
the options with Justin it was not clear if MFS is just a worse kind of
TMPFS or something else?
You should have looked more carefully. MFS is FFS with a memory backend
instead of a disk backend. It only replaces the FFS bmap/strategy
operations. If you mount MFS with an empty FFS, you get something like
tmpfs. If you mount it with a populated one ...
You can literally point the MFS mount routine to a memory (addr,size)
tuple, like in this example I did 5+ years ago:
http://nxr.netbsd.org/xref/src/usr.sbin/puffs/rump_nqmfs/rump_nqmfs.c#69
As for bundling fs-utils and using an in-memory FFS, how suitable is FFS
for in-memory usage? Also, wouldn't bundling fs-utils require yet another
build of rumpsrc?
I don't know what fs-utils would require from the build -- probably what
you say -- but the point is that we need something like it anyway
(extract logs from guest, etc). In the best case scenario fs-utils
would be installed on the host as a binary package.
Therefore, if I want to run a minimal and small[*] service as a rumprun
unikernel in the cloud *today*, and that service does not need any
persistent data, why should I bother with an (external) block device at all?
Ok, good reason. Can you point me to a resource giving an overview?
I clearly need to read up on the subject.
I'm probably not the best person to ask that, what I know is based on
reading the EC2 website, CLI tools manuals and API, and looking at the
websites of the other providers offering compute nodes.
Perhaps Justin can suggest something?
Andrew's input was most illuminating, and not only because it seemed to
suggest that you need to provision block storage in all cases :/
1: binary code
2: block device data
3: rumprun launch parameters, including application command line
(4: now-proposed alternative way to supply data)
This part (distributing the rumprun parameters) seems easy:
Rumpconfig could process configuration information in this order:
1) platform-specific mechanism (cmdline, Xenstore)
2) a '/rumpconfig.json' file
1) would override 2). It would also eliminate the current similar hack used
by the "iso" target.
That also means that you need a method for fetching the default
configuration from the binary. You also need an easy way to make small
adjustments.
Doesn't seem particularly easy.
Maybe rumprun shouldn't handle configuration at all, i.e. should be only
and purely a launch tool? I sort of like the idea of
compile+bake+launch_with_parameters, but if it doesn't cover enough
cases in a sensible way, then let's not get hung up on trying to make it
work. (thinking out loud here)
I can make the implementation even more minimal and allow for including
only a single directory tree if it bothers you. The reason I implemented
multiple -R's was to provide a (granted, poor/minimal but working) ability
to accomplish:
rumpbake -R ...path/to/default/root -R data/ ...
ie. Give us the ability to provide a default /etc in the rumprun
repository.
So with a single -R parameter, is the user is responsible for
figuring out where to get /etc from?
Either that, or they get it by default (and can override the default if
they don't want it). In either case, for the purposes of rumprun unikernels
I'm treating /etc as "legacy OS cruft that needs to be included to make
things work", not as "a legitimate place to put your configuration". This
is what my packages also follow, application configuration goes in
/data/conf, not /etc.
You're not defining "override". What happens if you want to *remove* a
file from there? Let's say you simply don't want /etc/services (not
sure why, maybe you're just trying to save space?). Furthermore, what
happens when you want to use a custom /etc/passwd and keep everything
else in the default /etc the same?
So either way, regardless of if "override" means "override individual
files" or "override the entire default /etc" a user will, depending on
the case, be screwed with -R. Let's not screw users.
Also, when you say "legacy OS cruft that needs to be included to make
things work" all I can see is "needs to be included to make things work" ...
