Re: s6-linux-init, alpine linux, and initramfs
Oh, now I understand what you and Laurent meant: apart from
`/proc/cmdline', the kernel also exports its boot parameters (except for
`root=...', parameters without a value and possibly something else) into
the environment of the init process. This is really useful when not all
parameters need to be scanned, which is often the case.
On Wed, Feb 01, 2017 at 10:11:12AM +0100, Guillaume Perréal wrote:
> For my tests, I used the following snippet as an init script:
>
> #!/bin/execlineb -p
> foreground { /usr/bin/s6-printenv }
> /bin/sh
>
> Neither execlineb nor s6-printenv touch the environment. Yet, I had
> "modules" and "initrd" variables (which are in the kernel command line)
> but not the "root" one.
--
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: s6-linux-init, alpine linux, and initramfs
Le 01/02/2017 à 08:42, Casper Ti. Vector a écrit :
From those distros I have used, it seems that the kernel does not eat
anything from its command line, so I think it is unnecessary to
duplicate the `rootfs' parameter.
Once it has all the information it needs and has found its rootfs, the
initramfs script doesn't need environment variables anymore, so it
cleans up its environment. You should do the same: be as transparent as
possible, do not leak into /sbin/init anything it doesn't strictly need.
/sbin/init may have the environment variables set by the kernel, but it
definitely shouldn't have any variables set by your initramfs script.
In addition, the cause for `KOPT_*' not showing up in Alpine init is
that sh(1) does not leak manually set variables except when told to
`export'. In contrast, rc(1) does not have the `export' builtin, and
simply exports all variables. Anyway, you can use the following
chainloading command to clean up the environment just before switching
root:
For my tests, I used the following snippet as an init script:
#!/bin/execlineb -p
foreground { /usr/bin/s6-printenv }
/bin/sh
Neither execlineb nor s6-printenv touch the environment. Yet, I had
"modules" and "initrd" variables (which are in the kernel command line)
but not the "root" one.
--
Guillaume.
Re: s6-linux-init, alpine linux, and initramfs
On Tue, Jan 31, 2017 at 09:35:55PM +, Laurent Bercot wrote:
> It won't do it in every case: parsing /proc/cmdline is hazardous and
> much more difficult than it appears. (There could be quotes, and quoted
> spaces, in the elements.) We had a discussion about this on the Alpine
> development IRC channel, and it appeared that you can't do it safely
> with less than 50 lines of shell.
Perhaps more than 50 lines if you insist on use sh(1) because of the
usage of `eval' to set up dynamically-named variables, eg.:
> eval "${opt%%=*}='${opt#*=}'"
which, for opt="a=b' dont-rm -rf /'", becomes
> a='b' dont-rm -rf /''
However, the code can be very simple if you use a shell that does not
forbid "$key=val", and restrict the kernel command line as a
space-separated list of strings in the `key[=val1,val2,...]' form,
where `key's are valid shell variable names, and `val's are strings of
non-whitespace non-comma printable characters. Here's an example
written in rc(1) (the Byron Rakitzis implementation):
> for (opt in `{cat /proc/cmdline}) {
> if (~ $opt *'='*) {
> key=`{echo $opt | sed 's/=.*$//'}
> KOPT_$key=`{echo $opt | sed 's/^[^=]*=//; s/,/ /g'}
> } else if (~ $opt no*) {
> KOPT_`{echo $opt | sed 's/^no//'}=no
> } else KOPT_$opt=yes
> }
> The simplest solution, if you control the kernel command line, is to
> duplicate the root= argument with something the kernel will let through:
> The kernel keeps the "root" argument, but puts "rootfs" in the
> environment, which you can easily read from.
>From those distros I have used, it seems that the kernel does not eat
anything from its command line, so I think it is unnecessary to
duplicate the `rootfs' parameter.
> Once it has all the information it needs and has found its rootfs, the
> initramfs script doesn't need environment variables anymore, so it
> cleans up its environment. You should do the same: be as transparent as
> possible, do not leak into /sbin/init anything it doesn't strictly need.
> /sbin/init may have the environment variables set by the kernel, but it
> definitely shouldn't have any variables set by your initramfs script.
In addition, the cause for `KOPT_*' not showing up in Alpine init is
that sh(1) does not leak manually set variables except when told to
`export'. In contrast, rc(1) does not have the `export' builtin, and
simply exports all variables. Anyway, you can use the following
chainloading command to clean up the environment just before switching
root:
> /bin/busybox env -i [key1=val1 key2=val2 ...] \
> /bin/busybox switch_root $root $init
--
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: s6-linux-init, alpine linux, and initramfs
I had to mount /dev and to resort to busybox's switch_root because the one built using execline tools had trouble executing. Despite using executables from the actual root filesystem, it had issue spawing tools at some point in the loop. I guess this is because they are not statically compiled. Yes, obviously, all the binaries in your initramfs need to be statically compiled. Or you could put a libc.so and a dynamic linker in your initramfs' /lib, if you prefer. musl makes it easy enough (only one .so), but it also makes it easy to statically compile, so, your choice. The "root=" seems a bit tricker: it is only available through parsing /proc/cmdline. I think something like this will do it: It won't do it in every case: parsing /proc/cmdline is hazardous and much more difficult than it appears. (There could be quotes, and quoted spaces, in the elements.) We had a discussion about this on the Alpine development IRC channel, and it appeared that you can't do it safely with less than 50 lines of shell. So you could: - ignore the problem and hope you'll never need funky options - add a /bin/sh to your rootfs and the code to parse /proc/cmdline safely - find another solution. The simplest solution, if you control the kernel command line, is to duplicate the root= argument with something the kernel will let through: root=/dev/sda1 rootfs=/dev/sda1 The kernel keeps the "root" argument, but puts "rootfs" in the environment, which you can easily read from. I have seen alpine's intit script use KOPT_* variables (like KOPT_root, KOPT_quiet and so on) but I have found no reference to them nor have I found them in the environment of process 1. The Alpine initramfs is a bit more complex than my simple skeleton, for several reasons. One of the reasons is that they want to parse /proc/cmdline the right way, so they have that giant block of shell code; it's that piece of code that exports the elements of /proc/cmdline as KOPT_something. Once it has all the information it needs and has found its rootfs, the initramfs script doesn't need environment variables anymore, so it cleans up its environment. You should do the same: be as transparent as possible, do not leak into /sbin/init anything it doesn't strictly need. /sbin/init may have the environment variables set by the kernel, but it definitely shouldn't have any variables set by your initramfs script. -- Laurent
Re: s6-linux-init, alpine linux, and initramfs
Thank you very much !
This have helped me a lot.
I had to mount /dev and to resort to busybox's switch_root because the
one built using execline tools had trouble executing. Despite using
executables from the actual root filesystem, it had issue spawing tools
at some point in the loop. I guess this is because they are not
statically compiled.
One improvement would be to use the kernel commandline to select the
modules to load and detect the root filesystem. Right now I just use
hard-coded values.
The "modules=" option is added as an environment variable by the kernel
so this should do the trick:
import -d, -s modules modprobe -a ${modules}
The "root=" seems a bit tricker: it is only available through parsing
/proc/cmdline. I think something like this will do it:
backtick -ni ROOTFS {
pipeline {
forbacktickx OPT { redirfr -r 0 /proc/cmdline s6-cat }
import -u OPT
s6-echo $OPT
}
pipeline { s6-grep ^root= }
s6-cut -d= -f2-
}
import -u ROOTFS
backtick -ni ROOTFSTYPE { blkid -s TYPE -o value $ROOTFS }
import -u ROOTFSTYPE
I have seen alpine's intit script use KOPT_* variables (like KOPT_root,
KOPT_quiet and so on) but I have found no reference to them nor have I
found them in the environment of process 1.
Now i will take a look at s6-rc examples scripts.
--
Guillaume.
Le 30/01/2017 à 15:19, Laurent Bercot a écrit :
2) have a very small init script to load the modules, mount the
filesystems (/dev, /proc, /sys, /), and finally pivot-chroot into
s6-linux-init phase 1. This would be less elegent but it might be
easier to set up.
This. If you need or want an initramfs, you need to comply with the
implicit initramfs contract: when you exec into /sbin/init, it must be
the only process running on your machine, just as if the kernel started
it; and any init system should be able to work, the initramfs should not
tie you to a specific init. So, spawning a supervision tree in the
initramfs is a no-no, because it breaks both aspects.
You can see an initramfs as a mini-system that you set up to do what
needs to be done *and then tear down* before exec'ing into the real
system.
So, do that: load your modules, find your rootfs, pivot-chroot into it,
and start your real system with your init of choosing.
Ideally, you'd even unmount /proc and /sys (which you likely need during
your initramfs execution) before entering /sbin/init. But obviously
that's not practical since your boot sequence will mount then again
very soon, so the separation between "pre-init" and "post-init" can
be a bit less strict. You can document that the state of your system
at init time is "pristine as if the kernel had directly started init,
except that /proc and /sys are already mounted", for instance, and
that's acceptable. (/dev is not even a question - you should have a
devtmpfs mounted at boot anyway, and mount --move it after your
pivot_root.)
But "after initramfs, I already have a supervision system and just
need to run the rest of the boot sequence" is not acceptable - if only
because you then need to keep supervision executables in RAM, and a
component of your pivoted initramfs in your PATH.
Just make initramfs as transparent as possible, it's a lot cleaner.
Oh, by the way, pivot_root works with initrd, but not initramfs: see
https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt
so you can use busybox/toybox's switch_root instead, or you can do the
switch_root by hand.
I have a skeleton /init here that only needs in your initramfs:
- empty /dev, /proc, /rootfs and /sys directories. OK if the kernel
mounts devtmpfs at boot (which it should).
- a /sbin directory with a "mdev" static binary inside. (busybox with
only mdev selected will still be 100ish kB, that's unfortunately normal.)
- a /command directory with static cd, execlineb, export, foreground,
if, redirfd, s6-echo and s6-mount binaries inside. Also "define" for the
skeleton but you'd replace it with something else for your rootfs
detection.
- whatever else you need to do your job - you could add modutils to your
busybox build, for instance, if you want to load modules. You may want
a /etc/mdev.conf depending on the devices you're expecting to detect.
- also execline, s6-portable-utils and s6-linux-utils binaries
accessible
in the /command directory of your real root filesystem.
You can get it at http://pastebin.com/KZfdETy5
My gzipped initramfs image made with that is about 104kB, for x86_64.
HTH,
--
Laurent
Re: s6-linux-init, alpine linux, and initramfs
2) have a very small init script to load the modules, mount the filesystems (/dev, /proc, /sys, /), and finally pivot-chroot into s6-linux-init phase 1. This would be less elegent but it might be easier to set up. This. If you need or want an initramfs, you need to comply with the implicit initramfs contract: when you exec into /sbin/init, it must be the only process running on your machine, just as if the kernel started it; and any init system should be able to work, the initramfs should not tie you to a specific init. So, spawning a supervision tree in the initramfs is a no-no, because it breaks both aspects. You can see an initramfs as a mini-system that you set up to do what needs to be done *and then tear down* before exec'ing into the real system. So, do that: load your modules, find your rootfs, pivot-chroot into it, and start your real system with your init of choosing. Ideally, you'd even unmount /proc and /sys (which you likely need during your initramfs execution) before entering /sbin/init. But obviously that's not practical since your boot sequence will mount then again very soon, so the separation between "pre-init" and "post-init" can be a bit less strict. You can document that the state of your system at init time is "pristine as if the kernel had directly started init, except that /proc and /sys are already mounted", for instance, and that's acceptable. (/dev is not even a question - you should have a devtmpfs mounted at boot anyway, and mount --move it after your pivot_root.) But "after initramfs, I already have a supervision system and just need to run the rest of the boot sequence" is not acceptable - if only because you then need to keep supervision executables in RAM, and a component of your pivoted initramfs in your PATH. Just make initramfs as transparent as possible, it's a lot cleaner. Oh, by the way, pivot_root works with initrd, but not initramfs: see https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt so you can use busybox/toybox's switch_root instead, or you can do the switch_root by hand. I have a skeleton /init here that only needs in your initramfs: - empty /dev, /proc, /rootfs and /sys directories. OK if the kernel mounts devtmpfs at boot (which it should). - a /sbin directory with a "mdev" static binary inside. (busybox with only mdev selected will still be 100ish kB, that's unfortunately normal.) - a /command directory with static cd, execlineb, export, foreground, if, redirfd, s6-echo and s6-mount binaries inside. Also "define" for the skeleton but you'd replace it with something else for your rootfs detection. - whatever else you need to do your job - you could add modutils to your busybox build, for instance, if you want to load modules. You may want a /etc/mdev.conf depending on the devices you're expecting to detect. - also execline, s6-portable-utils and s6-linux-utils binaries accessible in the /command directory of your real root filesystem. You can get it at http://pastebin.com/KZfdETy5 My gzipped initramfs image made with that is about 104kB, for x86_64. HTH, -- Laurent
s6-linux-init, alpine linux, and initramfs
Hello there, Me again. After tweaking my Xsession for a stater, I am trying to build a VM with s6-linux-init. I am starting from Alpine Linux, because I am not into recompiling Linux and all tools from scratch (well, not yet) and this distro already provides binaries for all skarnet tools. Like many distro, they use an initramfs, because most of the drivers (including sd-mod, scsi and ext*) are built as modules. I have found how to customize the initramfs. Now I am facing a choice: what should I put in the initramfs ? 1) put s6-linux-init phase 1 into the initramfs, use it at /init, then use an embedded /etc/rc.init to load modules, mount / and exec into the root's /etc/rc.init. The advantage would be a full s6 boot process. One drawback is that I have to put all execline and s6 tools (but s6-rc) in the initramfs. Another one is that the phase 1 of s6-linux-init is not very verbose and does not have any emergency fallback. 2) have a very small init script to load the modules, mount the filesystems (/dev, /proc, /sys, /), and finally pivot-chroot into s6-linux-init phase 1. This would be less elegent but it might be easier to set up. Any idea on this ? I know the right way would be to recompile linux with the right modules to boot directly into s6-linux-init phase 1 from the root partition. ++ -- Guillaume.
