Re: RFC: bootloader/initramfs protocol v2
Hi Bastian!
On Wed, Nov 23, 2022 at 08:53:10AM +0100, Bastian Blank wrote:
> On Mon, Nov 07, 2022 at 11:40:46AM +0100, Jörg Behrmann wrote:
> > On Tue, Nov 01, 2022 at 09:29:07PM +0100, Bastian Blank wrote:
> > > ## Prior works
> > >
> > > [..]
> > > - systemd install-kernel: only BLS as target, which nothing used by
> > > default in Debian can read
> > > [..]
> >
> > To maybe clarify this a bit: While kernel-install does target BLS
> > primarily, it
> > has support for differing layouts via the KERNEL_INSTALL_LAYOUT environment
> > variable (set via layout= in {/usr/lib,/etc}/kernel/install.conf).
>
> Okay. However no multi-selection.
>
I'm not sure I fully understand what you mean, but I think what you mean is,
that you would like to be be able to generate stuff in /boot for multiple
different values of KERNEL_INSTALL_LAYOUT, e.g. boot loader spec and something
else.
While the variable itself is single-valued, kernel-install can of course be run
multiple times for different values of KERNEL_INSTALL_LAYOUT or with different
configs wholesale (kernel-install reads environment variables from
/etc/kernel/install.conf, but this can be configured via the environment
variable KERNEL_INSTALL_CONF_ROOT).
I'd be grateful if you could flesh out the scenario for multi-selection a bit,
because the systems I think of will usually have a single way they are booted
and while being able to switch between different setups is important for
development work, I'm not sure whether this case common in production
deployments.
> > kernel-install scripts are also just that, executables with two CLI entry
> > points
> > (add and remove) and a set of fixed environment variables that they
> > receive. Currently all of this is written in POSIX shell (compatible with
> > dash) - they can be made to do everything.
>
> Add and remove are not enough for what I would like to have.
>
Fair enough. I'd be grateful, though, for a bit more information on what your
plan is, because, looking at the goals section of your initial mail, I think
that these could be achieved within this framework.
> And I'm
> really not in the mood to try that in POSIX shell.
>
Fair enough, I'm not a lover of POSIX shell myself. :) While kernel-install
itself is written in POSIX shell (for compatibility with dash, I think, because
it initially was bash and used bashisms and was ported later), the install
scripts can be whatever you want - even compiled binaries.
The API is described in the kernel-install manpage, although maybe a bit
haphazardly, and consists of arguments the scripts/binaries in
{/usr/lib,/etc}/install.d will be called with, environment variables they can
expect and exit codes that re understood.
> Is it even able to
> make sure stuff is actually written on the disk?
This question has multiple answers. I'm not sure whether anybody does this yet,
but you can add a script to run sync at the end and report an error if that was
unsuccessful. Also, kernel-install does provide a temporary staging directory,
so you can output everything there and have it copied to /boot at the end and
keep the interactions with dumb filesystems to a minimum.
> > kernel-install and BLS support a bit more than the machine ID here, The
> > relevant
> > keyword to look for in the kernel-install manpage and BLS is "entry-token".
> > This was added for golden master setups, because the machine ID will
> > probably
> > only generated on first boot, but the initrd would be generated beforehand.
>
> And I assume it got no way to migrate from an pre-defined entry-token to
> the machine ID later on, which would be kind of useful to convert a
> golden image into a real system where you can add unrelated systems
> later.
>
The entry-token is just the content of the /etc/kernel/entry-token file, so it
can be reset later and if no entry token exists it falls back to machine id,
IMAGE_ID and ID in the os-release file in that order.
best regards,
Jörg
Re: RFC: bootloader/initramfs protocol v2
Hi Luca On Sun, Nov 06, 2022 at 02:14:54PM +, Luca Boccassi wrote: > On Tue, 2022-11-01 at 21:29 +0100, Bastian Blank wrote: > > ## Goals > > > > - Setup complete boot entries from packaged and generated files > > - Support dumb file systems for /boot by default, so boot loaders can > > drop complex file system support. > > - Re-create stuff in /boot from scratch > > - Remove symlink handling from kernel package > > - Single entry point for packages and admins, aka no tool specific > > "update-initramfs" anymore > > Could you clarify what you mean by "single entry point" here? It's the > only point I can't quite decode. A trigger? You have one tool to call, not right now multiple (the kernel maintainer scripts do some, "update-initramfs" scribles to it as well). > I would like to suggestion this as an additional, explicit goal, rather > than implicit: > End result should be fully compatible with the BLS (for the readers: > https://uapi-group.org/specifications/specs/boot_loader_specification/ > ) Yes. We will need extensions, some may be incompatible as well. > > ## Open questions > > > > - How to select default entry if supported, just sort by version and > > use > > newest? This also works somewhat in BLS. > > Yes, we should follow BLS on this, so that end result is predictable, > well-defined and doesn't vary wildly from other distros. Okay. > In fact, if Grub can do UKIs, do we even need Type 1 entries (separate > textual config files) for anything at that point? grub needs to be able to read and disect UKI for non-EFI without loader code (aka for ppc, sparc, bios and xen). And we need a way to create them without the really heavy binutils. And for all the old bootloaders, we have no way to use UKI at all. So if we want to ship UKI in packages, we also need to be able to disect them in userspace. Can you currently create UKI with multiple multi-boot capable initrd attachments? For Xen. > > ### Distribution file system (/usr) > > > > * /usr/lib/boot/$package(_$modifier)/ > > * ./data: raw data for item > > * ./metadata: info about item in undetermined format > > What would 'metadata' be in this context? A description what it is. We might have raw kernel blobs for some architectures. We might have Xen kernels, which require a nested Linux. And all the information that need to go into the type 1 file and comes from the package, like version, name, architecture. Regards, Bastian -- She won' go Warp 7, Cap'n! The batteries are dead!
Re: RFC: bootloader/initramfs protocol v2
Hi Jörg
On Mon, Nov 07, 2022 at 11:40:46AM +0100, Jörg Behrmann wrote:
> On Tue, Nov 01, 2022 at 09:29:07PM +0100, Bastian Blank wrote:
> > ## Prior works
> >
> > [..]
> > - systemd install-kernel: only BLS as target, which nothing used by
> > default in Debian can read
> > [..]
>
> To maybe clarify this a bit: While kernel-install does target BLS primarily,
> it
> has support for differing layouts via the KERNEL_INSTALL_LAYOUT environment
> variable (set via layout= in {/usr/lib,/etc}/kernel/install.conf).
Okay. However no multi-selection.
> kernel-install scripts are also just that, executables with two CLI entry
> points
> (add and remove) and a set of fixed environment variables that they
> receive. Currently all of this is written in POSIX shell (compatible with
> dash) - they can be made to do everything.
Add and remove are not enough for what I would like to have. And I'm
really not in the mood to try that in POSIX shell. Is it even able to
make sure stuff is actually written on the disk?
> kernel-install and BLS support a bit more than the machine ID here, The
> relevant
> keyword to look for in the kernel-install manpage and BLS is "entry-token".
> This was added for golden master setups, because the machine ID will probably
> only generated on first boot, but the initrd would be generated beforehand.
And I assume it got no way to migrate from an pre-defined entry-token to
the machine ID later on, which would be kind of useful to convert a
golden image into a real system where you can add unrelated systems
later.
Bastian
--
Those who hate and fight must stop themselves -- otherwise it is not stopped.
-- Spock, "Day of the Dove", stardate unknown
Re: RFC: bootloader/initramfs protocol v2
Having written a similar inquiry not too long ago, I welcome this discussion,
thanks! :)
On Tue, Nov 01, 2022 at 09:29:07PM +0100, Bastian Blank wrote:
> ## Prior works
>
> [..]
> - systemd install-kernel: only BLS as target, which nothing used by
> default in Debian can read
> [..]
To maybe clarify this a bit: While kernel-install does target BLS primarily, it
has support for differing layouts via the KERNEL_INSTALL_LAYOUT environment
variable (set via layout= in {/usr/lib,/etc}/kernel/install.conf).
kernel-install scripts are also just that, executables with two CLI entry points
(add and remove) and a set of fixed environment variables that they
receive. Currently all of this is written in POSIX shell (compatible with
dash) - they can be made to do everything.
The BLS type 1 loader entries (an example can be found in the BLS [1]) can in
the worst case be read with a line of AWK or shell, but Unified Kernel Images
(UKIs / BLS type 2) are fully self-contained and can do without any sidecar
config files.
> ## File system layout
>
> Some initial ideas about how stuff could look.
>
> ### Boot file system (/boot)
>
> This file system might be shared, so everything is somewhat referenced
> to the machine id. This should be somewhat compatible with BLS (type
> 1).
>
> * /boot/$machineid/
> * ./grub/: config snippets, so we can do "no overwrite"
>
kernel-install and BLS support a bit more than the machine ID here, The relevant
keyword to look for in the kernel-install manpage and BLS is "entry-token".
This was added for golden master setups, because the machine ID will probably
only generated on first boot, but the initrd would be generated beforehand.
best regards,
Jörg
[1] https://uapi-group.org/specifications/specs/boot_loader_specification/
Re: RFC: bootloader/initramfs protocol v2
On Tue, 2022-11-01 at 21:29 +0100, Bastian Blank wrote: > [Cc Ben as he gave feedback to the last iteration, Luca as he wanted > something actionable] > > Hi folks > > As I abondened the last try and also learned some new things in the > meantime, I'd like to discuss another try at re-organizing how Debian > does boot loaders and initramfs. This mail mostly tries to get the > goals > and requirements straight. > > Please provide feedback. Also for missing stuff. > > Regards, > Bastian Thank you, this is great. > ## Goals > > - Setup complete boot entries from packaged and generated files > - Support dumb file systems for /boot by default, so boot loaders can > drop complex file system support. > - Re-create stuff in /boot from scratch > - Remove symlink handling from kernel package > - Single entry point for packages and admins, aka no tool specific > "update-initramfs" anymore Could you clarify what you mean by "single entry point" here? It's the only point I can't quite decode. A trigger? I would like to suggestion this as an additional, explicit goal, rather than implicit: End result should be fully compatible with the BLS (for the readers: https://uapi-group.org/specifications/specs/boot_loader_specification/ ) Given we are doing something new, it's very well worth to be fully compatible with cross-distro standards so that we can leverage existing toolchains. > ## Requirements > > - Read package files in /usr > - Uses observed state, not changes provided by maintainer scripts > - Dumb writes to /boot. No rename, no sym, hard links. Can use ref > links if possible. > - Generate initramfs if needed, creates new entry if re-generated > - Keep older stuff (like previous kernel, initramfs) for a short > while > - Possible to support multiple targets, like grub, zipl, flash-kernel > - Multiple inputs, plain kernel, UKI, also in one version > - Combinations of inputs, Xen+Linux > - Completely outside of kernel package > - Backward compatible support for stuff packaged in /boot > - Use config for kernel command line > > ## Open questions > > - How to select default entry if supported, just sort by version and > use > newest? This also works somewhat in BLS. Yes, we should follow BLS on this, so that end result is predictable, well-defined and doesn't vary wildly from other distros. > - How to interface with boot loaders, just absorb all knowledge about > config and only run install tools if necessary (like with zipl as > block map based loader)? grub might get support for BLS, at least > there exists a patch somewhere, that will make it easier, and we > can > just iterate over config files defining one entry each. Self-described and auto-discoverable images should be the primary mean. For reference, the patch that adds support for the BLS to Grub is here: https://github.com/osteffenrh/grub2/commit/d0c402c96159423242cf7b612773126ccc11a83b UKIs will be proposed for Fedora, this should land as part of that work and do a lot of the heavy lifting for us: https://fedoraproject.org/wiki/Changes/Unified_Kernel_Support_Phase_1 In fact, if Grub can do UKIs, do we even need Type 1 entries (separate textual config files) for anything at that point? > ## Prior works > > - Current Debian: change based, overwrites by dpkg in /boot, > versioned > by ABI > - systemd install-kernel: only BLS as target, which nothing used by > default in Debian can read > More? For reference, Debian images can be built using dracut + sd-boot + kernel-install, images built with mkosi work like that: https://github.com/systemd/mkosi > ## File system layout > > Some initial ideas about how stuff could look. > > ### Boot file system (/boot) > > This file system might be shared, so everything is somewhat > referenced > to the machine id. This should be somewhat compatible with BLS (type > 1). This just dropped talking about lots of these concepts: https://0pointer.net/blog/linux-boot-partitions.html > * /boot/$machineid/ > * ./grub/: config snippets, so we can do "no overwrite" > > ### Distribution file system (/usr) > > * /usr/lib/boot/$package(_$modifier)/ > * ./data: raw data for item > * ./metadata: info about item in undetermined format What would 'metadata' be in this context? -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part
RFC: bootloader/initramfs protocol v2
[Cc Ben as he gave feedback to the last iteration, Luca as he wanted something actionable] Hi folks As I abondened the last try and also learned some new things in the meantime, I'd like to discuss another try at re-organizing how Debian does boot loaders and initramfs. This mail mostly tries to get the goals and requirements straight. Please provide feedback. Also for missing stuff. Regards, Bastian ## Goals - Setup complete boot entries from packaged and generated files - Support dumb file systems for /boot by default, so boot loaders can drop complex file system support. - Re-create stuff in /boot from scratch - Remove symlink handling from kernel package - Single entry point for packages and admins, aka no tool specific "update-initramfs" anymore ## Requirements - Read package files in /usr - Uses observed state, not changes provided by maintainer scripts - Dumb writes to /boot. No rename, no sym, hard links. Can use ref links if possible. - Generate initramfs if needed, creates new entry if re-generated - Keep older stuff (like previous kernel, initramfs) for a short while - Possible to support multiple targets, like grub, zipl, flash-kernel - Multiple inputs, plain kernel, UKI, also in one version - Combinations of inputs, Xen+Linux - Completely outside of kernel package - Backward compatible support for stuff packaged in /boot - Use config for kernel command line ## Open questions - How to select default entry if supported, just sort by version and use newest? This also works somewhat in BLS. - How to interface with boot loaders, just absorb all knowledge about config and only run install tools if necessary (like with zipl as block map based loader)? grub might get support for BLS, at least there exists a patch somewhere, that will make it easier, and we can just iterate over config files defining one entry each. ## Prior works - Current Debian: change based, overwrites by dpkg in /boot, versioned by ABI - systemd install-kernel: only BLS as target, which nothing used by default in Debian can read More? ## File system layout Some initial ideas about how stuff could look. ### Boot file system (/boot) This file system might be shared, so everything is somewhat referenced to the machine id. This should be somewhat compatible with BLS (type 1). * /boot/$machineid/ * ./grub/: config snippets, so we can do "no overwrite" ### Distribution file system (/usr) * /usr/lib/boot/$package(_$modifier)/ * ./data: raw data for item * ./metadata: info about item in undetermined format -- I'm a soldier, not a diplomat. I can only tell the truth. -- Kirk, "Errand of Mercy", stardate 3198.9
