Bug#950086: base-installer: please support configuring initramfs compression
On 30/01/2020 16:43, Ben Hutchings wrote: Oh, right, then I misunderstood what you were doing. I don't understand how the two templates work together, so it may well be that the first version was OK. I don't really grok debconf, but from what I can tell: 1. at build-time, x may be set differently per-arch 2. either x or y may be preseeded with a value by the user 3. if y is empty, it's set to the value of x at install-time 4. installer asks about y (based on priority) 5. value of y is used to configure initramfs-tools where: x is base-installer/kernel/linux/initramfs-tools/* (string) y is base-installer/initramfs-tools/* (select) If either is set to a wrong value, y is somehow reset to it's first choice. I searched a bit and came across bug #192889 which says it's the frontend that does sanitization, so step 4. The first version is probably good enough, except in contrived situations designed to break it. (You didn't reply to the bts, so I'm not doing either. Just pointing it out in case it was by accident. If you resend to bts, feel free to forward this mail as well.)
Bug#950086: base-installer: please support configuring initramfs compression
On Thu, 2020-01-30 at 20:10 +0300, Alper Nebi Yasak wrote: > On 30/01/2020 16:43, Ben Hutchings wrote: > > Oh, right, then I misunderstood what you were doing. I don't > > understand how the two templates work together, so it may well be that > > the first version was OK. > I don't really grok debconf, but from what I can tell: >1. at build-time, x may be set differently per-arch >2. either x or y may be preseeded with a value by the user >3. if y is empty, it's set to the value of x at install-time >4. installer asks about y (based on priority) >5. value of y is used to configure initramfs-tools > where: >x is base-installer/kernel/linux/initramfs-tools/* (string) >y is base-installer/initramfs-tools/* (select) > > If either is set to a wrong value, y is somehow reset to it's first > choice. I searched a bit and came across bug #192889 which says it's the > frontend that does sanitization, so step 4. > > The first version is probably good enough, except in contrived > situations designed to break it. Right, thanks for the explanation and sorry for wasting your time with my initial objection. > (You didn't reply to the bts, so I'm not doing either. Just pointing it > out in case it was by accident. If you resend to bts, feel free to > forward this mail as well.) That was accidental, so I've now bounced both messages. Ben. -- Ben Hutchings For every complex problem there is a solution that is simple, neat, and wrong. signature.asc Description: This is a digitally signed message part
Bug#950086: base-installer: please support configuring initramfs compression
On Thu, 2020-01-30 at 16:35 +0300, Alper Nebi Yasak wrote: > On 29/01/2020 21:31, Ben Hutchings wrote: > > This is definitely a rather niche option, so "wishlist" is appropriate. > > Thanks! (My reasoning was that it's a new feature.) > > > So it's possible to set any arbitrary string, but if you set it to > > anything other than "gzip", "xz", or "lzma" then the required > > compressor might not be installed. (And it's possible to set a string > > that mkinitramfs doesn't install either.) > > > > I think this needs to be a "select" type question, so that only the > > supported compression types can be selected. > > Allowing arbitrary strings wasn't the intention. I had mimicked the > driver-policy implementation. The template you quoted is only used to > set a "select" type question, and I assumed doing that with an invalid > value would cause an error. [...] Oh, right, then I misunderstood what you were doing. I don't understand how the two templates work together, so it may well be that the first version was OK. Ben. -- Ben Hutchings For every complex problem there is a solution that is simple, neat, and wrong. signature.asc Description: This is a digitally signed message part
Bug#950086: base-installer: please support configuring initramfs compression
On 29/01/2020 21:31, Ben Hutchings wrote: This is definitely a rather niche option, so "wishlist" is appropriate. Thanks! (My reasoning was that it's a new feature.) So it's possible to set any arbitrary string, but if you set it to anything other than "gzip", "xz", or "lzma" then the required compressor might not be installed. (And it's possible to set a string that mkinitramfs doesn't install either.) I think this needs to be a "select" type question, so that only the supported compression types can be selected. Allowing arbitrary strings wasn't the intention. I had mimicked the driver-policy implementation. The template you quoted is only used to set a "select" type question, and I assumed doing that with an invalid value would cause an error. While testing I noticed that it would use the first choice when given invalid values (e.g. with kernel cmdline [0]), so I reordered the compression choices to match initramfs.conf, with "gzip" as the first choice. Anyway, I don't think checking the values would hurt, so I did that for my patch and for the driver-policy question too (as patch 1/2). I'm just running "exit 1" in the invalid cases, I'm not sure that's entirely right. Also I'm now calling apt-install before writing the configuration, and made it fail the installation step with (hopefully the appropriate error) if it can't install the compressor package. Attaching a series of two patch files to replace the previous one. Hope they resolve your concerns. [0] quiet console=tty0 priority=critical \ base-installer/initramfs-tools/compression=git \ base-installer/kernel/linux/initramfs-tools/compression=git >From e6949630a32e4930561e5fa9d9f443ab8ac044bc Mon Sep 17 00:00:00 2001 From: Alper Nebi Yasak Date: Thu, 30 Jan 2020 14:21:30 +0300 Subject: [PATCH 1/2] Verify answers to initramfs-tools driver-policy questions Also set "base-installer/kernel/linux/initramfs-tools/driver-policy" to type "select" with choices "most" and "dep". Signed-off-by: Alper Nebi Yasak --- debian/templates-arch | 3 ++- library.sh| 9 +++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/debian/templates-arch b/debian/templates-arch index 30e8f7a8..fbb18372 100644 --- a/debian/templates-arch +++ b/debian/templates-arch @@ -6,7 +6,8 @@ Description: for internal use only Use an (initramfs) initrd (linux 2.6 and later only) Template: base-installer/kernel/linux/initramfs-tools/driver-policy -Type: string +Type: select +Choices: most, dep Default: most Default[armel]: dep Description: for internal use diff --git a/library.sh b/library.sh index d7f05024..890f7087 100644 --- a/library.sh +++ b/library.sh @@ -583,12 +583,17 @@ EOF fi db_get base-installer/initramfs-tools/driver-policy - if [ "$RET" != most ]; then + IT_MODULES="$RET" + if [ "$IT_MODULES" != most ] && [ "$IT_MODULES" != dep ]; then + exit 1 + fi + + if [ "$IT_MODULES" != most ]; then cat > $IT_CONFDIR/driver-policy <>From f0dad1298cfb8f0e32616243f44ba776b647dc75 Mon Sep 17 00:00:00 2001 From: Alper Nebi Yasak Date: Thu, 30 Jan 2020 14:42:26 +0300 Subject: [PATCH 2/2] Support configuring initramfs compression Some machines/bootloaders do not support initramfs images larger than a certain size. Setting MODULES=dep in initramfs-tools config is usually enough to satisfy these size limits, and base-installer already asks a question for this (driver-policy, with priority medium). This patch implements a similar question for initramfs-tools' COMPRESS setting which can further reduce the initramfs size for these machines. Signed-off-by: Alper Nebi Yasak --- debian/bootstrap-base.templates | 11 +++ debian/templates-arch | 7 +++ library.sh | 34 + 3 files changed, 52 insertions(+) diff --git a/debian/bootstrap-base.templates b/debian/bootstrap-base.templates index 78a6f29a..8529fbf7 100644 --- a/debian/bootstrap-base.templates +++ b/debian/bootstrap-base.templates @@ -88,6 +88,17 @@ _Description: Drivers to include in the initrd: smaller targeted initrd there is a very small chance that not all needed drivers are included. +Template: base-installer/initramfs-tools/compression +Type: select +Choices: gzip, bzip2, lz4, lzma, lzop, xz +# :sl3: +_Description: Compression method for the initramfs image: + Compressing the initramfs is usually a trade-off between initramfs + size, memory use, compression speed, and the time your system takes to + boot (decompression speed). Gzip is reasonably balanced, xz and lzma + makes the initramfs significantly smaller, but lz4 has the highest + decompression speed. + Template: base-installer/kernel/failed-install Type: error # :sl2: diff --git a/debian/templates-arch b/debian/templates-arch index fbb18372..29d01721 100644 --- a/debian/templates-arch +++ b/debian/templates-arch @@ -13,6 +13,13 @@ Default[armel]: dep Description: for internal use Default
Bug#950086: base-installer: please support configuring initramfs compression
On Wed, 2020-01-29 at 00:01 +0300, Alper Nebi Yasak wrote: > Source: base-installer > Version: 1.192 > Severity: wishlist > Tags: patch This is definitely a rather niche option, so "wishlist" is appropriate. > Dear Maintainers, > > Debian-installer can already configure initramfs-tools' MODULES setting > to get a smaller initramfs. I've written a patch for configuring its > COMPRESS setting as well, please consider applying it. I've tested it > using an arm64 virtual machine with a custom built netboot image. > > I'm doing something like flash-kernel & flash-kernel-installer for > chromebooks, which have size limits on kernel + initramfs + etc. If the > installer step fails due to initramfs size I'll be asking these two > questions to the user with a higher priority, regenerate the initramfs, > and retry the step. Such a flow could be implemented for flash-kernel as > well, and I think having at least the question template here would be great. Looking at the patch: [...] > --- a/debian/templates-arch > +++ b/debian/templates-arch > @@ -12,6 +12,12 @@ Default[armel]: dep > Description: for internal use > Default driver inclusion policy for initramfs-tools > > +Template: base-installer/kernel/linux/initramfs-tools/compression > +Type: string > +Default: gzip > +Description: for internal use > + Default compression method for initramfs-tools [...] > --- a/library.sh > +++ b/library.sh [...] > + if [ "$RET" = xz ] || [ "$RET" = lzma ]; then > + log-output -t base-installer apt-install > xz-utils > + else > + log-output -t base-installer apt-install "$RET" > + fi So it's possible to set any arbitrary string, but if you set it to anything other than "gzip", "xz", or "lzma" then the required compressor might not be installed. (And it's possible to set a string that mkinitramfs doesn't install either.) I think this needs to be a "select" type question, so that only the supported compression types can be selected. Ben. -- Ben Hutchings I'm not a reverse psychological virus. Please don't copy me into your signature. signature.asc Description: This is a digitally signed message part
Bug#950086: base-installer: please support configuring initramfs compression
Source: base-installer Version: 1.192 Severity: wishlist Tags: patch Dear Maintainers, Debian-installer can already configure initramfs-tools' MODULES setting to get a smaller initramfs. I've written a patch for configuring its COMPRESS setting as well, please consider applying it. I've tested it using an arm64 virtual machine with a custom built netboot image. I'm doing something like flash-kernel & flash-kernel-installer for chromebooks, which have size limits on kernel + initramfs + etc. If the installer step fails due to initramfs size I'll be asking these two questions to the user with a higher priority, regenerate the initramfs, and retry the step. Such a flow could be implemented for flash-kernel as well, and I think having at least the question template here would be great. -- System Information: Debian Release: bullseye/sid APT prefers unstable APT policy: (500, 'unstable'), (1, 'experimental') Architecture: arm64 (aarch64) Kernel: Linux 5.4.0-3-arm64 (SMP w/6 CPU cores) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB:en (charmap=UTF-8) Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled >From 2864137a59bcfbc957a7e8960b65e3e39977106a Mon Sep 17 00:00:00 2001 From: Alper Nebi Yasak Date: Mon, 27 Jan 2020 21:16:10 +0300 Subject: [PATCH] Support configuring initramfs compression Some machines/bootloaders do not support initramfs images larger than a certain size. Setting MODULES=dep in initramfs-tools config is usually enough to satisfy these size limits, and base-installer already asks a question for this (driver-policy, with priority medium). This patch implements a similar question for initramfs-tools' COMPRESS setting which can further reduce the initramfs size for these machines. Signed-off-by: Alper Nebi Yasak --- debian/bootstrap-base.templates | 11 +++ debian/templates-arch | 6 ++ library.sh | 22 ++ 3 files changed, 39 insertions(+) diff --git a/debian/bootstrap-base.templates b/debian/bootstrap-base.templates index 78a6f29a..7a53cad3 100644 --- a/debian/bootstrap-base.templates +++ b/debian/bootstrap-base.templates @@ -88,6 +88,17 @@ _Description: Drivers to include in the initrd: smaller targeted initrd there is a very small chance that not all needed drivers are included. +Template: base-installer/initramfs-tools/compression +Type: select +Choices: lzop, lz4, gzip, bzip2, xz, lzma +# :sl3: +_Description: Compression method for the initramfs image: + Compressing the initramfs is usually a trade-off between initramfs + size, memory use, compression speed, and the time your system takes to + boot (decompression speed). Gzip is reasonably balanced, xz and lzma + makes the initramfs significantly smaller, but lz4 has the highest + decompression speed. + Template: base-installer/kernel/failed-install Type: error # :sl2: diff --git a/debian/templates-arch b/debian/templates-arch index 30e8f7a8..873dc001 100644 --- a/debian/templates-arch +++ b/debian/templates-arch @@ -12,6 +12,12 @@ Default[armel]: dep Description: for internal use Default driver inclusion policy for initramfs-tools +Template: base-installer/kernel/linux/initramfs-tools/compression +Type: string +Default: gzip +Description: for internal use + Default compression method for initramfs-tools + Template: base-installer/kernel/linux/extra-packages Type: string Default: diff --git a/library.sh b/library.sh index d7f05024..d2859566 100644 --- a/library.sh +++ b/library.sh @@ -575,8 +575,15 @@ EOF db_get base-installer/kernel/linux/initramfs-tools/driver-policy db_set base-installer/initramfs-tools/driver-policy "$RET" fi + if db_get base-installer/initramfs-tools/compression && \ + [ -z "$RET" ]; then + # Get default for architecture + db_get base-installer/kernel/linux/initramfs-tools/compression + db_set base-installer/initramfs-tools/compression "$RET" + fi db_settitle debian-installer/bootstrap-base/title db_input medium base-installer/initramfs-tools/driver-policy || true + db_input medium base-installer/initramfs-tools/compression || true if ! db_go; then db_progress stop exit 10 @@ -591,6 +598,21 @@ EOF MODULES=$RET EOF fi + + db_get base-installer/initramfs-tools/compression + if [ "$RET" != gzip ]; then + cat > $IT_CONFDIR/compression <
