Re: Reducing initramfs size and speed up the generation
On 7/8/23 21:42, Steve Langasek wrote: Hi Heinrich, On Sat, Jul 08, 2023 at 10:44:58AM +0200, Heinrich Schuchardt wrote: Hello Benjamin, some of the Ubuntu supported boards (e.g. LicheeRV) have so little RAM that a default initrd cannot be decompressed. Some boards may need modules that are not in initrd by default. Can you clarify where exactly this limitation is? Is this because the system must have room in memory both for the original initramfs image as it is loaded into memory from disk, regardless of compression; plus an uncompressed copy used as the VFS? If that's the case, wouldn't having the kernel modules compressed inside of an uncompressed cpio archive reduce overall memory usage, because the unpacking of the initramfs doesn't uncompress these compressed files? This requires rebuilding the kernel with CONFIG_MODULE_COMPRESS_ZSTD=y. The kmod package which provide depmod is already build with --with-zstd. In can look into this. Benjamin Drung schrieb am Sa., 8. Juli 2023, 02:19: Hi all, a year ago we changed the default compression and level for the initramfs to zstd -1. This fixed the very slow creation times on development boards (see bug #1958148), but that leads to bigger initramfs sizes that triggered other bugs (like bug #1842320). Big initramfs sizes can also fill up small sized /boot partitions easily (grooming the 850 initramfs-tools bugs revealed several such reports). Using xz -9 would give very good compression, but it takes very long (especially on slow development boards) and a lot of memory (good luck on Raspberry Pis with small memory like Pi Zeros). I propose following approach to address the drawback: Create cpio archives (compressed with xz -9) for the kernel modules and firmware files when building the kernel/firmware Debian package. Then ship those cpio archives in the package (or in a separate binary package). Then the CPU load it put on the builders. The cpio archives would contain the modules for MODULES=most. mkinitramfs will then look for those cpio archives and uses those in case they are present. Such a initramfs would look like this: * AMD/Intel microcode cpio archive (on amd64) * main cpio archive compressed with zstd -1 * kernel modules from the Debian package compressed with xz -9 * firmware files from the Debian package compressed with xz -9 After working on initramfs-tools as part my day job, my fingers were itching and I had to create a quick and dirty draft in my free night time. You can find the result of the last two hours in [1]. This draft has a mkinitramfs-kernel script that creates a cpio archive containing the kernel modules and firmware (that needs to be split later on). The lunar test result on my AMD Ryzen 7 5700G look promising: Building 6.2.0-24-generic-modules-most.cpio.xz takes around 90 seconds and is 54.9 MiB in size. Creating the initramfs speeds up from around 8.7 seconds to 3.5 seconds (saves 60 %). The size reduces from 133.1 MiB to 80.7 MiB (saves 39.4 %). So the boot needs 52.4 MiB less, but /lib/modules need 54.9 MiB for the cpio archive. The drawback is that building the kernel would take longer, the package takes more space on the archive and mirrors, and downloading them could take longer on slow connections. Implementing my proposal would be relative easy for initramfs-tools, but would mean some work for the kernel team. What do you think? Will the user still be able to add further modules and will machine specific configuration files (e.g. for booting from iSCSI) still be included into the initrd? Best regards Heinrich [1] https://code.launchpad.net/~bdrung/ubuntu/+source/initramfs-tools/+git/initramfs-tools/+ref/ubuntu/prebuilt -- Benjamin Drung Debian & Ubuntu Developer -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Reducing initramfs size and speed up the generation
Hi Heinrich, On Sat, Jul 08, 2023 at 10:44:58AM +0200, Heinrich Schuchardt wrote: > Hello Benjamin, > some of the Ubuntu supported boards (e.g. LicheeRV) have so little RAM that > a default initrd cannot be decompressed. Some boards may need modules that > are not in initrd by default. Can you clarify where exactly this limitation is? Is this because the system must have room in memory both for the original initramfs image as it is loaded into memory from disk, regardless of compression; plus an uncompressed copy used as the VFS? If that's the case, wouldn't having the kernel modules compressed inside of an uncompressed cpio archive reduce overall memory usage, because the unpacking of the initramfs doesn't uncompress these compressed files? > Benjamin Drung schrieb am Sa., 8. Juli 2023, 02:19: > > > Hi all, > > > > a year ago we changed the default compression and level for the > > initramfs to zstd -1. This fixed the very slow creation times on > > development boards (see bug #1958148), but that leads to bigger > > initramfs sizes that triggered other bugs (like bug #1842320). > > Big initramfs sizes can also fill up small sized /boot partitions easily > > (grooming the 850 initramfs-tools bugs revealed several such reports). > > > > Using xz -9 would give very good compression, but it takes very long > > (especially on slow development boards) and a lot of memory (good luck > > on Raspberry Pis with small memory like Pi Zeros). > > > > I propose following approach to address the drawback: Create cpio > > archives (compressed with xz -9) for the kernel modules and firmware > > files when building the kernel/firmware Debian package. Then ship those > > cpio archives in the package (or in a separate binary package). Then the > > CPU load it put on the builders. The cpio archives would contain the > > modules for MODULES=most. > > > > mkinitramfs will then look for those cpio archives and uses those in > > case they are present. Such a initramfs would look like this: > > > > * AMD/Intel microcode cpio archive (on amd64) > > * main cpio archive compressed with zstd -1 > > * kernel modules from the Debian package compressed with xz -9 > > * firmware files from the Debian package compressed with xz -9 > > > > After working on initramfs-tools as part my day job, my fingers were > > itching and I had to create a quick and dirty draft in my free night > > time. You can find the result of the last two hours in [1]. This draft > > has a mkinitramfs-kernel script that creates a cpio archive containing > > the kernel modules and firmware (that needs to be split later on). > > > > The lunar test result on my AMD Ryzen 7 5700G look promising: Building > > 6.2.0-24-generic-modules-most.cpio.xz takes around 90 seconds and is > > 54.9 MiB in size. Creating the initramfs speeds up from around 8.7 > > seconds to 3.5 seconds (saves 60 %). The size reduces from 133.1 MiB to > > 80.7 MiB (saves 39.4 %). So the boot needs 52.4 MiB less, but > > /lib/modules need 54.9 MiB for the cpio archive. > > > > The drawback is that building the kernel would take longer, the package > > takes more space on the archive and mirrors, and downloading them could > > take longer on slow connections. > > > > Implementing my proposal would be relative easy for initramfs-tools, but > > would mean some work for the kernel team. > > > > What do you think? > > > > [1] > > https://code.launchpad.net/~bdrung/ubuntu/+source/initramfs-tools/+git/initramfs-tools/+ref/ubuntu/prebuilt > > > > -- > > Benjamin Drung > > Debian & Ubuntu Developer -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Reducing initramfs size and speed up the generation
Hello Benjamin, some of the Ubuntu supported boards (e.g. LicheeRV) have so little RAM that a default initrd cannot be decompressed. Some boards may need modules that are not in initrd by default. Best regards Heinrich Benjamin Drung schrieb am Sa., 8. Juli 2023, 02:19: > Hi all, > > a year ago we changed the default compression and level for the > initramfs to zstd -1. This fixed the very slow creation times on > development boards (see bug #1958148), but that leads to bigger > initramfs sizes that triggered other bugs (like bug #1842320). > Big initramfs sizes can also fill up small sized /boot partitions easily > (grooming the 850 initramfs-tools bugs revealed several such reports). > > Using xz -9 would give very good compression, but it takes very long > (especially on slow development boards) and a lot of memory (good luck > on Raspberry Pis with small memory like Pi Zeros). > > I propose following approach to address the drawback: Create cpio > archives (compressed with xz -9) for the kernel modules and firmware > files when building the kernel/firmware Debian package. Then ship those > cpio archives in the package (or in a separate binary package). Then the > CPU load it put on the builders. The cpio archives would contain the > modules for MODULES=most. > > mkinitramfs will then look for those cpio archives and uses those in > case they are present. Such a initramfs would look like this: > > * AMD/Intel microcode cpio archive (on amd64) > * main cpio archive compressed with zstd -1 > * kernel modules from the Debian package compressed with xz -9 > * firmware files from the Debian package compressed with xz -9 > > After working on initramfs-tools as part my day job, my fingers were > itching and I had to create a quick and dirty draft in my free night > time. You can find the result of the last two hours in [1]. This draft > has a mkinitramfs-kernel script that creates a cpio archive containing > the kernel modules and firmware (that needs to be split later on). > > The lunar test result on my AMD Ryzen 7 5700G look promising: Building > 6.2.0-24-generic-modules-most.cpio.xz takes around 90 seconds and is > 54.9 MiB in size. Creating the initramfs speeds up from around 8.7 > seconds to 3.5 seconds (saves 60 %). The size reduces from 133.1 MiB to > 80.7 MiB (saves 39.4 %). So the boot needs 52.4 MiB less, but > /lib/modules need 54.9 MiB for the cpio archive. > > The drawback is that building the kernel would take longer, the package > takes more space on the archive and mirrors, and downloading them could > take longer on slow connections. > > Implementing my proposal would be relative easy for initramfs-tools, but > would mean some work for the kernel team. > > What do you think? > > [1] > https://code.launchpad.net/~bdrung/ubuntu/+source/initramfs-tools/+git/initramfs-tools/+ref/ubuntu/prebuilt > > -- > Benjamin Drung > Debian & Ubuntu Developer > > -- > ubuntu-devel mailing list > ubuntu-devel@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel > -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
