Package: initramfs-tools
Version: 0.140
Severity: wishlist
This is a vote for
https://salsa.debian.org/kernel-team/initramfs-tools/-/merge_requests/52
I did this investigation 2 months ago, but AFAICT I forgot to push it to
bugs.debian.org.
https://github.com/cyberitsolutions/bootstrap2020/blob/main/doc/NNNNN-ramdisk-compression.rst
Are pigz and xz *REALLY* the best choices for rd compression?
Surely lz4 and zstd are better tradeoffs?
Looking at [a Debian Live chroot]::
# apt install pixz pigz zstd lz4 xz-utils firmware-misc-nonfree
# for i in lz4 gzip xz zstd;
do
echo === $i === &&
echo COMPRESS=$i >/etc/initramfs-tools/conf.d/test &&
time update-initramfs -u -k all &&
ls -hl /boot/initrd.img-5.14.0-0.bpo.2-amd64;
done
COMPRESS real user sys size
lz4 0m10.125s 0m9.263s 0m1.242s 55M
gzip 0m5.724s 0m11.860s 0m1.123s 47M (really
pigz)
xz 0m18.556s 1m15.392s 0m1.307s 32M
zstd 0m25.993s 1m20.542s 0m1.237s 35M
So:
• pigz greatly beats lz4 for wall-clock time.
pigz beats lz4 for size.
lz4 slightly beats pigz for CPU time (meh).
pigz is the best choice for --optimize=speed.
• xz slightly beats zstd for size.
xz beats zstd for wall-clock time.
xz slightly beats zstd for CPU time (meh).
xz is the best choice for --optimize=size.
BUT /usr/sbin/mkinitramfs makes these UNFAIR COMPARISONS.
It uses the HIGHEST compression level for lz4 and zstd, but
the DEFAULT (best tradeoff) compression for gzip and xz. ::
case "${compress}" in
gzip) # If we're doing a reproducible build, use gzip -n
if [ -n "${SOURCE_DATE_EPOCH}" ]; then
compress="gzip -n"
# Otherwise, substitute pigz if it's available
elif command -v pigz >/dev/null; then
compress=pigz
fi
;;
lz4) compress="lz4 -9 -l" ;;
zstd) compress="zstd -q -19 -T0" ;;
xz) compress="xz --check=crc32"
# If we're not doing a reproducible build, enable multithreading
test -z "${SOURCE_DATE_EPOCH}" && compress="$compress --threads=0"
;;
bzip2|lzma|lzop)
# no parameters needed
;;
*) echo "W: Unknown compression command ${compress}" >&2 ;;
esac
Just for my peace of mind, let's re-test this with the -9 and -19 removed::
# sed -rsi /usr/sbin/mkinitramfs -e 's/ -19 / /' -e 's/ -9 / /'
# apt install pixz pigz zstd lz4 xz-utils firmware-misc-nonfree
# for i in lz4 gzip xz zstd;
do
echo === $i === &&
echo COMPRESS=$i >/etc/initramfs-tools/conf.d/test &&
time update-initramfs -u -k all &&
ls -hl /boot/initrd.img-5.14.0-0.bpo.2-amd64;
done
COMPRESS real user sys size
lz4 0m5.070s 0m4.207s 0m1.209s 67M
gzip 0m5.572s 0m11.308s 0m1.197s 47M (really
pigz)
xz 0m18.646s 1m14.563s 0m1.204s 32M
zstd 0m5.159s 0m5.334s 0m1.137s 43M
So:
• When lz4 isn't forced into a bad time/size tradeoff,
it's as fast as pigz, but much bigger. Fail.
• When zstd isn't forced into a bad time/size tradeoff,
it's a little smaller than pigz,
it's as fast as pigz, and
it's MUCH faster than xz.
Clear win.
It seems to me that the following changes should be made:
• Don't pass -19 to zstd.
• Don't pass -T0 to zstd when [ -n $SOURCE_DATE_EPOCH ] (same as other -T0
cases).
• Encourage people to switch to zstd? ;-)
-- Package-specific info:
-- initramfs sizes
-rw-r--r-- 1 root root 53M 2021-11-19 09:17
/boot/initrd.img-5.14.0-0.bpo.2-amd64
-- /proc/cmdline
root=ZFS=hera/hera quiet splash noresume initrd=\initrd.img-5.14.0-0.bpo.2-amd64
-- /proc/filesystems
fuseblk
ext3
ext2
ext4
vfat
-- lsmod
Module Size Used by
ccm 20480 6
rfcomm 90112 0
cmac 16384 7
algif_hash 16384 3
algif_skcipher 16384 3
af_alg 32768 14 algif_hash,algif_skcipher
bnep 28672 2
binfmt_misc 24576 1
intel_pmc_core_pltdrv 16384 0
intel_pmc_core 45056 0
snd_sof_pci_intel_cnl 16384 0
snd_sof_intel_hda_common 106496 1 snd_sof_pci_intel_cnl
x86_pkg_temp_thermal 20480 0
soundwire_intel 45056 1 snd_sof_intel_hda_common
intel_powerclamp 20480 0
coretemp 20480 0
soundwire_generic_allocation 16384 1 soundwire_intel
soundwire_cadence 36864 1 soundwire_intel
snd_sof_intel_hda 20480 1 snd_sof_intel_hda_common
kvm_intel 323584 0
snd_sof_pci 20480 2 snd_sof_intel_hda_common,snd_sof_pci_intel_cnl
snd_sof_xtensa_dsp 16384 1 snd_sof_intel_hda_common
snd_sof 147456 2 snd_sof_pci,snd_sof_intel_hda_common
soundwire_bus 94208 3
soundwire_intel,soundwire_generic_allocation,soundwire_cadence
snd_hda_codec_hdmi 73728 1
kvm 1019904 1 kvm_intel
snd_soc_skl 180224 0
snd_soc_hdac_hda 24576 2 snd_sof_intel_hda_common,snd_soc_skl
btusb 65536 0
snd_hda_ext_core 36864 4
snd_sof_intel_hda_common,snd_soc_hdac_hda,snd_soc_skl,snd_sof_intel_hda
snd_ctl_led 24576 0
btrtl 28672 1 btusb
wireguard 94208 0
snd_soc_sst_ipc 20480 1 snd_soc_skl
btbcm 20480 1 btusb
btintel 32768 1 btusb
snd_soc_sst_dsp 36864 1 snd_soc_skl
snd_soc_acpi_intel_match 53248 3
snd_sof_intel_hda_common,snd_soc_skl,snd_sof_pci_intel_cnl
irqbypass 16384 1 kvm
snd_soc_acpi 16384 3
snd_soc_acpi_intel_match,snd_sof_intel_hda_common,snd_soc_skl
snd_hda_codec_realtek 159744 1
libchacha20poly1305 16384 1 wireguard
ghash_clmulni_intel 16384 0
mei_hdcp 24576 0
snd_hda_codec_generic 98304 1 snd_hda_codec_realtek
intel_rapl_msr 20480 0
snd_soc_core 331776 5
soundwire_intel,snd_sof,snd_sof_intel_hda_common,snd_soc_hdac_hda,snd_soc_skl
bluetooth 757760 33 btrtl,btintel,btbcm,bnep,btusb,rfcomm
chacha_x86_64 28672 1 libchacha20poly1305
iwlmvm 352256 0
poly1305_x86_64 28672 1 libchacha20poly1305
libblake2s 16384 1 wireguard
snd_compress 32768 1 snd_soc_core
nls_ascii 16384 1
nls_cp437 20480 1
aesni_intel 380928 14
crypto_simd 16384 1 aesni_intel
blake2s_x86_64 20480 1 libblake2s
cryptd 24576 5 crypto_simd,ghash_clmulni_intel
snd_hda_intel 57344 2
vfat 20480 1
fat 86016 1 vfat
rapl 20480 0
curve25519_x86_64 36864 1 wireguard
snd_intel_dspcfg 28672 3
snd_hda_intel,snd_sof_intel_hda_common,snd_soc_skl
libcurve25519_generic 49152 2 curve25519_x86_64,wireguard
snd_intel_sdw_acpi 20480 2 snd_sof_intel_hda_common,snd_intel_dspcfg
jitterentropy_rng 16384 1
intel_cstate 20480 0
libchacha 16384 1 chacha_x86_64
mac80211 1048576 1 iwlmvm
libblake2s_generic 20480 1 blake2s_x86_64
snd_hda_codec 176128 5
snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec_realtek,snd_soc_hdac_hda
libarc4 16384 1 mac80211
ip6_udp_tunnel 16384 1 wireguard
udp_tunnel 20480 1 wireguard
ext4 917504 1
sha512_ssse3 49152 1
intel_uncore 192512 0
snd_hda_core 110592 10
snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hda_intel,snd_hda_ext_core,snd_hda_codec,snd_hda_codec_realtek,snd_sof_intel_hda_common,snd_soc_hdac_hda,snd_soc_skl,snd_sof_intel_hda
serio_raw 20480 0
pcspkr 16384 0
efi_pstore 16384 0
sha512_generic 16384 1 sha512_ssse3
snd_hwdep 16384 1 snd_hda_codec
uvcvideo 118784 0
wmi_bmof 16384 0
iwlwifi 311296 1 iwlmvm
mbcache 16384 1 ext4
iTCO_wdt 16384 0
videobuf2_vmalloc 20480 1 uvcvideo
intel_pmc_bxt 16384 1 iTCO_wdt
intel_wmi_thunderbolt 20480 0
iTCO_vendor_support 16384 1 iTCO_wdt
videobuf2_memops 20480 1 videobuf2_vmalloc
snd_pcm 143360 10
snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec,soundwire_intel,snd_sof,snd_sof_intel_hda_common,snd_compress,snd_soc_core,snd_soc_skl,snd_hda_core
jbd2 167936 1 ext4
videobuf2_v4l2 36864 1 uvcvideo
thinkpad_acpi 131072 0
cdc_ether 24576 0
videobuf2_common 69632 4
videobuf2_vmalloc,videobuf2_v4l2,uvcvideo,videobuf2_memops
usbnet 57344 1 cdc_ether
nft_reject_inet 16384 1
nf_reject_ipv4 16384 1 nft_reject_inet
nf_reject_ipv6 20480 1 nft_reject_inet
nft_reject 16384 1 nft_reject_inet
elan_i2c 53248 0
watchdog 28672 1 iTCO_wdt
snd_timer 49152 1 snd_pcm
ctr 16384 2
videodev 270336 3 videobuf2_v4l2,uvcvideo,videobuf2_common
joydev 28672 0
r8152 126976 0
nvram 16384 1 thinkpad_acpi
platform_profile 16384 1 thinkpad_acpi
ledtrig_audio 16384 4
snd_ctl_led,snd_hda_codec_generic,snd_sof,thinkpad_acpi
mc 65536 4
videodev,videobuf2_v4l2,uvcvideo,videobuf2_common
drbg 40960 1
cfg80211 1024000 3 iwlmvm,iwlwifi,mac80211
mii 16384 2 usbnet,r8152
snd 110592 16
snd_ctl_led,snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hwdep,snd_hda_intel,snd_hda_codec,snd_hda_codec_realtek,snd_timer,snd_compress,thinkpad_acpi,snd_soc_core,snd_pcm
processor_thermal_device_pci_legacy 16384 0
ansi_cprng 16384 0
ucsi_acpi 16384 0
typec_ucsi 49152 1 ucsi_acpi
processor_thermal_device 20480 1 processor_thermal_device_pci_legacy
roles 16384 1 typec_ucsi
nft_ct 20480 1
processor_thermal_rfim 16384 1 processor_thermal_device
mei_me 45056 1
processor_thermal_mbox 16384 2
processor_thermal_rfim,processor_thermal_device
processor_thermal_rapl 20480 1 processor_thermal_device
ecdh_generic 16384 2 bluetooth
intel_rapl_common 28672 2 intel_rapl_msr,processor_thermal_rapl
mei 151552 3 mei_hdcp,mei_me
intel_pch_thermal 20480 0
intel_soc_dts_iosf 20480 1 processor_thermal_device_pci_legacy
typec 65536 1 typec_ucsi
nf_conntrack 176128 1 nft_ct
soundcore 16384 2 snd_ctl_led,snd
ecc 40960 1 ecdh_generic
crc16 16384 2 bluetooth,ext4
rfkill 32768 9 bluetooth,thinkpad_acpi,cfg80211
int3403_thermal 20480 0
ac 20480 0
int340x_thermal_zone 20480 2 int3403_thermal,processor_thermal_device
nf_defrag_ipv6 24576 1 nf_conntrack
nf_defrag_ipv4 16384 1 nf_conntrack
acpi_pad 184320 0
evdev 28672 19
int3400_thermal 20480 0
acpi_thermal_rel 16384 1 int3400_thermal
nf_tables 262144 27 nft_ct,nft_reject_inet,nft_reject
sch_fq_codel 20480 3
libcrc32c 16384 2 nf_conntrack,nf_tables
crc32c_generic 16384 0
nfnetlink 20480 1 nf_tables
sunrpc 663552 1
msr 16384 0
i2c_dev 24576 0
fuse 167936 3
configfs 57344 1
efivarfs 16384 1
ip_tables 36864 0
x_tables 53248 1 ip_tables
autofs4 53248 2
hid_lenovo 28672 0
hid_generic 16384 0
usbhid 65536 0
hid 151552 3 usbhid,hid_generic,hid_lenovo
zfs 4587520 11
zunicode 335872 1 zfs
zzstd 569344 1 zfs
zlua 184320 1 zfs
zavl 16384 1 zfs
icp 331776 1 zfs
zcommon 102400 2 zfs,icp
znvpair 110592 2 zfs,zcommon
spl 118784 6 zfs,icp,zzstd,znvpair,zcommon,zavl
mmc_block 53248 0
i915 2965504 7
rtsx_pci_sdmmc 32768 0
mmc_core 200704 2 rtsx_pci_sdmmc,mmc_block
i2c_algo_bit 16384 1 i915
ttm 86016 1 i915
nvme 49152 4
xhci_pci 20480 0
drm_kms_helper 307200 1 i915
e1000e 311296 0
xhci_hcd 307200 1 xhci_pci
crc32_pclmul 16384 0
psmouse 184320 0
crc32c_intel 24576 3
ptp 32768 1 e1000e
i2c_i801 32768 0
pps_core 24576 1 ptp
nvme_core 139264 6 nvme
cec 61440 2 drm_kms_helper,i915
i2c_smbus 20480 1 i2c_i801
t10_pi 16384 1 nvme_core
rc_core 65536 1 cec
usbcore 331776 9
xhci_hcd,usbnet,usbhid,typec,uvcvideo,btusb,xhci_pci,cdc_ether,r8152
thunderbolt 331776 0
crc_t10dif 20480 1 t10_pi
crct10dif_generic 16384 0
crct10dif_pclmul 16384 1
crct10dif_common 16384 3 crct10dif_generic,crc_t10dif,crct10dif_pclmul
rtsx_pci 110592 1 rtsx_pci_sdmmc
drm 634880 8 drm_kms_helper,i915,ttm
usb_common 16384 3 xhci_hcd,usbcore,uvcvideo
wmi 36864 2 intel_wmi_thunderbolt,wmi_bmof
battery 28672 1 thinkpad_acpi
video 57344 2 thinkpad_acpi,i915
button 24576 0
-- /etc/initramfs-tools/modules
-- /etc/initramfs-tools/initramfs.conf
MODULES=most
BUSYBOX=auto
KEYMAP=n
COMPRESS=gzip
DEVICE=
NFSROOT=auto
RUNSIZE=10%
FSTYPE=auto
-- /etc/initramfs-tools/update-initramfs.conf
update_initramfs=yes
backup_initramfs=no
-- mkinitramfs hooks
/etc/initramfs-tools/hooks/:
/usr/share/initramfs-tools/hooks:
amd64_microcode
dmsetup
fsck
fuse
intel_microcode
keymap
klibc-utils
kmod
ntfs_3g
plymouth
resume
thermal
udev
zdev
zfs
zfsunlock
zz-busybox
-- System Information:
Debian Release: 11.2
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500,
'proposed-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 5.14.0-0.bpo.2-amd64 (SMP w/8 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE,
TAINT_UNSIGNED_MODULE
Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
