Re: Revisiting default initramfs compression

2022-03-15 Thread Michael Hudson-Doyle
On Fri, 11 Mar 2022 at 02:39, Dave Jones wrote: > On Thu, Mar 10, 2022 at 12:10:39PM +0100, Julian Andres Klode wrote: > >On Wed, Mar 09, 2022 at 01:24:55PM +, Dave Jones wrote: > [snip] > >> Firstly I actually think lz4 -2 is probably the ideal level for that > >> compressor. There's a

Re: Revisiting default initramfs compression

2022-03-10 Thread Dave Jones
On Thu, Mar 10, 2022 at 12:10:39PM +0100, Julian Andres Klode wrote: On Wed, Mar 09, 2022 at 01:24:55PM +, Dave Jones wrote: [snip] Firstly I actually think lz4 -2 is probably the ideal level for that compressor. There's a large difference in compression performance between lz4 -1 and lz4

Re: Revisiting default initramfs compression

2022-03-10 Thread Julian Andres Klode
On Wed, Mar 09, 2022 at 01:24:55PM +, Dave Jones wrote: > Hi Michael (and others), > > Julian's summarised this near perfectly, but I'll try and add a little > detail from the data I've gathered [1] (with others' generous help, in > particular Heinrich for the RISC-V bits): > > On Wed, Mar

Re: Revisiting default initramfs compression

2022-03-09 Thread Michael Hudson-Doyle
On Thu, 10 Mar 2022 at 02:24, Dave Jones wrote: > Hi Michael (and others), > > Julian's summarised this near perfectly, but I'll try and add a little > detail from the data I've gathered [1] (with others' generous help, in > particular Heinrich for the RISC-V bits): > > On Wed, Mar 09, 2022 at

Re: Revisiting default initramfs compression

2022-03-09 Thread Dave Jones
Hi Michael (and others), Julian's summarised this near perfectly, but I'll try and add a little detail from the data I've gathered [1] (with others' generous help, in particular Heinrich for the RISC-V bits): On Wed, Mar 09, 2022 at 09:46:19AM +0100, Julian Andres Klode wrote: On Wed, Mar

Re: Revisiting default initramfs compression

2022-03-09 Thread Paul Sladen
On Wed, 9 Mar 2022, Julian Andres Klode wrote: > On Wed, Mar 09, 2022 at 02:10:57PM +1300, Michael Hudson-Doyle wrote: > > On Thu, 9 Dec 2021 at 06:13, Julian Andres Klode > > > > wrote: > > > changed from lz4 -9 to zstd -19. This caused significant problems: > > change to initramfs-tools to

Re: Revisiting default initramfs compression

2022-03-09 Thread Julian Andres Klode
On Wed, Mar 09, 2022 at 02:10:57PM +1300, Michael Hudson-Doyle wrote: > On Thu, 9 Dec 2021 at 06:13, Julian Andres Klode > wrote: > > > Hi all, > > > > some time ago, the default compressor for initramfs was changed > > from lz4 -9 to zstd -19. This caused significant problems: > > > > Exactly

Re: Revisiting default initramfs compression

2022-03-08 Thread Michael Hudson-Doyle
On Thu, 9 Dec 2021 at 06:13, Julian Andres Klode wrote: > Hi all, > > some time ago, the default compressor for initramfs was changed > from lz4 -9 to zstd -19. This caused significant problems: > Exactly three months later... we still haven't taken any action on this. Time to do something! I

Re: Revisiting default initramfs compression

2022-01-26 Thread Christian Ehrhardt
On Wed, Jan 26, 2022 at 4:18 PM Robie Basak wrote: > > It was noted in ubuntu-devel-discuss@ that changing the compression > method impacts Xen's use of pygrub. I thought that was worth mentioning > in this thread as something worth considering when making any decision. Agreed > >

Re: Revisiting default initramfs compression

2022-01-26 Thread Robie Basak
It was noted in ubuntu-devel-discuss@ that changing the compression method impacts Xen's use of pygrub. I thought that was worth mentioning in this thread as something worth considering when making any decision. https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2022-January/019165.html

Re: Revisiting default initramfs compression

2021-12-09 Thread Oliver Grawert
hi, Am Donnerstag, dem 09.12.2021 um 08:51 +0200 schrieb Mark Shuttleworth: > On 08/12/2021 23:02, Julian Andres Klode wrote: > > I'd kind of like us to ship "default" initramfs in like > > linux-initrd-$uname-r > > and linux-initrd-generic and so on. Maybe even signed somehow so > > that > > the

Re: Revisiting default initramfs compression

2021-12-09 Thread Mark Shuttleworth
On 08/12/2021 23:02, Julian Andres Klode wrote: I'd kind of like us to ship "default" initramfs in like linux-initrd-$uname-r and linux-initrd-generic and so on. Maybe even signed somehow so that the kernel can verify its integrity when booting. Such that booting with authenticated FDE is fully

Re: Revisiting default initramfs compression

2021-12-08 Thread Dimitri John Ledkov
On Wed, 8 Dec 2021, 17:13 Julian Andres Klode, wrote: > Hi all, > > some time ago, the default compressor for initramfs was changed > from lz4 -9 to zstd -19. This caused significant problems: > > - it is very slow > - it uses a lot of memory > > The former is a problem for everyone, the latter

Re: Revisiting default initramfs compression

2021-12-08 Thread Dave Jones
Firstly, many thanks for Julian for starting this thread; I've had far too much fun delving back into some SQL+jupyter notebooks today while gathering a bit more info on this... If you just want to go see or play with the data (and possibly expand it with some more samples; I'd love to see

Re: Revisiting default initramfs compression

2021-12-08 Thread Steve Langasek
Thanks for raising this discussion, Julian! On Wed, Dec 08, 2021 at 06:12:43PM +0100, Julian Andres Klode wrote: > ## Adaptive compression > zstd also supports adaptive compression, compressing as hard as > it can while not impacting I/O speed. So hardware with slow I/O > like a Pi would

Re: Revisiting default initramfs compression

2021-12-08 Thread Michael Hudson-Doyle
On Thu, 9 Dec 2021 at 10:02, Julian Andres Klode wrote: > On Thu, Dec 09, 2021 at 09:21:35AM +1300, Michael Hudson-Doyle wrote: > > On Thu, 9 Dec 2021 at 08:52, Julian Andres Klode < > julian.kl...@canonical.com> > > wrote: > > > > > The most interesting solution would be to create the cpip

Re: Revisiting default initramfs compression

2021-12-08 Thread Julian Andres Klode
On Thu, Dec 09, 2021 at 09:21:35AM +1300, Michael Hudson-Doyle wrote: > On Thu, 9 Dec 2021 at 08:52, Julian Andres Klode > wrote: > > > The most interesting solution would be to create the cpip archive > > dynamically by giving cpio a list of files over a pipe that we create > > dynamically in

Re: Revisiting default initramfs compression

2021-12-08 Thread Michael Hudson-Doyle
On Thu, 9 Dec 2021 at 08:52, Julian Andres Klode wrote: > The most interesting solution would be to create the cpip archive > dynamically by giving cpio a list of files over a pipe that we create > dynamically in the hooks as we copy files in, then pipe that to zstd > --adapt; then it would all

Re: Revisiting default initramfs compression

2021-12-08 Thread Heinrich Schuchardt
On 12/8/21 09:12, Julian Andres Klode wrote: Hi all, some time ago, the default compressor for initramfs was changed from lz4 -9 to zstd -19. This caused significant problems: - it is very slow - it uses a lot of memory The former is a problem for everyone, the latter means that zstd just

Re: Revisiting default initramfs compression

2021-12-08 Thread Julian Andres Klode
On Wed, Dec 08, 2021 at 09:51:44AM -0800, Heinrich Schuchardt wrote: > The memory should only be one factor to be taken into account. The SiFive > Unmatched and the RPi 4 boards both have 8 GiB but slow CPUs. It would be > advisable to use a lower compression level on these. The Pi 4 is limited