Re: [XEN PATCH 2/2] x86/Dom0: Use streaming decompression for ZSTD compressed kernels

2023-05-10 Thread Jan Beulich
On 10.05.2023 19:30, Rafaël Kooi wrote: > On 10/05/2023 11:48, Jan Beulich wrote: >> On 10.05.2023 10:51, Rafaël Kooi wrote: >>> On 10/05/2023 10:03, Jan Beulich wrote:> On 10.05.2023 02:18, Rafaël Kooi >>> wrote: > --- a/xen/common/decompress.c > +++ b/xen/common/decompress.c > @@ -3,

Re: [XEN PATCH 2/2] x86/Dom0: Use streaming decompression for ZSTD compressed kernels

2023-05-10 Thread Rafaël Kooi
On 10/05/2023 11:48, Jan Beulich wrote:> First of all - please don't drop Cc-s when replying. I'm restoring xen-devel@ here at least. Apologies, I didn't notice that replying dropped the Cc-s. Should I send the emails again with the proper Cc-s? On 10.05.2023 10:51, Rafaël Kooi wrote: On 1

Re: [XEN PATCH 2/2] x86/Dom0: Use streaming decompression for ZSTD compressed kernels

2023-05-10 Thread Jan Beulich
First of all - please don't drop Cc-s when replying. I'm restoring xen-devel@ here at least. On 10.05.2023 10:51, Rafaël Kooi wrote: > On 10/05/2023 10:03, Jan Beulich wrote:> On 10.05.2023 02:18, Rafaël Kooi > wrote: >>> On Arch Linux kernel decompression will fail when Xen has been unified >>>

Re: [XEN PATCH 2/2] x86/Dom0: Use streaming decompression for ZSTD compressed kernels

2023-05-10 Thread Jan Beulich
On 10.05.2023 02:18, Rafaël Kooi wrote: > On Arch Linux kernel decompression will fail when Xen has been unified > with the kernel and initramfs as a single binary. This change works for > both streaming and non-streaming ZSTD content. This could do with better explaining what "unified" means, and

[XEN PATCH 2/2] x86/Dom0: Use streaming decompression for ZSTD compressed kernels

2023-05-09 Thread Rafaël Kooi
On Arch Linux kernel decompression will fail when Xen has been unified with the kernel and initramfs as a single binary. This change works for both streaming and non-streaming ZSTD content. Signed-off-by: Rafaël Kooi --- xen/common/decompress.c | 37 +++-- 1 file

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-26 Thread Ian Jackson
Jan Beulich writes ("Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels"): > Anyway - I guess we should continue from v3, which I hope to post > later this morning. Thanks, yes. I just wanted to reply to one bit of this: > My initial attempt to avoid confi

Re: [PATCH v3 02/15] libxenguest: support zstd compressed kernels

2021-01-26 Thread Ian Jackson
Jan Beulich writes ("[PATCH v3 02/15] libxenguest: support zstd compressed kernels"): > This follows the logic used for other decompression methods utilizing an > external library, albeit here we can't ignore the 32-bit size field > appended to the compressed ima

[PATCH v3 02/15] libxenguest: support zstd compressed kernels

2021-01-26 Thread Jan Beulich
This follows the logic used for other decompression methods utilizing an external library, albeit here we can't ignore the 32-bit size field appended to the compressed image - its presence causes decompression to fail. Leverage the field instead to allocate the output buffer in one go, i.e. without

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Jan Beulich
On 25.01.2021 18:30, Ian Jackson wrote: > Jan Beulich writes ("Re: [PATCH v2.5 1/5] libxenguest: support zstd > compressed kernels"): >> On 25.01.2021 17:17, Ian Jackson wrote: >>> I think we had concluded not to print a warning ? >> >> Yes. Even in the

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Ian Jackson
The quoted-reply part of this message may be going off into the weeds. Feel free to ignore it, or parts of it, if you think you can make progress without disabusing me of what I think are my misunderstandings... Jan Beulich writes ("Re: [PATCH v2.5 1/5] libxenguest: support zstd compr

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Jan Beulich
On 25.01.2021 17:17, Ian Jackson wrote: > Jan Beulich writes ("Re: [PATCH v2.5 1/5] libxenguest: support zstd > compressed kernels"): >> On 25.01.2021 15:53, Ian Jackson wrote: >>> Well how about passing "true" for the fourth argument then ? >> &g

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Ian Jackson
I have a feeling we may be talking at cross purposes rather too much. Jan Beulich writes ("Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels"): > On 25.01.2021 15:53, Ian Jackson wrote: > > Well how about passing "true" for the fourth argume

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Jan Beulich
On 25.01.2021 15:53, Ian Jackson wrote: > Jan Beulich writes ("Re: [PATCH v2.5 1/5] libxenguest: support zstd > compressed kernels"): >> On 25.01.2021 14:51, Ian Jackson wrote: >>> I mean, the parts where you examine libzstd_PKG_ERRORS. >> >> The only

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Ian Jackson
Jan Beulich writes ("Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels"): > On 25.01.2021 14:51, Ian Jackson wrote: > > I mean, the parts where you examine libzstd_PKG_ERRORS. > > The only thing I make use of is it being (non-)empty. Do you > really t

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Jan Beulich
On 25.01.2021 14:51, Ian Jackson wrote: > Jan Beulich writes ("Re: [PATCH v2.5 1/5] libxenguest: support zstd > compressed kernels"): >> On 25.01.2021 12:30, Ian Jackson wrote: >>>> As far as configure.ac goes, I'm pretty sure there is a bet

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Ian Jackson
Jan Beulich writes ("Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels"): > On 25.01.2021 12:30, Ian Jackson wrote: > >> As far as configure.ac goes, I'm pretty sure there is a better (more > >> "standard") way of using PKG_CHECK_

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Jan Beulich
On 25.01.2021 12:30, Ian Jackson wrote: > Hi. Thanks for this. Firstly, RM hat: this is the tools half of zstd > decompression support which I think is a blocker for the release. So > I am going to waive the last posting date requirement. Therefore, > > Assuming it's committed this week: > >

Re: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-25 Thread Ian Jackson
Hi. Thanks for this. Firstly, RM hat: this is the tools half of zstd decompression support which I think is a blocker for the release. So I am going to waive the last posting date requirement. Therefore, Assuming it's committed this week: Release-Acked-by: Ian Jackson Secondly, I think it

[PATCH v2.5 1/5] libxenguest: support zstd compressed kernels

2021-01-21 Thread Jan Beulich
This follows the logic used for other decompression methods utilizing an external library, albeit here we can't ignore the 32-bit size field appended to the compressed image - its presence causes decompression to fail. Leverage the field instead to allocate the output buffer in one go, i.e. without

Re: [PATCH v2 1/5] libxenguest: support zstd compressed kernels

2021-01-21 Thread Wei Liu
On Thu, Jan 21, 2021 at 04:05:39PM +0100, Jan Beulich wrote: > On 21.01.2021 16:01, Wei Liu wrote: > > On Tue, Jan 19, 2021 at 04:15:25PM +0100, Jan Beulich wrote: > >> This follows the logic used for other decompression methods utilizing an > >> external library, albeit here we can't ignore the 32

Re: [PATCH v2 1/5] libxenguest: support zstd compressed kernels

2021-01-21 Thread Jan Beulich
On 21.01.2021 16:01, Wei Liu wrote: > On Tue, Jan 19, 2021 at 04:15:25PM +0100, Jan Beulich wrote: >> This follows the logic used for other decompression methods utilizing an >> external library, albeit here we can't ignore the 32-bit size field >> appended to the compressed image - its presence ca

Re: [PATCH v2 1/5] libxenguest: support zstd compressed kernels

2021-01-21 Thread Wei Liu
On Tue, Jan 19, 2021 at 04:15:25PM +0100, Jan Beulich wrote: > This follows the logic used for other decompression methods utilizing an > external library, albeit here we can't ignore the 32-bit size field > appended to the compressed image - its presence causes decompression to > fail. Leverage th

Re: zstd compressed kernels

2021-01-20 Thread Jan Beulich
On 20.01.2021 00:10, Michael Young wrote: > I have been trying the "[PATCH v2 1/5] libxenguest: support zstd > compressed kernel" patch, and (assuming I haven't broken anything trying > to migrate it to 4.14) it fails with > > onfigure: error: Package requirements (libzstd) were not met: > > Pa

Re: zstd compressed kernels

2021-01-19 Thread Michael Young
I have been trying the "[PATCH v2 1/5] libxenguest: support zstd compressed kernel" patch, and (assuming I haven't broken anything trying to migrate it to 4.14) it fails with onfigure: error: Package requirements (libzstd) were not met: Package 'libzstd', required by 'virtual:world', not found

[PATCH v2 1/5] libxenguest: support zstd compressed kernels

2021-01-19 Thread Jan Beulich
This follows the logic used for other decompression methods utilizing an external library, albeit here we can't ignore the 32-bit size field appended to the compressed image - its presence causes decompression to fail. Leverage the field instead to allocate the output buffer in one go, i.e. without

Re: zstd compressed kernels

2021-01-16 Thread Michael Young
On Fri, 15 Jan 2021, Jan Beulich wrote: As you will have seen, I've posted a series apparently doing this a little differently, in particular without said LONG_MAX -> INT_MAX transformation. While it works fine this way for me, it would be nice if you could double check it also does for you. Y

Re: [PATCH 3/3] x86/Dom0: support zstd compressed kernels

2021-01-15 Thread Andrew Cooper
On 15/01/2021 10:06, Jan Beulich wrote: > Taken from Linux at commit 1c4dd334df3a ("lib: decompress_unzstd: Limit > output size") for unzstd.c (renamed from decompress_unzstd.c) and > 36f9ff9e03de ("lib: Fix fall-through warnings for Clang") for zstd/, > with bits from linux/zstd.h merged into suit

Re: zstd compressed kernels

2021-01-15 Thread Jan Beulich
On 24.11.2020 00:01, Michael Young wrote: > On Tue, 17 Nov 2020, Andrew Cooper wrote: > >> If you're willing to have a go: >> >> For dom0 support, port Linux's decompressor into xen/common/ and plumb >> it into xen/common/decompress.c >> >> For domU's, tools/libs/guest/xg_dom_bzimageloader.c and >

[PATCH 0/3] x86/Dom0: support zstd compressed kernels

2021-01-15 Thread Jan Beulich
/Dom0: support zstd compressed kernels Jan

Re: zstd compressed kernels

2020-11-17 Thread Andy Smith
On Tue, Nov 17, 2020 at 08:48:25PM +, Andrew Cooper wrote: > For domU's, tools/libs/guest/xg_dom_bzimageloader.c and > xc_dom_probe_bzimage_kernel() > > (Wow this plumbing is ugly and in need of some rationalisation...) Though not part of Xen, the PV part of grub could also do with some love

Re: zstd compressed kernels

2020-11-17 Thread Andrew Cooper
On 17/11/2020 20:27, Michael Young wrote: > Is anyone else looking at vmlinuz files which use zstd compression? > Fedora has started doing so with its 5.9 kernels. Well volunteered ;) Yes - I'm aware that it is an area needing working on, but it is not sufficiently urgent on my TODO stack to look

zstd compressed kernels

2020-11-17 Thread Michael Young
Is anyone else looking at vmlinuz files which use zstd compression? Fedora has started doing so with its 5.9 kernels. Michael Young