Re: [Xen-devel] bug: unable to LZ4 decompress ub1910 installer kernel when launching domU
On Wed, Dec 04, 2019 at 02:27:26PM +0100, Jan Beulich wrote: > On 04.12.2019 08:14, Jeremi Piotrowski wrote: > > Any suggestions how to proceed? > > Actually here's a better version (I think). > > Jan > > lz4: refine commit 9143a6c55ef7 for the 64-bit case > > I clearly went too far there: While the LZ4_WILDCOPY() instances indeed > need prior guarding, LZ4_SECURECOPY() needs this only in the 32-bit case > (where it simply aliases LZ4_WILDCOPY()). "cpy" can validly point > (slightly) below "op" in these cases, due to > > cpy = op + length - (STEPSIZE - 4); > > where length can be as low as 0 and STEPSIZE is 8. However, instead of > removing the check via "#if !LZ4_ARCH64", refine it such that it would > also properly work in the 64-bit case, aborting decompression instead > of continuing on bogus input. > > Reported-by: Mark Pryor > Reported-by: Jeremi Piotrowski > Signed-off-by: Jan Beulich > > --- unstable.orig/xen/common/lz4/decompress.c > +++ unstable/xen/common/lz4/decompress.c > @@ -147,7 +147,7 @@ static int INIT lz4_uncompress(const uns > goto _output_error; > continue; > } > - if (unlikely((unsigned long)cpy < (unsigned long)op)) > + if (unlikely((unsigned long)cpy < (unsigned long)op - (STEPSIZE > - 4))) > goto _output_error; > LZ4_SECURECOPY(ref, op, cpy); > op = cpy; /* correction */ > @@ -279,7 +279,7 @@ static int lz4_uncompress_unknownoutputs > goto _output_error; > continue; > } > - if (unlikely((unsigned long)cpy < (unsigned long)op)) > + if (unlikely((unsigned long)cpy < (unsigned long)op - (STEPSIZE > - 4))) > goto _output_error; > LZ4_SECURECOPY(ref, op, cpy); > op = cpy; /* correction */ > Thanks Jan, this works. I tested it with direct kernel boot. Like Pry wrote pvgrub2 is another story, seems there is no support for lz4 compressed kernels in it , and ubuntu patches in support to the grub that they ship. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] bug: unable to LZ4 decompress ub1910 installer kernel when launching domU
On Tue, Dec 03, 2019 at 09:02:18AM +0100, Jan Beulich wrote: > On 01.12.2019 18:47, Jeremi Piotrowski wrote: > > On Thu, Oct 24, 2019 at 10:12:19AM +0200, Jan Beulich wrote: > >> On 23.10.2019 22:33, Pry Mar wrote: > >>> Hello xen-devel, > >>> > >>> https://paste.debian.net/plain/1109374 > >>> > >>> shows my traces from a healthy CentOS 8, xen-4.12.1 dom0 when trying > >>> to launch a pv install of the newly released ub1910. The source is a > >>> block-attached ISO and the kernel/ramdisk was copied off locally. > >> > >> Would you please increase verbosity (xl -vvv create ...) such that we > >> can see what exactly the decompression code doesn't like about this > >> kernel image? > > > > I stumbled across the same issue, below is the xl - create output. > > > > Parsing config from ubuntu.cfg > > libxl: debug: libxl_create.c:1693:do_domain_create: Domain 0:ao > > 0x55a598e77190: create: how=(nil) callback=(nil) poller=0x55a598e74040 > > libxl: debug: libxl_device.c:397:libxl__device_disk_set_backend: Disk > > vdev=xvda spec.backend=unknown > > libxl: debug: libxl_device.c:358:disk_try_backend: Disk vdev=xvda, backend > > phy unsuitable due to format qcow2 > > libxl: debug: libxl_device.c:431:libxl__device_disk_set_backend: Disk > > vdev=xvda, using backend qdisk > > libxl: debug: libxl_create.c:1018:initiate_domain_create: Domain 11:running > > bootloader > > libxl: debug: libxl_bootloader.c:334:libxl__bootloader_run: Domain 11:no > > bootloader configured, using user supplied kernel > > libxl: debug: libxl_event.c:689:libxl__ev_xswatch_deregister: watch > > w=0x55a598e827a8: deregister unregistered > > domainbuilder: detail: xc_dom_allocate: cmdline="", features="" > > libxl: debug: libxl_dom.c:799:libxl__build_pv: pv kernel mapped 0 path > > /tank/xenscratch/ubuntu/vmlinuz-5.3.0-23-generic > > domainbuilder: detail: xc_dom_kernel_file: > > filename="/tank/xenscratch/ubuntu/vmlinuz-5.3.0-23-generic" > > domainbuilder: detail: xc_dom_malloc_filemap: 11132 kB > > domainbuilder: detail: xc_dom_boot_xen_init: ver 4.12, caps xen-3.0-x86_64 > > xen-3.0-x86_32p > > domainbuilder: detail: xc_dom_parse_image: called > > domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader > > ... > > domainbuilder: detail: loader probe failed > > domainbuilder: detail: xc_dom_find_loader: trying HVM-generic loader ... > > domainbuilder: detail: loader probe failed > > domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... > > domainbuilder: detail: LZ4 decompression error: decoding failed > > This suggests that the decoding logic didn't like the input. Since as > per the other mail manual decompression works, this will likely need > debugging by someone able to repro. > > Jan I'm able to repro, and I isolated the code from xc_dom_bzimageloader.c, xc_dom_decompress_lz4.c and /xen/common/lz4/decompress.c such that I can test more easily (I'm using code from 4.12.1). I'm testing with vmlinuz-5.3.0-23-generic installed in ubuntu-19.10. What I see is that the code fails at the first frame at decompress.c:282 (if (unlikely((unsigned long)cpy < (unsigned long)op))). because cpy == (op - 1). decompress.c:265 (cpy = op + length - (STEPSIZE-4);) gets executed twice and prints: length=4 length=3 STEPSIZE is 8 (x86_64). So this has to fail. The STEPSIZE gave me the idea to rebuild the code as 32-bit and decompression works correctly. Any suggestions how to proceed? ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] bug: unable to LZ4 decompress ub1910 installer kernel when launching domU
On Thu, Oct 24, 2019 at 10:12:19AM +0200, Jan Beulich wrote: > On 23.10.2019 22:33, Pry Mar wrote: > > Hello xen-devel, > > > > https://paste.debian.net/plain/1109374 > > > > shows my traces from a healthy CentOS 8, xen-4.12.1 dom0 when trying > > to launch a pv install of the newly released ub1910. The source is a > > block-attached ISO and the kernel/ramdisk was copied off locally. > > Would you please increase verbosity (xl -vvv create ...) such that we > can see what exactly the decompression code doesn't like about this > kernel image? > > Jan > Hi Jan, I stumbled across the same issue, below is the xl - create output. Parsing config from ubuntu.cfg libxl: debug: libxl_create.c:1693:do_domain_create: Domain 0:ao 0x55a598e77190: create: how=(nil) callback=(nil) poller=0x55a598e74040 libxl: debug: libxl_device.c:397:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=unknown libxl: debug: libxl_device.c:358:disk_try_backend: Disk vdev=xvda, backend phy unsuitable due to format qcow2 libxl: debug: libxl_device.c:431:libxl__device_disk_set_backend: Disk vdev=xvda, using backend qdisk libxl: debug: libxl_create.c:1018:initiate_domain_create: Domain 11:running bootloader libxl: debug: libxl_bootloader.c:334:libxl__bootloader_run: Domain 11:no bootloader configured, using user supplied kernel libxl: debug: libxl_event.c:689:libxl__ev_xswatch_deregister: watch w=0x55a598e827a8: deregister unregistered domainbuilder: detail: xc_dom_allocate: cmdline="", features="" libxl: debug: libxl_dom.c:799:libxl__build_pv: pv kernel mapped 0 path /tank/xenscratch/ubuntu/vmlinuz-5.3.0-23-generic domainbuilder: detail: xc_dom_kernel_file: filename="/tank/xenscratch/ubuntu/vmlinuz-5.3.0-23-generic" domainbuilder: detail: xc_dom_malloc_filemap: 11132 kB domainbuilder: detail: xc_dom_boot_xen_init: ver 4.12, caps xen-3.0-x86_64 xen-3.0-x86_32p domainbuilder: detail: xc_dom_parse_image: called domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ... domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying HVM-generic loader ... domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... domainbuilder: detail: LZ4 decompression error: decoding failed xc: error: panic: xc_dom_bzimageloader.c:766: xc_dom_probe_bzimage_kernel unable to LZ4 decompress kernel : Invalid kernel domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... domainbuilder: detail: loader probe failed xc: error: panic: xc_dom_core.c:691: xc_dom_find_loader: no loader found: Invalid kernel libxl: error: libxl_dom.c:737:libxl__build_dom: xc_dom_parse_image failed domainbuilder: detail: xc_dom_release: called libxl: error: libxl_create.c:1286:domcreate_rebuild_done: Domain 11:cannot (re-)build domain: -3 libxl: debug: libxl_domain.c:1194:devices_destroy_cb: Domain 11:Forked pid 71592 for destroy of domain libxl: debug: libxl_create.c:1730:do_domain_create: Domain 0:ao 0x55a598e77190: inprogress: poller=0x55a598e74040, flags=i libxl: debug: libxl_event.c:1873:libxl__ao_complete: ao 0x55a598e77190: complete, rc=-3 libxl: debug: libxl_event.c:1842:libxl__ao__destroy: ao 0x55a598e77190: destroy libxl: debug: libxl_domain.c:902:libxl_domain_destroy: Domain 11:ao 0x55a598e76db0: create: how=(nil) callback=(nil) poller=0x55a598e74040 libxl: error: libxl_domain.c:1038:libxl__destroy_domid: Domain 11:Non-existant domain libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain 11:Unable to destroy guest libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 11:Destruction of domain failed libxl: debug: libxl_event.c:1873:libxl__ao_complete: ao 0x55a598e76db0: complete, rc=-21 libxl: debug: libxl_domain.c:911:libxl_domain_destroy: Domain 11:ao 0x55a598e76db0: inprogress: poller=0x55a598e74040, flags=ic libxl: debug: libxl_event.c:1842:libxl__ao__destroy: ao 0x55a598e76db0: destroy xencall:buffer: debug: total allocations:75 total releases:75 xencall:buffer: debug: current allocations:0 maximum allocations:3 xencall:buffer: debug: cache current size:3 xencall:buffer: debug: cache hits:61 misses:3 toobig:11 xencall:buffer: debug: total allocations:0 total releases:0 xencall:buffer: debug: current allocations:0 maximum allocations:0 xencall:buffer: debug: cache current size:0 xencall:buffer: debug: cache hits:0 misses:0 toobig:0 ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
