Re: Booting uncompressed kernel images

[Ben Hutchings]

On Fri, 2016-02-05 at 15:52 +0100, Andreas Ziegler wrote:
> Hi,
> 
> please allow me to dive in here, i do have a very similar problem.
> 
> one of my VPS is running on a providers host system, that seems to use
> an old Xen version, which can't boot the Jessie-Kernel.
> at the moment im using the wheezy-Kernel on jessie, but that's not the
> best solution...
> 
> do i have to self-build the kernel or is there another way to get an
> uncompressed oder gz-compressed kernel?
> (building it from upstream is easy, is there a guide to build it
> directly from debians sources?)
[...]

Yes, see
.

Ben.

-- 
Ben Hutchings
It is a miracle that curiosity survives formal education. - Albert Einstein

signature.asc
Description: This is a digitally signed message part

Re: Booting uncompressed kernel images

[Andreas Ziegler]

Hi,

please allow me to dive in here, i do have a very similar problem.

one of my VPS is running on a providers host system, that seems to use
an old Xen version, which can't boot the Jessie-Kernel.
at the moment im using the wheezy-Kernel on jessie, but that's not the
best solution...

do i have to self-build the kernel or is there another way to get an
uncompressed oder gz-compressed kernel?
(building it from upstream is easy, is there a guide to build it
directly from debians sources?)

i have no influence on the host system whatsoever and even need to
contact their support, if the boot fails - like it does with jessies kernel.

Regards

Andreas


Am 05.02.2016 um 12:05 schrieb Ian Campbell:
> On Thu, 2016-02-04 at 15:40 -0200, Tiago Ilieve wrote:
>> Sorry for the delay in my response. In the past couple days I was
>> confirming with Oracle if my findings (using virt-what, as you
>> suggested) where right and, indeed, they are supporting Xen HVM right
>> now.
> 
> Great!
> 
>>
>> So, there's no need for an uncompressed/gzipped kernel anymore and the
>> default one boots just fine. Although I'm still curious regarding the
>> possibility of booting an uncompressed kernel on
>> native/full-virtualization, I guess this does not makes sense.
> 
> WRT virtualisation, not in general no.
> 
> If possible you really want to do the decompression in guest context, to
> avoid issues with potentially malicious compressed binaries.
> 
> For native there generally isn't much point, but for x86 at least there is
> also a bunch of necessary setup and gathering of information (e.g. from the
> BIOS in 16 bit mode) which is done in the same preamble as where the
> decompressor runs (more or less) and which you would need to replicate
> before booting the uncompressed image's entry point -- it's really not
> worth the effort in general.
> 
>> I'm really thankful for you support and inclination to help us on the
>> matter.
> 
> No problem!
> 
> Ian.
> 

Re: Booting uncompressed kernel images

[Ian Campbell]

On Thu, 2016-02-04 at 15:40 -0200, Tiago Ilieve wrote:
> Sorry for the delay in my response. In the past couple days I was
> confirming with Oracle if my findings (using virt-what, as you
> suggested) where right and, indeed, they are supporting Xen HVM right
> now.

Great!

> 
> So, there's no need for an uncompressed/gzipped kernel anymore and the
> default one boots just fine. Although I'm still curious regarding the
> possibility of booting an uncompressed kernel on
> native/full-virtualization, I guess this does not makes sense.

WRT virtualisation, not in general no.

If possible you really want to do the decompression in guest context, to
avoid issues with potentially malicious compressed binaries.

For native there generally isn't much point, but for x86 at least there is
also a bunch of necessary setup and gathering of information (e.g. from the
BIOS in 16 bit mode) which is done in the same preamble as where the
decompressor runs (more or less) and which you would need to replicate
before booting the uncompressed image's entry point -- it's really not
worth the effort in general.

> I'm really thankful for you support and inclination to help us on the
> matter.

No problem!

Ian.

Re: Booting uncompressed kernel images

[Tiago Ilieve]

Ian,

On 2 February 2016 at 15:32, Ian Campbell  wrote:
> Yes, although I think/suspect that my 47B4 is created by binwalk
> decompressing 47B4.xz as a convenience.
>
> (...)
>
> This looks like a file which I would expect to be bootable as a Xen PV
> guest. Using "readelf -n" should show lots of:
>
> Displaying notes found at file offset 0x00716774 with length 0x01d8:
>   Owner Data size   Description
>   Xen  0x0006   Unknown note type: (0x0006)
>
> Which would imply this.
>
> (...)
>
> It depends ;-).
>
> Grub2 can support many different platforms, including "native PC" and "Xen
> PV". http://wiki.xen.org/wiki/PvGrub2#Background:_Introduction_to_Xen_PV_Bo
> otloaders has some background which I hope will be useful here (the second
> part on actually using grub2 is not really relevant since you are at the
> mercy of Oracle Cloud).
>
> "Xen HVM" == "native PC" from a booting point of view, so in a Xen HVM
> guest you would use the native grub, in a Xen PV guest it's possible that
> you are using the "Xen PV" version of Grub2.
>
> A native version of Grub will not be able to boot the raw ELF file
> extracting from the vmlinuz, but it should be able to boot the vmlinuz
> itself just fine.
>
> Virtual box == "native PC" too, I think it unlikely it would be able to
> boot a raw ELF file.
>
> (...)
>
> Do you need the custom kernel on your local install?
>
> Having grub2 installed in a PV guest does not necessarily mean it is being
> used, since the first stage bootloader is provided by the host. The package
> supporting Xen PV is "grub-xen"
>
> If you are booting PV grub2 then perhaps you just need to add "insmod xzio"
> to your grub.cfg?
>
> (...)
>
> I think you should ignore VirtualBox for the purposes of diagnosing what is
> going on here, behaviour on VB tells us next to nothing about behaviour on
> Xen.
>
> (...)
>
> You really need to figure out for sure if you are booting in an HVM or PV
> guest, it makes a very large difference to what kernel features you
> want/need. From there it should become pretty clear what bootloader is in
> use.
>
> If you can boot the guest in some way then virt-what ought to tell you for 
> pretty sure what you are running in.

Sorry for the delay in my response. In the past couple days I was
confirming with Oracle if my findings (using virt-what, as you
suggested) where right and, indeed, they are supporting Xen HVM right
now.

So, there's no need for an uncompressed/gzipped kernel anymore and the
default one boots just fine. Although I'm still curious regarding the
possibility of booting an uncompressed kernel on
native/full-virtualization, I guess this does not makes sense.

I'm really thankful for you support and inclination to help us on the matter.

Very best regards,
Tiago.

-- 
Tiago "Myhro" Ilieve
Blog: https://blog.myhro.info/
GitHub: https://github.com/myhro
LinkedIn: https://br.linkedin.com/in/myhro
Montes Claros - MG, Brasil

Re: Booting uncompressed kernel images

[Tiago Ilieve]

Hi Ian,

On 2 February 2016 at 09:24, Ian Campbell  wrote:
> Did you see my other replies on debian-kernel yesterday? There are some
> questions there which it would be useful to know the answers to.

It turned out that I though that I was subscribed to
"debian-kernel@l.d.o", but I wasn't. This is solved now.

> What does file(1) say about them? For me I see:
>
> _vmlinuz-3.16.0-4-amd64.extracted/2F6EF8.xz: XZ compressed data
> _vmlinuz-3.16.0-4-amd64.extracted/47B4:  ELF 64-bit LSB executable, 
> x86-64, version 1 (SYSV), statically linked, 
> BuildID[sha1]=2883400c6927fe339cdd2c321d3d154c472ef418, stripped
> _vmlinuz-3.16.0-4-amd64.extracted/47B4.xz:   XZ compressed data

Are you able to extract three files from it? Here's what I got:

$ file _vmlinuz-3.16.0-4-amd64.extracted/*
_vmlinuz-3.16.0-4-amd64.extracted/2F6EF8.tar: XZ compressed data
_vmlinuz-3.16.0-4-amd64.extracted/47B4.tar:   XZ compressed data

Only two that I can't extract again.

> Do any of them match what you get out of extract-linux?

$ ./extract-vmlinux /boot/vmlinuz-3.16.0-4-amd64 > vmlinux
$ file vmlinux
vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
statically linked,
BuildID[sha1]=2883400c6927fe339cdd2c321d3d154c472ef418, stripped

The SHA1 is the same.

> _vmlinuz-3.16.0-4-amd64.extracted/47B4 looks to me to be an ELF file which
> I would expect to be bootable as a Xen PV guest, it has the required ELF
> notes etc.

>From your previous e-mails, looks like Grub2 can't boot it, opposite
to a Xen PV guest, right?

On 1 February 2016 at 08:06, Ian Campbell  wrote:

> Is this booting via pvgrub[1], if not then do you know how? What does the
> grub.cfg stanza look like?

This is hard to answer properly. The image has "grub2" (the last
2.02~beta2-22+deb8u1 available on Jessie) installed and I can boot it
on a local Xen installation as well on Oracle Cloud (using the custom
kernel compressed with gzip). What I tried is the same process
(extract-linux, generate initrd, update-grub) both on VirtualBox and
Xen on Oracle Cloud. In VirtualBox, I got the error pasted on my first
message. In Oracle Cloud I have no access to the boot logs to confirm
if it happens to be exactly the same.

-- 
Tiago "Myhro" Ilieve
Blog: https://blog.myhro.info/
GitHub: https://github.com/myhro
LinkedIn: https://br.linkedin.com/in/myhro
Montes Claros - MG, Brasil

Re: Booting uncompressed kernel images

[Ian Campbell]

On Tue, 2016-02-02 at 15:18 -0200, Tiago Ilieve wrote:
> Hi Ian,
> 
> On 2 February 2016 at 09:24, Ian Campbell  wrote:
> > Did you see my other replies on debian-kernel yesterday? There are some
> > questions there which it would be useful to know the answers to.
> 
> It turned out that I though that I was subscribed to
> "debian-kernel@l.d.o", but I wasn't. This is solved now.
> 
> > What does file(1) say about them? For me I see:
> > 
> > _vmlinuz-3.16.0-4-amd64.extracted/2F6EF8.xz: XZ compressed data
> > _vmlinuz-3.16.0-4-amd64.extracted/47B4:  ELF 64-bit LSB executable,
> > x86-64, version 1 (SYSV), statically linked,
> > BuildID[sha1]=2883400c6927fe339cdd2c321d3d154c472ef418, stripped
> > _vmlinuz-3.16.0-4-amd64.extracted/47B4.xz:   XZ compressed data
> 
> Are you able to extract three files from it?

Yes, although I think/suspect that my 47B4 is created by binwalk
decompressing 47B4.xz as a convenience.

>  Here's what I got:
> 
> $ file _vmlinuz-3.16.0-4-amd64.extracted/*
> _vmlinuz-3.16.0-4-amd64.extracted/2F6EF8.tar: XZ compressed data
> _vmlinuz-3.16.0-4-amd64.extracted/47B4.tar:   XZ compressed data
> 
> Only two that I can't extract again.
> 
> > Do any of them match what you get out of extract-linux?
> 
> $ ./extract-vmlinux /boot/vmlinuz-3.16.0-4-amd64 > vmlinux
> $ file vmlinux
> vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
> statically linked,
> BuildID[sha1]=2883400c6927fe339cdd2c321d3d154c472ef418, stripped

This looks like a file which I would expect to be bootable as a Xen PV
guest. Using "readelf -n" should show lots of:

Displaying notes found at file offset 0x00716774 with length 0x01d8:
  Owner Data size   Description
  Xen  0x0006   Unknown note type: (0x0006)

Which would imply this.

> The SHA1 is the same.
> 
> > _vmlinuz-3.16.0-4-amd64.extracted/47B4 looks to me to be an ELF file
> > which
> > I would expect to be bootable as a Xen PV guest, it has the required
> > ELF
> > notes etc.
> 
> From your previous e-mails, looks like Grub2 can't boot it, opposite
> to a Xen PV guest, right?

It depends ;-).

Grub2 can support many different platforms, including "native PC" and "Xen
PV". http://wiki.xen.org/wiki/PvGrub2#Background:_Introduction_to_Xen_PV_Bo
otloaders has some background which I hope will be useful here (the second
part on actually using grub2 is not really relevant since you are at the
mercy of Oracle Cloud).

"Xen HVM" == "native PC" from a booting point of view, so in a Xen HVM
guest you would use the native grub, in a Xen PV guest it's possible that
you are using the "Xen PV" version of Grub2.

A native version of Grub will not be able to boot the raw ELF file
extracting from the vmlinuz, but it should be able to boot the vmlinuz
itself just fine.

Virtual box == "native PC" too, I think it unlikely it would be able to
boot a raw ELF file.

> On 1 February 2016 at 08:06, Ian Campbell  wrote:
> 
> > Is this booting via pvgrub[1], if not then do you know how? What does
> > the
> > grub.cfg stanza look like?
> 
> This is hard to answer properly. The image has "grub2" (the last
> 2.02~beta2-22+deb8u1 available on Jessie) installed and I can boot it
> on a local Xen installation as well on Oracle Cloud (using the custom
> kernel compressed with gzip).

Do you need the custom kernel on your local install?

Having grub2 installed in a PV guest does not necessarily mean it is being
used, since the first stage bootloader is provided by the host. The package
supporting Xen PV is "grub-xen"

If you are booting PV grub2 then perhaps you just need to add "insmod xzio"
to your grub.cfg?

>  What I tried is the same process
> (extract-linux, generate initrd, update-grub) both on VirtualBox and
> Xen on Oracle Cloud. In VirtualBox, I got the error pasted on my first
> message.

I think you should ignore VirtualBox for the purposes of diagnosing what is
going on here, behaviour on VB tells us next to nothing about behaviour on
Xen.

>  In Oracle Cloud I have no access to the boot logs to confirm
> if it happens to be exactly the same.

You really need to figure out for sure if you are booting in an HVM or PV
guest, it makes a very large difference to what kernel features you
want/need. From there it should become pretty clear what bootloader is in
use.

If you can boot the guest in some way then virt-what ought to tell you for 
pretty sure what you are running in.

Ian.

Re: Booting uncompressed kernel images

[Ian Campbell]

On Mon, 2016-02-01 at 23:31 -0200, Tiago Ilieve wrote:
> > PS I have also found binwalk [2] useful when examining contents of
> > compressed kernel
> > apt-get install binwalk
> 
> Thanks for the tip - although I got a little bit surprised with so
> many dependencies in what should be a simple command-line utility.
> 
> Here's what I got from "binwalk /boot/vmlinuz-3.16.0-4-amd64":
> 
> DECIMAL   HEXADECIMAL DESCRIPTION
> ---
> -
> 0 0x0 Microsoft portable executable
> 18356 0x47B4  xz compressed data
> 3108600   0x2F6EF8xz compressed data
> 
> Not sure about what bytes "0-18355" means. Maybe a false-positive?
> 
> If I run it with "-e", it get two files ("47B4.tar" and "2F6EF8.tar")
> that can't be decompressed with "tar"/"unxz".

What does file(1) say about them? For me I see:

_vmlinuz-3.16.0-4-amd64.extracted/2F6EF8.xz: XZ compressed data
_vmlinuz-3.16.0-4-amd64.extracted/47B4:  ELF 64-bit LSB executable, x86-64, 
version 1 (SYSV), statically linked, 
BuildID[sha1]=2883400c6927fe339cdd2c321d3d154c472ef418, stripped
_vmlinuz-3.16.0-4-amd64.extracted/47B4.xz:   XZ compressed data

Do any of them match what you get out of extract-linux?

_vmlinuz-3.16.0-4-amd64.extracted/47B4 looks to me to be an ELF file which
I would expect to be bootable as a Xen PV guest, it has the required ELF
notes etc.

Did you see my other replies on debian-kernel yesterday? There are some
questions there which it would be useful to know the answers to.

Ian.

Re: Booting uncompressed kernel images

[Tiago Ilieve]

Hi Jose,

I'm dropping off the cc of "debian-cloud at l.d.o", because this is
really a kernel matter, so I see no reason for cross-posting. I can
post the outcome in there when we reach a solution.

On 1 February 2016 at 08:37, Jose R R  wrote:
> Does the kernel boot after after you rebuild it with GZIP=y?

Yes it does. The only problem with this approach is the effort needed
to rebuild the package and keep it updated.

> The script is not without its issues, apparently, but you can modify
> xxx[1] in script if that is causing your hang.

>From what I understood in the reference, one should do this only if
the "mktemp" call fails. This doesn't looks to be related to the
problem I encountered.

> In Google Cloud Engine (GCE) I have not experienced those issues as I
> have created several images in Jessie and Stretch -- manually at the
> moment. They are Debian vanilla except for Reiser4 -patched kernel and
> root (/) Reiser4 -formatted.

You are rebuilding/recompiling the kernel packaging like I did? Or are
you talking about decompressing the kernel image? Because, as I said,
I had no problem with the first one. What I'm reporting here is solely
related to an uncompressed (or compressed with other method than XZ)
kernel.

> PS I have also found binwalk [2] useful when examining contents of
> compressed kernel
> apt-get install binwalk

Thanks for the tip - although I got a little bit surprised with so
many dependencies in what should be a simple command-line utility.

Here's what I got from "binwalk /boot/vmlinuz-3.16.0-4-amd64":

DECIMAL   HEXADECIMAL DESCRIPTION

0 0x0 Microsoft portable executable
18356 0x47B4  xz compressed data
3108600   0x2F6EF8xz compressed data

Not sure about what bytes "0-18355" means. Maybe a false-positive?

If I run it with "-e", it get two files ("47B4.tar" and "2F6EF8.tar")
that can't be decompressed with "tar"/"unxz".

-

Ben,

On 1 February 2016 at 06:03, Tiago Ilieve  wrote:

> Yesterday, Ben Hutchings itself (...)

Sorry. It should be "himself". If it helps you to accept my apologies,
I have written that message at 6 AM after tackling this issue
overnight. :-(

Regards,
Tiago.

-- 
Tiago "Myhro" Ilieve
Blog: https://blog.myhro.info/
GitHub: https://github.com/myhro
LinkedIn: https://br.linkedin.com/in/myhro
Montes Claros - MG, Brasil

Re: Booting uncompressed kernel images

[Jose R R]

On Mon, Feb 1, 2016 at 12:03 AM, Tiago Ilieve  wrote:
> Hi,
>
> I have a scenario[1] where the default Linux kernel compressed with XZ
> (from Jessie and up) cannot be booted. The first thing that I've tried
> was to uncompress it using "extract-linux"[2] and it didn't worked by
> the time, so I decided to rebuild the entire "linux-image-*" package
> changing "CONFIG_KERNEL_XZ=y" to "CONFIG_KERNEL_GZIP=y". This, of
> course, implies that it would be needed to recompile the kernel every
> time a new version of the package is released, which is an overkill
> for a such simple requirement.

Does the kernel boot after after you rebuild it with GZIP=y?

>
> Yesterday, Ben Hutchings itself suggested[3] me to write a hook that
> decompresses the kernel at package installation time, something which
> I find a great idea. The problem is that, again, I couldn't boot a
> machine (tried on VirtualBox and Xen) after uncompressing its
> "/boot/vmlinuz-*" using "extract-linux".

The script is not without its issues, apparently, but you can modify
xxx[1] in script if that is causing your hang.

> I can generate an initrd file
> from this uncompressed image, "update-grub" detects it fine, but if I
> reboot it the following error appears:
>
> Loading Linux 3.16.0-4-amd64 ...
> error: invalid magic number.
> Loading initial ramdisk ...
> error: you need to load the kernel first.
>
> The same error happens if this uncompressed image is compressed again
> with "gzip", for instance.
>
> Given this situation, I have two questions:
>
> - If a kernel is configured to be compressed at build time, it can't
> be booted if uncompressed later? I have some doubts about this,
> because there is more than one report of successfully booting
> uncompressed kernels[4][5].
> - Is "extract-linux" stripping some essential information (the script
> looks for an offset to start the decompression process) from the
> kernel image that is needed to boot it later? If so, is there a way to
> recover and insert it on the uncompressed image?

In Google Cloud Engine (GCE) I have not experienced those issues as I
have created several images in Jessie and Stretch -- manually at the
moment. They are Debian vanilla except for Reiser4 -patched kernel and
root (/) Reiser4 -formatted.


Best Professional Regards.

PS I have also found binwalk [2] useful when examining contents of
compressed kernel
apt-get install binwalk


Was not aware that Oracle had such an outdated cloud infrastructure ;-)

[1] 
http://stackoverflow.com/questions/12002315/extract-vmlinux-from-vmlinuz-or-bzimage
[2] 
http://unix.stackexchange.com/questions/163346/why-is-it-that-my-initrd-only-has-one-directory-namely-kernel

-- 
Jose R R
http://metztli.it
-
Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
-
from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!
-

Re: Booting uncompressed kernel images

[Ian Campbell]

On Mon, 2016-02-01 at 10:06 +, Ian Campbell wrote:
> On Mon, 2016-02-01 at 06:03 -0200, Tiago Ilieve wrote:
> > Hi,
> > 
> > I have a scenario[1] where the default Linux kernel compressed with XZ
> > (from Jessie and up) cannot be booted. The first thing that I've tried
> > was to uncompress it using "extract-linux"[2] and it didn't worked by
> > the time, so I decided to rebuild the entire "linux-image-*" package
> > changing "CONFIG_KERNEL_XZ=y" to "CONFIG_KERNEL_GZIP=y". This, of
> > course, implies that it would be needed to recompile the kernel every
> > time a new version of the package is released, which is an overkill
> > for a such simple requirement.
> > 
> > Yesterday, Ben Hutchings itself suggested[3] me to write a hook that
> > decompresses the kernel at package installation time, something which
> > I find a great idea. The problem is that, again, I couldn't boot a
> > machine (tried on VirtualBox and Xen) after uncompressing its
> > "/boot/vmlinuz-*" using "extract-linux".
> 
> What does file(1) say about the resulting file? It should be a plain ELF
> file of some sort.
> 
> I wouldn't expect such a kernel to be bootable by very much other than
> Xen
> PV. It won't work as native for sure, nor as Xen HVM.
> 
> >  I can generate an initrd file
> > from this uncompressed image, "update-grub" detects it fine, but if I
> > reboot it the following error appears:
> > 
> > Loading Linux 3.16.0-4-amd64 ...
> > error: invalid magic number.
> > Loading initial ramdisk ...
> > error: you need to load the kernel first.
> 
> Is this booting via pvgrub[1], if not then do you know how? What does the
> grub.cfg stanza look like?

Hrm, that "invalid magic number" message is from upstream grub2, native
code paths, AFAICT.

Were you trying to boot this kernel natively? Or are the guests on Oracle
HVM ones (in which case the compression shouldn't matter, since the kernel
native boot path will take care of it).

> 
> > - Is "extract-linux" stripping some essential information (the script
> > looks for an offset to start the decompression process) from the
> > kernel image that is needed to boot it later? If so, is there a way to
> > recover and insert it on the uncompressed image?
> 
> It should just be the raw ELF file payload form the bzImage. Xen requires
> some particular ELF notes to be present, but I very much doubt extract-
> linux would be stripping those (since that would involve diving into the
> contents of the payload)
> 
> Ian.
> 
> [1] http://wiki.xen.org/wiki/PvGrub
> 
> 
> 
> > 
> > Regards,
> > Tiago.
> > 
> > [1]: https://lists.debian.org/debian-cloud/2016/01/msg00052.html
> > [2]: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/pl
> > ai
> > n/scripts/extract-vmlinux
> > [3]: https://lists.debian.org/debian-cloud/2016/01/msg00060.html
> > [4]: http://www.he1ix.org/2015/01/creating-a-xen-domu-on-debian-squeeze
> > -6
> > -0-6/
> > [5]: http://noone.org/blog/English/Computer/Debian/Running%20a%20Sid%20
> > Do
> > mU%20on%20a%20Squeeze%20Dom0.html
> > 
> 
> 

Re: Booting uncompressed kernel images

[Ian Campbell]

On Mon, 2016-02-01 at 06:03 -0200, Tiago Ilieve wrote:
> Hi,
> 
> I have a scenario[1] where the default Linux kernel compressed with XZ
> (from Jessie and up) cannot be booted. The first thing that I've tried
> was to uncompress it using "extract-linux"[2] and it didn't worked by
> the time, so I decided to rebuild the entire "linux-image-*" package
> changing "CONFIG_KERNEL_XZ=y" to "CONFIG_KERNEL_GZIP=y". This, of
> course, implies that it would be needed to recompile the kernel every
> time a new version of the package is released, which is an overkill
> for a such simple requirement.
> 
> Yesterday, Ben Hutchings itself suggested[3] me to write a hook that
> decompresses the kernel at package installation time, something which
> I find a great idea. The problem is that, again, I couldn't boot a
> machine (tried on VirtualBox and Xen) after uncompressing its
> "/boot/vmlinuz-*" using "extract-linux".

What does file(1) say about the resulting file? It should be a plain ELF
file of some sort.

I wouldn't expect such a kernel to be bootable by very much other than Xen
PV. It won't work as native for sure, nor as Xen HVM.

>  I can generate an initrd file
> from this uncompressed image, "update-grub" detects it fine, but if I
> reboot it the following error appears:
> 
> Loading Linux 3.16.0-4-amd64 ...
> error: invalid magic number.
> Loading initial ramdisk ...
> error: you need to load the kernel first.

Is this booting via pvgrub[1], if not then do you know how? What does the
grub.cfg stanza look like?

> - Is "extract-linux" stripping some essential information (the script
> looks for an offset to start the decompression process) from the
> kernel image that is needed to boot it later? If so, is there a way to
> recover and insert it on the uncompressed image?

It should just be the raw ELF file payload form the bzImage. Xen requires
some particular ELF notes to be present, but I very much doubt extract-
linux would be stripping those (since that would involve diving into the
contents of the payload)

Ian.

[1] http://wiki.xen.org/wiki/PvGrub



> 
> Regards,
> Tiago.
> 
> [1]: https://lists.debian.org/debian-cloud/2016/01/msg00052.html
> [2]: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plai
> n/scripts/extract-vmlinux
> [3]: https://lists.debian.org/debian-cloud/2016/01/msg00060.html
> [4]: http://www.he1ix.org/2015/01/creating-a-xen-domu-on-debian-squeeze-6
> -0-6/
> [5]: http://noone.org/blog/English/Computer/Debian/Running%20a%20Sid%20Do
> mU%20on%20a%20Squeeze%20Dom0.html
> 

Booting uncompressed kernel images

[Tiago Ilieve]

Hi,

I have a scenario[1] where the default Linux kernel compressed with XZ
(from Jessie and up) cannot be booted. The first thing that I've tried
was to uncompress it using "extract-linux"[2] and it didn't worked by
the time, so I decided to rebuild the entire "linux-image-*" package
changing "CONFIG_KERNEL_XZ=y" to "CONFIG_KERNEL_GZIP=y". This, of
course, implies that it would be needed to recompile the kernel every
time a new version of the package is released, which is an overkill
for a such simple requirement.

Yesterday, Ben Hutchings itself suggested[3] me to write a hook that
decompresses the kernel at package installation time, something which
I find a great idea. The problem is that, again, I couldn't boot a
machine (tried on VirtualBox and Xen) after uncompressing its
"/boot/vmlinuz-*" using "extract-linux". I can generate an initrd file
from this uncompressed image, "update-grub" detects it fine, but if I
reboot it the following error appears:

Loading Linux 3.16.0-4-amd64 ...
error: invalid magic number.
Loading initial ramdisk ...
error: you need to load the kernel first.

The same error happens if this uncompressed image is compressed again
with "gzip", for instance.

Given this situation, I have two questions:

- If a kernel is configured to be compressed at build time, it can't
be booted if uncompressed later? I have some doubts about this,
because there is more than one report of successfully booting
uncompressed kernels[4][5].
- Is "extract-linux" stripping some essential information (the script
looks for an offset to start the decompression process) from the
kernel image that is needed to boot it later? If so, is there a way to
recover and insert it on the uncompressed image?

Regards,
Tiago.

[1]: https://lists.debian.org/debian-cloud/2016/01/msg00052.html
[2]: 
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/scripts/extract-vmlinux
[3]: https://lists.debian.org/debian-cloud/2016/01/msg00060.html
[4]: http://www.he1ix.org/2015/01/creating-a-xen-domu-on-debian-squeeze-6-0-6/
[5]: 
http://noone.org/blog/English/Computer/Debian/Running%20a%20Sid%20DomU%20on%20a%20Squeeze%20Dom0.html

-- 
Tiago "Myhro" Ilieve
Blog: https://blog.myhro.info/
GitHub: https://github.com/myhro
LinkedIn: https://br.linkedin.com/in/myhro
Montes Claros - MG, Brasil