Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Kunihiko Hayashi]

On 2017/06/03 4:18, nicolas.deche...@linaro.org (Nicolas Dechesne) wrote:

Le ven. 2 juin 2017 ? 20:32, Kunihiko Hayashi <
hayashi.kunih...@socionext.com> a ?crit :



On 2017/06/03 1:24, nicolas.deche...@linaro.org (Nicolas Dechesne) wrote:

On Fri, Jun 2, 2017 at 6:16 PM, Kunihiko Hayashi
 wrote:

I already suggested the patch to solve this issue.

https://patchwork.openembedded.org/patch/138596/

Although I've not received any comments on this yet, could you try to

apply

it?


thanks for sending this out. Well, unfortunately, I think your patch
is doing something wrong.

As discussed earlier on that thread, we should be using kernel
Makefile to generate the target as much as possible. The linux build
system can generate vmlinux, Image and Image.gz, but not vmlinux.gz.
So we should compress only vmlinux 'outside' of the linux build
system, and for Image.gz we shouldn't do that in OE. In other words,
Image.gz should be included in typeformake. Your patch does not do
that, and that is something which is taken care by mine.


Surely we should use Image.gz generated by kernel build system and my patch
doesn't do that. I also think that OE should compress only images that the
kernel build system doesn't generate.
Sorry my patch won't help.



No problem. Can you please review and tes this one then, and provide
feedback?


Yes. I'll try your suggestion and put feedback after the patch mail.

Best Regards,
Kunihiko Hayashi
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Nicolas Dechesne]

Le ven. 2 juin 2017 à 20:32, Kunihiko Hayashi <
hayashi.kunih...@socionext.com> a écrit :

>
> On 2017/06/03 1:24, nicolas.deche...@linaro.org (Nicolas Dechesne) wrote:
> > On Fri, Jun 2, 2017 at 6:16 PM, Kunihiko Hayashi
> >  wrote:
> >> I already suggested the patch to solve this issue.
> >>
> >> https://patchwork.openembedded.org/patch/138596/
> >>
> >> Although I've not received any comments on this yet, could you try to
> apply
> >> it?
> >
> > thanks for sending this out. Well, unfortunately, I think your patch
> > is doing something wrong.
> >
> > As discussed earlier on that thread, we should be using kernel
> > Makefile to generate the target as much as possible. The linux build
> > system can generate vmlinux, Image and Image.gz, but not vmlinux.gz.
> > So we should compress only vmlinux 'outside' of the linux build
> > system, and for Image.gz we shouldn't do that in OE. In other words,
> > Image.gz should be included in typeformake. Your patch does not do
> > that, and that is something which is taken care by mine.
>
> Surely we should use Image.gz generated by kernel build system and my patch
> doesn't do that. I also think that OE should compress only images that the
> kernel build system doesn't generate.
> Sorry my patch won't help.


No problem. Can you please review and tes this one then, and provide
feedback?

Thanks!


>
> Best Regards,
> Kunihiko Hayashi
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Kunihiko Hayashi]

On 2017/06/03 1:24, nicolas.deche...@linaro.org (Nicolas Dechesne) wrote:

On Fri, Jun 2, 2017 at 6:16 PM, Kunihiko Hayashi
 wrote:

I already suggested the patch to solve this issue.

https://patchwork.openembedded.org/patch/138596/

Although I've not received any comments on this yet, could you try to apply
it?


thanks for sending this out. Well, unfortunately, I think your patch
is doing something wrong.

As discussed earlier on that thread, we should be using kernel
Makefile to generate the target as much as possible. The linux build
system can generate vmlinux, Image and Image.gz, but not vmlinux.gz.
So we should compress only vmlinux 'outside' of the linux build
system, and for Image.gz we shouldn't do that in OE. In other words,
Image.gz should be included in typeformake. Your patch does not do
that, and that is something which is taken care by mine.


Surely we should use Image.gz generated by kernel build system and my patch  
doesn't do that. I also think that OE should compress only images that the  
kernel build system doesn't generate.

Sorry my patch won't help.

Best Regards,
Kunihiko Hayashi
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Kunihiko Hayashi]

On 2017/05/30 6:13, nicolas.deche...@linaro.org (Nicolas Dechesne) wrote:
> hi,
>
> i am trying to set  KERNEL_IMAGETYPES="Image.gz" in my machine conf
> file. But it doesn't work. I am seeing this error during
> kernel_do_compile():
>
> run.do_compile.30019: cannot open Image: No such file
>
> In fact the code snippet that tries to compress the Image file into
> Image.gz expects the Image file to be in ${B}, but when compiling
> kernel Image is actually in ${B}/arch/arm64/boot:
>
> $ find tmp-rpb-glibc/work/dragonboard_410c-linaro-linux/linux-linaro-qcomlt/
> -name Image
>  
tmp-rpb-glibc/work/dragonboard_410c-linaro-linux/linux-linaro-qcomlt/4.9-r0/build/arch/arm64/boot/Image

>
> The piece of code in question is in kernel.bbclass:
>
>   gzip -9c < "${typeformake}" > "${KERNEL_OUTPUT_DIR}/${type}"
> It really sounds to me like something obvious is missing, i doubt I
> would be the first one to try Image.gz..

I already suggested the patch to solve this issue.

https://patchwork.openembedded.org/patch/138596/

Although I've not received any comments on this yet, could you try to apply it?

>
> Any thoughts?
>
> thanks
> nico
>

Best Regards,
Kunihiko Hayashi
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Nicolas Dechesne]

On Fri, Jun 2, 2017 at 6:16 PM, Kunihiko Hayashi
 wrote:
> I already suggested the patch to solve this issue.
>
> https://patchwork.openembedded.org/patch/138596/
>
> Although I've not received any comments on this yet, could you try to apply
> it?

thanks for sending this out. Well, unfortunately, I think your patch
is doing something wrong.

As discussed earlier on that thread, we should be using kernel
Makefile to generate the target as much as possible. The linux build
system can generate vmlinux, Image and Image.gz, but not vmlinux.gz.
So we should compress only vmlinux 'outside' of the linux build
system, and for Image.gz we shouldn't do that in OE. In other words,
Image.gz should be included in typeformake. Your patch does not do
that, and that is something which is taken care by mine.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Nicolas Dechesne]

On Thu, Jun 1, 2017 at 12:56 AM, Andre McCurdy  wrote:
> Unfortunately this doesn't work if "vmlinux" isn't already included in

yes. correct. i noticed that when I wrote my patch.

> KERNEL_IMAGETYPES etc, so we either need to search and replace as
> before, or add something like:
>
> typeformake = types
> if 'vmlinux.gz' in types.split():
> typeformake = typeformake.replace('vmlinux.gz', '').strip()
> if 'vmlinux' not in typeformake.split():
> typeformake = typeformake + ' vmlinux'
>
> which is a little more complex, but has the advantage of not risking
> duplicating "vmlinux" in typeformake, and therefore not calling "make
> vmlinux" twice in kernel_do_compile(), which I guess we sometimes do
> now...

well, i think you are trying to fix 'another' issue here, since we
already have this issue before my proposed patch... so i would prefer
to treat that separately.  this is a more generic issue (e.g.
duplicate in KERNEL_IMAGETYPES) and we could simply remove  duplicates
from typeformake.

I will send out a proper patch with my proposed change.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Andre McCurdy]

 On Wed, May 31, 2017 at 3:04 PM, Andre McCurdy  wrote:
> On Wed, May 31, 2017 at 3:26 AM, Nicolas Dechesne
>  wrote:
>> On Tue, May 30, 2017 at 11:56 PM, Andre McCurdy  wrote:
>>> On Tue, May 30, 2017 at 1:26 PM, Nicolas Dechesne 
>>>  wrote:
 so, to backtrack a bit, i think it should be safe to only treat
 vmlinux.gz as a 'special' case, and then it is there in
 KERNEL_IMAGETYPE, then we do a "make vmlinux" and we manually compress
 the file, otherwise we just trust that KERNEL_IMAGETYPE is a valid
 type and give it to kernel make command line.  that should be enough
 to fix the Image.gz cases on all platforms where it's relevant (like
 arm).
>>>
>>> That sounds reasonable.
>>
>> I've been playing with this today, and that *seems* to work fine. At
>> least I can still build vmlinux, vmlinux.gz and it fixes the  build of
>> Image.gz as well..
>>
>> I would appreciate some feedback on this one. I can then send a proper
>> patch. Could that break badly for anyone?
>>
>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>> index 7a134d5c29..459b6d66a3 100644
>> --- a/meta/classes/kernel.bbclass
>> +++ b/meta/classes/kernel.bbclass
>> @@ -44,7 +44,10 @@ python __anonymous () {
>>  types = (alttype + ' ' + types).strip()
>>  d.setVar('KERNEL_IMAGETYPES', types)
>>
>> -typeformake = re.sub(r'\.gz', '', types)
>> +# some commonly used kernel images aren't generated by the kernel
>> build system, such as vmlinux.gz
>> +# typeformake lists only valid kernel make targets, and post
>> processing can be done after the kernel
>> +# is built (such as using gzip to compress vmlinux)
>> +typeformake = re.sub(r'vmlinux\.gz', 'vmlinux', types)
>>  d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake)
>>
>>  for type in types.split():
>>
>
> Looks good to me. Below is a version which takes the cleanup slightly
> further (it will no doubt get mangled since I'm copying and pasting
> into gmail, but maybe gives some more ideas for the final patch).
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 8954b28..264d5b7 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -32,8 +32,6 @@ KERNEL_VERSION_PKG_NAME =
> "${@legitimize_package_name(d.getVar('KERNEL_VERSION')
>  KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}"
>
>  python __anonymous () {
> -import re
> -
>  # Merge KERNEL_IMAGETYPE and KERNEL_ALT_IMAGETYPE into KERNEL_IMAGETYPES
>  type = d.getVar('KERNEL_IMAGETYPE') or ""
>  alttype = d.getVar('KERNEL_ALT_IMAGETYPE') or ""
> @@ -44,7 +42,13 @@ python __anonymous () {
>  types = (alttype + ' ' + types).strip()
>  d.setVar('KERNEL_IMAGETYPES', types)
>
> -typeformake = re.sub(r'\.gz', '', types)
> +# Almost all commonly used kernel images are generated by the kernel 
> build
> +# system directly. However some, such as vmlinux.gz, are created as 
> special
> +# cases by post-processing steps in kernel_do_compile() and
> should therefore
> +# not be included in typeformake.
> +
> +typeformake = types.replace('vmlinux.gz', '').strip()

Unfortunately this doesn't work if "vmlinux" isn't already included in
KERNEL_IMAGETYPES etc, so we either need to search and replace as
before, or add something like:

typeformake = types
if 'vmlinux.gz' in types.split():
typeformake = typeformake.replace('vmlinux.gz', '').strip()
if 'vmlinux' not in typeformake.split():
typeformake = typeformake + ' vmlinux'

which is a little more complex, but has the advantage of not risking
duplicating "vmlinux" in typeformake, and therefore not calling "make
vmlinux" twice in kernel_do_compile(), which I guess we sometimes do
now...

> +
>  d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake)
>
>  for type in types.split():
> @@ -266,14 +270,11 @@ kernel_do_compile() {
> fi
> for typeformake in ${KERNEL_IMAGETYPE_FOR_MAKE} ; do
> oe_runmake ${typeformake} CC="${KERNEL_CC}"
> LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
> -   for type in ${KERNEL_IMAGETYPES} ; do
> -   if test "${typeformake}.gz" = "${type}"; then
> -   mkdir -p "${KERNEL_OUTPUT_DIR}"
> -   gzip -9c < "${typeformake}" >
> "${KERNEL_OUTPUT_DIR}/${type}"
> -   break;
> -   fi
> -   done
> done
> +   if (echo "${KERNEL_IMAGETYPES}" | grep -wq "vmlinux\.gz"); then
> +   mkdir -p "${KERNEL_OUTPUT_DIR}"
> +   gzip -9c < ${B}/vmlinux > "${KERNEL_OUTPUT_DIR}/vmlinux.gz"
> +   fi
>  }
>
>  do_compile_kernelmodules() {
-- 
___
Openembedded-core mailing list

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Andre McCurdy]

On Wed, May 31, 2017 at 3:26 AM, Nicolas Dechesne
 wrote:
> On Tue, May 30, 2017 at 11:56 PM, Andre McCurdy  wrote:
>> On Tue, May 30, 2017 at 1:26 PM, Nicolas Dechesne 
>>  wrote:
>>> so, to backtrack a bit, i think it should be safe to only treat
>>> vmlinux.gz as a 'special' case, and then it is there in
>>> KERNEL_IMAGETYPE, then we do a "make vmlinux" and we manually compress
>>> the file, otherwise we just trust that KERNEL_IMAGETYPE is a valid
>>> type and give it to kernel make command line.  that should be enough
>>> to fix the Image.gz cases on all platforms where it's relevant (like
>>> arm).
>>
>> That sounds reasonable.
>
> I've been playing with this today, and that *seems* to work fine. At
> least I can still build vmlinux, vmlinux.gz and it fixes the  build of
> Image.gz as well..
>
> I would appreciate some feedback on this one. I can then send a proper
> patch. Could that break badly for anyone?
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 7a134d5c29..459b6d66a3 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -44,7 +44,10 @@ python __anonymous () {
>  types = (alttype + ' ' + types).strip()
>  d.setVar('KERNEL_IMAGETYPES', types)
>
> -typeformake = re.sub(r'\.gz', '', types)
> +# some commonly used kernel images aren't generated by the kernel
> build system, such as vmlinux.gz
> +# typeformake lists only valid kernel make targets, and post
> processing can be done after the kernel
> +# is built (such as using gzip to compress vmlinux)
> +typeformake = re.sub(r'vmlinux\.gz', 'vmlinux', types)
>  d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake)
>
>  for type in types.split():
>

Looks good to me. Below is a version which takes the cleanup slightly
further (it will no doubt get mangled since I'm copying and pasting
into gmail, but maybe gives some more ideas for the final patch).

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 8954b28..264d5b7 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -32,8 +32,6 @@ KERNEL_VERSION_PKG_NAME =
"${@legitimize_package_name(d.getVar('KERNEL_VERSION')
 KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}"

 python __anonymous () {
-import re
-
 # Merge KERNEL_IMAGETYPE and KERNEL_ALT_IMAGETYPE into KERNEL_IMAGETYPES
 type = d.getVar('KERNEL_IMAGETYPE') or ""
 alttype = d.getVar('KERNEL_ALT_IMAGETYPE') or ""
@@ -44,7 +42,13 @@ python __anonymous () {
 types = (alttype + ' ' + types).strip()
 d.setVar('KERNEL_IMAGETYPES', types)

-typeformake = re.sub(r'\.gz', '', types)
+# Almost all commonly used kernel images are generated by the kernel build
+# system directly. However some, such as vmlinux.gz, are created as special
+# cases by post-processing steps in kernel_do_compile() and
should therefore
+# not be included in typeformake.
+
+typeformake = types.replace('vmlinux.gz', '').strip()
+
 d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake)

 for type in types.split():
@@ -266,14 +270,11 @@ kernel_do_compile() {
fi
for typeformake in ${KERNEL_IMAGETYPE_FOR_MAKE} ; do
oe_runmake ${typeformake} CC="${KERNEL_CC}"
LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
-   for type in ${KERNEL_IMAGETYPES} ; do
-   if test "${typeformake}.gz" = "${type}"; then
-   mkdir -p "${KERNEL_OUTPUT_DIR}"
-   gzip -9c < "${typeformake}" >
"${KERNEL_OUTPUT_DIR}/${type}"
-   break;
-   fi
-   done
done
+   if (echo "${KERNEL_IMAGETYPES}" | grep -wq "vmlinux\.gz"); then
+   mkdir -p "${KERNEL_OUTPUT_DIR}"
+   gzip -9c < ${B}/vmlinux > "${KERNEL_OUTPUT_DIR}/vmlinux.gz"
+   fi
 }

 do_compile_kernelmodules() {
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Nicolas Dechesne]

On Tue, May 30, 2017 at 11:56 PM, Andre McCurdy  wrote:
> On Tue, May 30, 2017 at 1:26 PM, Nicolas Dechesne
>  wrote:
>> On Tue, May 30, 2017 at 7:15 PM, Andre McCurdy  wrote:
   gzip -9c < "${typeformake}" > "${KERNEL_OUTPUT_DIR}/${type}"
 It really sounds to me like something obvious is missing, i doubt I
 would be the first one to try Image.gz..
>>>
>>> You may be the first to try for a non-MIPS machine.
>>>
>>> MIPS is a little special in that the MIPS kernel doesn't support self
>>> extracting compressed kernels (zImage, etc), so MIPS machines are the
>>> most likely to make use of a gzipped vmlinux image (which will then be
>>> decompressed by the bootloader) and KERNEL_IMAGETYPES="vmlinux.gz"
>>> works OK.
>>>
>>> If a compressed kernel is required, most non-MIPS machines use a self
>>> extracting image.
>>
>> this piece of is coming from this old commit:
>>
>> cb17b6c2a7 kernel.bbclass: support kernel image type of vmlinux.gz
>>
>> It is clear that the goal of this logic is only to support vmlinux.gz
>> since the kernel build system does not create it (only vmlinux). It
>> almost looks to me like we should treat vmlinux.gz as a special case
>> instead of treating each .gz target as 'special'
>>
>> -typeformake = re.sub(r'\.gz', '', types)
>> +typeformake = re.sub(r'vmlinux\.gz', '', types)
>>
>> e.g. the special case should be used *only* when the kernel build
>> system is not capable to create the compressed file. e.g. on ARM ,
>> Image, Image.gz and vmlinux are valid image that the kernel build can
>> generate, so it probably doesn't make any sense to have vmlinux.gz in
>> KERNEL_IMAGETYPES anyways..
>
> I agree that kernel.bbclass should not take over creation of an image
> type which the kernel can handle directly, and if that means special
> cases in kernel.bbclass to handle differences in which image types can
> be created by the kernel for each architecture then so be it.
> Documenting the special cases makes it less likely they will get
> overlooked and broken by future refactoring etc.
>
> I'm not sure about not making sense to have "vmlinux.gz" in
> KERNEL_IMAGETYPES though? It is a valid image type for MIPS and it
> should be possible for machine configs to choose to build it.

that's fine. i am not familiar with MIPS, if you say so, it's ok ;-)

>
>> In fact , looking more closely at kernel makefiles (for arch/mips,
>> since you brought this one up), it looks like uImage (and compressed
>> vmlinux) was added in 2013 (commit
>> 4defe4559e86e26545ab2f542656a4b966dcde53). So we might not need all
>> this logic at all anymore in kernel.bbclass and just get rid of it.
>
> I think vmlinux is an elf file and vmlinux.bin is the raw binary (?),
> so even though the kernel can create vmlinux.bin.gz directly it
> doesn't cover systems which need a gzipped elf file (ie vmlinux.gz).
>
>> so, to backtrack a bit, i think it should be safe to only treat
>> vmlinux.gz as a 'special' case, and then it is there in
>> KERNEL_IMAGETYPE, then we do a "make vmlinux" and we manually compress
>> the file, otherwise we just trust that KERNEL_IMAGETYPE is a valid
>> type and give it to kernel make command line.  that should be enough
>> to fix the Image.gz cases on all platforms where it's relevant (like
>> arm).
>
> That sounds reasonable.

I've been playing with this today, and that *seems* to work fine. At
least I can still build vmlinux, vmlinux.gz and it fixes the  build of
Image.gz as well..

I would appreciate some feedback on this one. I can then send a proper
patch. Could that break badly for anyone?

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 7a134d5c29..459b6d66a3 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -44,7 +44,10 @@ python __anonymous () {
 types = (alttype + ' ' + types).strip()
 d.setVar('KERNEL_IMAGETYPES', types)

-typeformake = re.sub(r'\.gz', '', types)
+# some commonly used kernel images aren't generated by the kernel
build system, such as vmlinux.gz
+# typeformake lists only valid kernel make targets, and post
processing can be done after the kernel
+# is built (such as using gzip to compress vmlinux)
+typeformake = re.sub(r'vmlinux\.gz', 'vmlinux', types)
 d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake)

 for type in types.split():


>
>> then a second step might be to check if we even need that mechanism
>> anymore.. but that might be a more tricky change.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Andre McCurdy]

On Tue, May 30, 2017 at 1:26 PM, Nicolas Dechesne
 wrote:
> On Tue, May 30, 2017 at 7:15 PM, Andre McCurdy  wrote:
>>>   gzip -9c < "${typeformake}" > "${KERNEL_OUTPUT_DIR}/${type}"
>>> It really sounds to me like something obvious is missing, i doubt I
>>> would be the first one to try Image.gz..
>>
>> You may be the first to try for a non-MIPS machine.
>>
>> MIPS is a little special in that the MIPS kernel doesn't support self
>> extracting compressed kernels (zImage, etc), so MIPS machines are the
>> most likely to make use of a gzipped vmlinux image (which will then be
>> decompressed by the bootloader) and KERNEL_IMAGETYPES="vmlinux.gz"
>> works OK.
>>
>> If a compressed kernel is required, most non-MIPS machines use a self
>> extracting image.
>
> this piece of is coming from this old commit:
>
> cb17b6c2a7 kernel.bbclass: support kernel image type of vmlinux.gz
>
> It is clear that the goal of this logic is only to support vmlinux.gz
> since the kernel build system does not create it (only vmlinux). It
> almost looks to me like we should treat vmlinux.gz as a special case
> instead of treating each .gz target as 'special'
>
> -typeformake = re.sub(r'\.gz', '', types)
> +typeformake = re.sub(r'vmlinux\.gz', '', types)
>
> e.g. the special case should be used *only* when the kernel build
> system is not capable to create the compressed file. e.g. on ARM ,
> Image, Image.gz and vmlinux are valid image that the kernel build can
> generate, so it probably doesn't make any sense to have vmlinux.gz in
> KERNEL_IMAGETYPES anyways..

I agree that kernel.bbclass should not take over creation of an image
type which the kernel can handle directly, and if that means special
cases in kernel.bbclass to handle differences in which image types can
be created by the kernel for each architecture then so be it.
Documenting the special cases makes it less likely they will get
overlooked and broken by future refactoring etc.

I'm not sure about not making sense to have "vmlinux.gz" in
KERNEL_IMAGETYPES though? It is a valid image type for MIPS and it
should be possible for machine configs to choose to build it.

> In fact , looking more closely at kernel makefiles (for arch/mips,
> since you brought this one up), it looks like uImage (and compressed
> vmlinux) was added in 2013 (commit
> 4defe4559e86e26545ab2f542656a4b966dcde53). So we might not need all
> this logic at all anymore in kernel.bbclass and just get rid of it.

I think vmlinux is an elf file and vmlinux.bin is the raw binary (?),
so even though the kernel can create vmlinux.bin.gz directly it
doesn't cover systems which need a gzipped elf file (ie vmlinux.gz).

> so, to backtrack a bit, i think it should be safe to only treat
> vmlinux.gz as a 'special' case, and then it is there in
> KERNEL_IMAGETYPE, then we do a "make vmlinux" and we manually compress
> the file, otherwise we just trust that KERNEL_IMAGETYPE is a valid
> type and give it to kernel make command line.  that should be enough
> to fix the Image.gz cases on all platforms where it's relevant (like
> arm).

That sounds reasonable.

> then a second step might be to check if we even need that mechanism
> anymore.. but that might be a more tricky change.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Nicolas Dechesne]

On Tue, May 30, 2017 at 7:15 PM, Andre McCurdy  wrote:
>>   gzip -9c < "${typeformake}" > "${KERNEL_OUTPUT_DIR}/${type}"
>> It really sounds to me like something obvious is missing, i doubt I
>> would be the first one to try Image.gz..
>
> You may be the first to try for a non-MIPS machine.
>
> MIPS is a little special in that the MIPS kernel doesn't support self
> extracting compressed kernels (zImage, etc), so MIPS machines are the
> most likely to make use of a gzipped vmlinux image (which will then be
> decompressed by the bootloader) and KERNEL_IMAGETYPES="vmlinux.gz"
> works OK.
>
> If a compressed kernel is required, most non-MIPS machines use a self
> extracting image.

this piece of is coming from this old commit:

cb17b6c2a7 kernel.bbclass: support kernel image type of vmlinux.gz

It is clear that the goal of this logic is only to support vmlinux.gz
since the kernel build system does not create it (only vmlinux). It
almost looks to me like we should treat vmlinux.gz as a special case
instead of treating each .gz target as 'special'

-typeformake = re.sub(r'\.gz', '', types)
+typeformake = re.sub(r'vmlinux\.gz', '', types)

e.g. the special case should be used *only* when the kernel build
system is not capable to create the compressed file. e.g. on ARM ,
Image, Image.gz and vmlinux are valid image that the kernel build can
generate, so it probably doesn't make any sense to have vmlinux.gz in
KERNEL_IMAGETYPES anyways..

In fact , looking more closely at kernel makefiles (for arch/mips,
since you brought this one up), it looks like uImage (and compressed
vmlinux) was added in 2013 (commit
4defe4559e86e26545ab2f542656a4b966dcde53). So we might not need all
this logic at all anymore in kernel.bbclass and just get rid of it.

so, to backtrack a bit, i think it should be safe to only treat
vmlinux.gz as a 'special' case, and then it is there in
KERNEL_IMAGETYPE, then we do a "make vmlinux" and we manually compress
the file, otherwise we just trust that KERNEL_IMAGETYPE is a valid
type and give it to kernel make command line.  that should be enough
to fix the Image.gz cases on all platforms where it's relevant (like
arm).

then a second step might be to check if we even need that mechanism
anymore.. but that might be a more tricky change.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Andre McCurdy]

On Mon, May 29, 2017 at 2:13 PM, Nicolas Dechesne
 wrote:
> hi,
>
> i am trying to set  KERNEL_IMAGETYPES="Image.gz" in my machine conf
> file. But it doesn't work. I am seeing this error during
> kernel_do_compile():
>
> run.do_compile.30019: cannot open Image: No such file
>
> In fact the code snippet that tries to compress the Image file into
> Image.gz expects the Image file to be in ${B}, but when compiling
> kernel Image is actually in ${B}/arch/arm64/boot:
>
> $ find tmp-rpb-glibc/work/dragonboard_410c-linaro-linux/linux-linaro-qcomlt/
> -name Image
> tmp-rpb-glibc/work/dragonboard_410c-linaro-linux/linux-linaro-qcomlt/4.9-r0/build/arch/arm64/boot/Image
>
> The piece of code in question is in kernel.bbclass:
>
>   gzip -9c < "${typeformake}" > "${KERNEL_OUTPUT_DIR}/${type}"
> It really sounds to me like something obvious is missing, i doubt I
> would be the first one to try Image.gz..

You may be the first to try for a non-MIPS machine.

MIPS is a little special in that the MIPS kernel doesn't support self
extracting compressed kernels (zImage, etc), so MIPS machines are the
most likely to make use of a gzipped vmlinux image (which will then be
decompressed by the bootloader) and KERNEL_IMAGETYPES="vmlinux.gz"
works OK.

If a compressed kernel is required, most non-MIPS machines use a self
extracting image.

> Any thoughts?
>
> thanks
> nico
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] KERNEL_IMAGETYPES and Image.gz

[Khem Raj]

On Mon, May 29, 2017 at 2:13 PM, Nicolas Dechesne
 wrote:
> hi,
>
> i am trying to set  KERNEL_IMAGETYPES="Image.gz" in my machine conf
> file. But it doesn't work. I am seeing this error during
> kernel_do_compile():
>
> run.do_compile.30019: cannot open Image: No such file
>
> In fact the code snippet that tries to compress the Image file into
> Image.gz expects the Image file to be in ${B}, but when compiling
> kernel Image is actually in ${B}/arch/arm64/boot:

is that always the case for all architectures ? that it puts the Image
ion arch/$ARCH/boot ?

>
> $ find tmp-rpb-glibc/work/dragonboard_410c-linaro-linux/linux-linaro-qcomlt/
> -name Image
> tmp-rpb-glibc/work/dragonboard_410c-linaro-linux/linux-linaro-qcomlt/4.9-r0/build/arch/arm64/boot/Image
>
> The piece of code in question is in kernel.bbclass:
>
>   gzip -9c < "${typeformake}" > "${KERNEL_OUTPUT_DIR}/${type}"
> It really sounds to me like something obvious is missing, i doubt I
> would be the first one to try Image.gz..
>
> Any thoughts?
>
> thanks
> nico
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core