Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On Mon, Jul 30, 2018 at 7:26 PM,   wrote:
> On Thu, 2018-07-26 at 10:04 -0400, Bruce Ashfield wrote:
>> > https://autobuilder.yocto.io/builders/nightly-multilib/builds/1139/
>> > steps/BuildImages_4/logs/stdio
>> >
>> > Error: Transaction check error:
>> >file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586
>> > conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
>> >file /usr/bin/libtoolize from install of libtool-2.4.6-r0.0.i586
>> > conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
>> >
>>
>> And that leaves this error.
>>
>> I've not been able to reproduce it, and I've been looking at the
>> elfutils RDEPENDS that is part of kernel devsrc, since that is what
>> pulls in libtool as a DEPENDS.
>
> I've just sent out two patches which together should fix this one, they
> did locally at least...

Nice. I see them!

I have a v2 of this patch ready to go, but am out of the office for
the next few days. When I'm to my final vacation destination, I'll log
into my build machine and send that patch.

Bruce

>
> Cheers,
>
> Richard
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[richard . purdie]

On Thu, 2018-07-26 at 10:04 -0400, Bruce Ashfield wrote:
> > https://autobuilder.yocto.io/builders/nightly-multilib/builds/1139/
> > steps/BuildImages_4/logs/stdio
> > 
> > Error: Transaction check error:
> >file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586
> > conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
> >file /usr/bin/libtoolize from install of libtool-2.4.6-r0.0.i586 
> > conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
> > 
> 
> And that leaves this error.
> 
> I've not been able to reproduce it, and I've been looking at the
> elfutils RDEPENDS that is part of kernel devsrc, since that is what
> pulls in libtool as a DEPENDS.

I've just sent out two patches which together should fix this one, they
did locally at least...

Cheers,

Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Richard Purdie]

On Thu, 2018-07-26 at 15:17 +0100, richard.pur...@linuxfoundation.org
wrote:
> On Thu, 2018-07-26 at 10:04 -0400, Bruce Ashfield wrote:
> > On 2018-07-10 6:21 AM, Richard Purdie wrote:
> > > and also a failure in one of the multilib builds, probably from
> > > different dependencies pulled in by kernel-devsrc:
> > > 
> > > https://autobuilder.yocto.io/builders/nightly-multilib/builds/113
> > > 9/
> > > steps/BuildImages_4/logs/stdio
> > > 
> > > Error: Transaction check error:
> > >file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586
> > > conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
> > >file /usr/bin/libtoolize from install of libtool-2.4.6-
> > > r0.0.i586 
> > > conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
> > > 
> > 
> > And that leaves this error.
> > 
> > I've not been able to reproduce it, and I've been looking at the
> > elfutils RDEPENDS that is part of kernel devsrc, since that is what
> > pulls in libtool as a DEPENDS.
> > 
> > I'm not seeing any difference in the way that it is pulled in,
> > versus
> > the other uses.
> > 
> > In particular, perf also has elfutils as a REDEPENDS, but yet it
> > builds
> > in the same configuration. Can you spot the difference in how it is
> > used ?
> > 
> > elfutils is potentially required at tools build time, hence why it
> > is
> > a RDEPENDS in devsrc, but a DEPENDS in the main linux-yocto
> > recipes.
> > If there's another way to express this, let me know and I can
> > switch
> > to that.
> > 
> > And finally, I did build the old reproducing configuration for
> > multlib that you mentioned before .. can you point me at this
> > config,
> > so I can try again ?
> 
> This is a timely reminder that I've been meaning to look at this. I
> think the config is:
> t
> MACHINE=qemux86
> require conf/multilib.conf 
> MULTILIBS = "multilib:lib64" 
> DEFAULTTUNE_virtclass-multilib-lib64 = "x86-64" 
> 
> then bitbake lib64-core-image-sato-sdk
> 
> which seems like an odd mix of 64bit with a 32 bit kernel but is
> nonetheless what it appears to be building. I suspect it will fail
> the same way with the x86-64 machine too though.
> 
> I have a build running now to check if I can replicate and perhaps
> dig into it further.

I've confirmed the above does reproduce the failure. The logs are also
helpful and show that whilst it mostly installs lib64-* packages, it
also installs a handful of "normal" ones.

What appears to happen is it installs kernel-devsrc, which depends on
elfutils. Since its a "devel" image, it installs the corresponding -dev
packages, which means elfutils-dev. elfutils-dev rrecommends libtool-
dev.

Since there are other lib64-* packages, one is bound to have a -dev
package which installs lib64-libtool-dev.

libtool-dev depends on libtool, lib64-libtool-dev on lib64-libtool.

Then we discover you can't install libtool and lib64-libtool onto the
system at the same time as they're not compatible with each other,
hence the error message.

I tried a quick hack to the elfutils recipe:

package_depchains_append () {
rreclist = 
bb.utils.explode_dep_versions2(d.getVar('RRECOMMENDS_elfutils-dev') or "")
if "libtool-dev" in rreclist:
del rreclist["libtool-dev"]
d.setVar('RRECOMMENDS_elfutils-dev', bb.utils.join_deps(rreclist, 
commasep=False))
}

which is pretty evil but proves this is the only issue we have to solve
since with the hack, the build completes.

So I'm not sure how to fix it but at least we can say why it breaks...

Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[richard . purdie]

On Thu, 2018-07-26 at 10:04 -0400, Bruce Ashfield wrote:
> On 2018-07-10 6:21 AM, Richard Purdie wrote:
> > and also a failure in one of the multilib builds, probably from
> > different dependencies pulled in by kernel-devsrc:
> > 
> > https://autobuilder.yocto.io/builders/nightly-multilib/builds/1139/
> > steps/BuildImages_4/logs/stdio
> > 
> > Error: Transaction check error:
> >file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586
> > conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
> >file /usr/bin/libtoolize from install of libtool-2.4.6-r0.0.i586 
> > conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
> > 
> 
> And that leaves this error.
> 
> I've not been able to reproduce it, and I've been looking at the
> elfutils RDEPENDS that is part of kernel devsrc, since that is what
> pulls in libtool as a DEPENDS.
> 
> I'm not seeing any difference in the way that it is pulled in, versus
> the other uses.
> 
> In particular, perf also has elfutils as a REDEPENDS, but yet it
> builds
> in the same configuration. Can you spot the difference in how it is
> used ?
> 
> elfutils is potentially required at tools build time, hence why it is
> a RDEPENDS in devsrc, but a DEPENDS in the main linux-yocto recipes.
> If there's another way to express this, let me know and I can switch
> to that.
> 
> And finally, I did build the old reproducing configuration for
> multlib that you mentioned before .. can you point me at this config,
> so I can try again ?

This is a timely reminder that I've been meaning to look at this. I
think the config is:
t
MACHINE=qemux86
require conf/multilib.conf 
MULTILIBS = "multilib:lib64" 
DEFAULTTUNE_virtclass-multilib-lib64 = "x86-64" 

then bitbake lib64-core-image-sato-sdk

which seems like an odd mix of 64bit with a 32 bit kernel but is
nonetheless what it appears to be building. I suspect it will fail the
same way with the x86-64 machine too though.

I have a build running now to check if I can replicate and perhaps dig
into it further.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 2018-07-10 6:21 AM, Richard Purdie wrote:

On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:

The existing kernel-devsrc package starts with a full copy of the
kernel
source and then starts to strip out elements that are not required.

This results in extra time (I/O) and extra space being taken up in
the
final package. The main purpose of the kernel-devsrc package has been
to
build modules against the running kernel, not to include a full copy
of
the source code for re-building the kernel. The end result was a
600M kernel-devsrc package.

This restructuring of the package uses an approach similar to other
distros, where the kernel-devsrc package is for building against the
running kernel and uses a curated set of copied infrastructure,
versus
a mass copy of the entire kernel.

The differences in this approach versus other is largely due to the
architecture support and the split build/source directory of the
kernel.

The result is a kernel-devsrc package of about 10M, which is capable
of running "make scripts" and compiling kernel modules against the
running kernel.

Along with the changes to the copying of the infrascture, we also
have the following changes:

  - a better/more explicit listing of dependencies for on-target
builds of "make scripts" or "make modules_prepare"

  - The kernel source is installed into /lib/modules//build
and a symlink created from /usr/src/kernel to the new location.
This aligns with the standard location for module support
code

  - There is also a symlink from /lib/modules//source ->
build
to reserve a spot for a new package that is simply the kernel
source. That package is not part of this update.

Signed-off-by: Bruce Ashfield 
---

v2: drop DEPENDS on perf. We no longer need to depend on perf since
the source
 is copied before modification.


It probably won't surprise you to know there were some issues with this
patch, sadly.

The main recurring issue (on all arches and poky+poky-lsb) is failure
of the kernelmodule.KernelModuleTest.test_kernel_module test. The exact
failure varies by arch, for x86-64:


FYI. I have fixed all of these errors, and will submit a v2 shortly.
See a few more comments below.



https://autobuilder.yocto.io/builders/nightly-x86-64/builds/1154/steps/Running%20Sanity%20Tests/logs/stdio

|   CHK include/generated/utsrelease.h
|   DESCEND  objtool
| /lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.build:37: 
/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include: No such 
file or directory
| make[4]: *** No rule to make target 
'/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include'.  Stop.
| make[3]: *** [Makefile:43: 
/lib/modules/4.14.48-yocto-standard/build/tools/objtool/fixdep-in.o] Error 2
| make[2]: *** 
[/lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.include:4: 
fixdep] Error 2
| make[1]: *** [Makefile:62: objtool] Error 2
| make: *** [Makefile:1647: tools/objtool] Error 2

For mips:

https://autobuilder.yocto.io/builders/nightly-mips/builds/1115

|   HOSTCC  scripts/sortextable
| make[1]: *** No rule to make target 'arch/mips/boot/tools/relocs_32.c', 
needed by 'arch/mips/boot/tools/relocs_32.o'.  Stop.
| make: *** [arch/mips/Makefile:16: archscripts] Error 2



I still can't build lttng-modules for mips, but I have fixed the missing
file and ensured that I can make scripts/prepare on the qemu targets.


(same for mips64)

For arm:

https://autobuilder.yocto.io/builders/nightly-arm/builds/1187/

|   HOSTCC  scripts/sortextable
| make[1]: *** No rule to make target 'arch/arm/tools/syscall.tbl', needed by 
'arch/arm/include/generated/uapi/asm/unistd-common.h'.  Stop.
| make: *** [arch/arm/Makefile:319: archheaders] Error 2

For arm64:

https://autobuilder.yocto.io/builders/nightly-arm64/builds/1101/steps/Running%20Sanity%20Tests/logs/stdio

|   CHK include/generated/uapi/linux/version.h
|   CHK include/generated/utsrelease.h
| make[1]: *** No rule to make target 'arch/arm64/kernel/vdso/vdso.lds', needed 
by 'arch/arm64/kernel/vdso/vdso.so.dbg'.  Stop.
| make: *** [arch/arm64/Makefile:160: vdso_prepare] Error 2
  


There was also a failure in building the build-appliance image:

https://autobuilder.yocto.io/builders/build-appliance/builds/1110

and also a failure in one of the multilib builds, probably from
different dependencies pulled in by kernel-devsrc:

https://autobuilder.yocto.io/builders/nightly-multilib/builds/1139/steps/BuildImages_4/logs/stdio

Error: Transaction check error:
   file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586 conflicts with 
file from package lib64-libtool-2.4.6-r0.0.x86_64
   file /usr/bin/libtoolize from install of libtool-2.4.6-r0.0.i586 conflicts 
with file from package lib64-libtool-2.4.6-r0.0.x86_64



And that leaves this error.

I've not been able to reproduce it, and I've been looking at the
elfutils RDEPENDS that is part of kernel devsrc, since that is what
pulls in libtool as a 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 2018-07-12 1:22 PM, Khem Raj wrote:

On 7/12/18 11:07 AM, Bruce Ashfield wrote:

On 2018-07-12 1:02 PM, Khem Raj wrote:

On 7/12/18 7:55 AM, Bruce Ashfield wrote:

On 2018-07-12 9:53 AM, Richard Purdie wrote:

On Thu, 2018-07-12 at 09:49 -0400, Bruce Ashfield wrote:

On 2018-07-10 5:41 PM, Richard Purdie wrote:

On Tue, 2018-07-10 at 12:38 -0400, Bruce Ashfield wrote:

On 07/10/2018 06:21 AM, Richard Purdie wrote:

On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:


I'll try the other configs, but clearly I have something
different in
my x86-64 build.

I can't run the self tests directly, but am copying the files
onto my
qemu session and running things myself:

root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make
ARCH=x86
scripts prepare
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
install
libelf-dev, libelf-devel or elfutils-libelf-devel"
   CHK scripts/mod/devicetable-offsets.h
   SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
   SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
   SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
   SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
   SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
   SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
   SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
   HOSTCC  arch/x86/tools/relocs_32.o
   HOSTCC  arch/x86/tools/relocs_64.o
   HOSTCC  arch/x86/tools/relocs_common.o
   HOSTLD  arch/x86/tools/relocs
   CHK include/config/kernel.release
   CHK include/generated/uapi/linux/version.h
   CHK include/generated/utsrelease.h
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
install
libelf-dev, libelf-devel or elfutils-libelf-devel"

root@qemux86-64:/tmp# make
make -C /usr/src/kernel M=/tmp modules
make[1]: Entering directory '/lib/modules/4.14.48-yocto-
standard/build'
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
install
libelf-dev, libelf-devel or elfutils-libelf-devel"
   Building modules, stage 2.
   MODPOST 1 modules
make[1]: Leaving directory '/lib/modules/4.14.48-yocto-
standard/build'
root@qemux86-64:/tmp# insmod hellomod.ko
[  419.211616] hellomod: loading out-of-tree module taints
kernel.
[  419.212586] Hello world!

This is on the core-image-kernel-dev image, which is the one that
I created to test all this.

What image is that using ? some sato sdk one ?


We really need to find you a way to run these. The problem is the
qemu
graphics piece? You can't have a dummy vncserver or something to
direct
them at?

The image is a core-image-sato-sdk...


I was able to trigger the objtool issue with this image, looking at
fixing it and the other arches now.


Since its image related, is it a missing package dependency? Just
curious why its happening with some images but not your test ones...


It is. I have extra toolchain elements being installed in
core-image-kernel-dev
that hold things together (i.e. the kernel's objtool doesn't need to
be built).

The obvious fix is to just add an RDEPENDS, but I'm being stubborn
and trying to make it build on target, since the build of objtool from
the kernel source is showing me some sort of include path issue that
I'd like to sort out:

    CC /lib/modules/4.14.48-yocto-standard/build/tools/objtool/exec-cmd.o
exec-cmd.c: In function 'get_pwd_cwd':
exec-cmd.c:49:4: error: implicit declaration of function 'strlcpy'; did
you mean 'strncpy'? [-Werror=implicit-function-declaration]
  strlcpy(cwd, pwd, PATH_MAX);
  ^~~
  strncpy
exec-cmd.c:49:4: error: nested extern declaration of 'strlcpy'
[-Werror=nested-externs]
cc1: all warnings being treated as errors



Kernel has this function defined in tools/include/linux/string.h so



Well yes, a grep locates that :D



however it has it like this

#if defined(__GLIBC__) && !defined(__UCLIBC__)
extern size_t strlcpy(char *dest, const char *src, size_t size);
#endif

Which means it will not work for musl and we need to ensure that we
use -D_GNU_SOURCE for it to be included from standard musl headers


I've never built or supported musl, and since this is using the
in kernel Makefiles, etc, it isn't something I can patch or fix
outside of linux-yocto.

I'm just worried about the glibc cases, we can do follow up patches
later.


you dont have to defer since you are already touching this area you
might very well address this too.
Once the -I path is figured out. All we need is ensure -D_GNU_SOURCE is
added to CFLAGS in same place.


I'll see how nasty it looks, but yah, if I'm in modifying things
anyway, I'll make sure that is added.

Bruce




Bruce





Cheers,

Bruce



Cheers,

Richard













--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Khem Raj]

On 7/12/18 11:07 AM, Bruce Ashfield wrote:
> On 2018-07-12 1:02 PM, Khem Raj wrote:
>> On 7/12/18 7:55 AM, Bruce Ashfield wrote:
>>> On 2018-07-12 9:53 AM, Richard Purdie wrote:
 On Thu, 2018-07-12 at 09:49 -0400, Bruce Ashfield wrote:
> On 2018-07-10 5:41 PM, Richard Purdie wrote:
>> On Tue, 2018-07-10 at 12:38 -0400, Bruce Ashfield wrote:
>>> On 07/10/2018 06:21 AM, Richard Purdie wrote:
 On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:
>>>
>>> I'll try the other configs, but clearly I have something
>>> different in
>>> my x86-64 build.
>>>
>>> I can't run the self tests directly, but am copying the files
>>> onto my
>>> qemu session and running things myself:
>>>
>>> root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make
>>> ARCH=x86
>>> scripts prepare
>>> Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
>>> install
>>> libelf-dev, libelf-devel or elfutils-libelf-devel"
>>>   CHK scripts/mod/devicetable-offsets.h
>>>   SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
>>>   SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
>>>   SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
>>>   SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
>>>   SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
>>>   SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
>>>   SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
>>>   HOSTCC  arch/x86/tools/relocs_32.o
>>>   HOSTCC  arch/x86/tools/relocs_64.o
>>>   HOSTCC  arch/x86/tools/relocs_common.o
>>>   HOSTLD  arch/x86/tools/relocs
>>>   CHK include/config/kernel.release
>>>   CHK include/generated/uapi/linux/version.h
>>>   CHK include/generated/utsrelease.h
>>> Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
>>> install
>>> libelf-dev, libelf-devel or elfutils-libelf-devel"
>>>
>>> root@qemux86-64:/tmp# make
>>> make -C /usr/src/kernel M=/tmp modules
>>> make[1]: Entering directory '/lib/modules/4.14.48-yocto-
>>> standard/build'
>>> Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
>>> install
>>> libelf-dev, libelf-devel or elfutils-libelf-devel"
>>>   Building modules, stage 2.
>>>   MODPOST 1 modules
>>> make[1]: Leaving directory '/lib/modules/4.14.48-yocto-
>>> standard/build'
>>> root@qemux86-64:/tmp# insmod hellomod.ko
>>> [  419.211616] hellomod: loading out-of-tree module taints
>>> kernel.
>>> [  419.212586] Hello world!
>>>
>>> This is on the core-image-kernel-dev image, which is the one that
>>> I created to test all this.
>>>
>>> What image is that using ? some sato sdk one ?
>>
>> We really need to find you a way to run these. The problem is the
>> qemu
>> graphics piece? You can't have a dummy vncserver or something to
>> direct
>> them at?
>>
>> The image is a core-image-sato-sdk...
>
> I was able to trigger the objtool issue with this image, looking at
> fixing it and the other arches now.

 Since its image related, is it a missing package dependency? Just
 curious why its happening with some images but not your test ones...
>>>
>>> It is. I have extra toolchain elements being installed in
>>> core-image-kernel-dev
>>> that hold things together (i.e. the kernel's objtool doesn't need to
>>> be built).
>>>
>>> The obvious fix is to just add an RDEPENDS, but I'm being stubborn
>>> and trying to make it build on target, since the build of objtool from
>>> the kernel source is showing me some sort of include path issue that
>>> I'd like to sort out:
>>>
>>>    CC /lib/modules/4.14.48-yocto-standard/build/tools/objtool/exec-cmd.o
>>> exec-cmd.c: In function 'get_pwd_cwd':
>>> exec-cmd.c:49:4: error: implicit declaration of function 'strlcpy'; did
>>> you mean 'strncpy'? [-Werror=implicit-function-declaration]
>>>  strlcpy(cwd, pwd, PATH_MAX);
>>>  ^~~
>>>  strncpy
>>> exec-cmd.c:49:4: error: nested extern declaration of 'strlcpy'
>>> [-Werror=nested-externs]
>>> cc1: all warnings being treated as errors
>>>
>>
>> Kernel has this function defined in tools/include/linux/string.h so
>>
> 
> Well yes, a grep locates that :D
> 
> 
>> however it has it like this
>>
>> #if defined(__GLIBC__) && !defined(__UCLIBC__)
>> extern size_t strlcpy(char *dest, const char *src, size_t size);
>> #endif
>>
>> Which means it will not work for musl and we need to ensure that we
>> use -D_GNU_SOURCE for it to be included from standard musl headers
> 
> I've never built or supported musl, and since this is using the
> in kernel Makefiles, etc, it isn't something I can patch or fix
> outside of linux-yocto.
> 
> I'm just worried about the glibc cases, we can do follow up patches
> later.
> 
you dont have to 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 2018-07-12 1:02 PM, Khem Raj wrote:

On 7/12/18 7:55 AM, Bruce Ashfield wrote:

On 2018-07-12 9:53 AM, Richard Purdie wrote:

On Thu, 2018-07-12 at 09:49 -0400, Bruce Ashfield wrote:

On 2018-07-10 5:41 PM, Richard Purdie wrote:

On Tue, 2018-07-10 at 12:38 -0400, Bruce Ashfield wrote:

On 07/10/2018 06:21 AM, Richard Purdie wrote:

On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:


I'll try the other configs, but clearly I have something
different in
my x86-64 build.

I can't run the self tests directly, but am copying the files
onto my
qemu session and running things myself:

root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make
ARCH=x86
scripts prepare
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
install
libelf-dev, libelf-devel or elfutils-libelf-devel"
  CHK scripts/mod/devicetable-offsets.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
  HOSTCC  arch/x86/tools/relocs_32.o
  HOSTCC  arch/x86/tools/relocs_64.o
  HOSTCC  arch/x86/tools/relocs_common.o
  HOSTLD  arch/x86/tools/relocs
  CHK include/config/kernel.release
  CHK include/generated/uapi/linux/version.h
  CHK include/generated/utsrelease.h
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
install
libelf-dev, libelf-devel or elfutils-libelf-devel"

root@qemux86-64:/tmp# make
make -C /usr/src/kernel M=/tmp modules
make[1]: Entering directory '/lib/modules/4.14.48-yocto-
standard/build'
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
install
libelf-dev, libelf-devel or elfutils-libelf-devel"
  Building modules, stage 2.
  MODPOST 1 modules
make[1]: Leaving directory '/lib/modules/4.14.48-yocto-
standard/build'
root@qemux86-64:/tmp# insmod hellomod.ko
[  419.211616] hellomod: loading out-of-tree module taints
kernel.
[  419.212586] Hello world!

This is on the core-image-kernel-dev image, which is the one that
I created to test all this.

What image is that using ? some sato sdk one ?


We really need to find you a way to run these. The problem is the
qemu
graphics piece? You can't have a dummy vncserver or something to
direct
them at?

The image is a core-image-sato-sdk...


I was able to trigger the objtool issue with this image, looking at
fixing it and the other arches now.


Since its image related, is it a missing package dependency? Just
curious why its happening with some images but not your test ones...


It is. I have extra toolchain elements being installed in
core-image-kernel-dev
that hold things together (i.e. the kernel's objtool doesn't need to
be built).

The obvious fix is to just add an RDEPENDS, but I'm being stubborn
and trying to make it build on target, since the build of objtool from
the kernel source is showing me some sort of include path issue that
I'd like to sort out:

   CC /lib/modules/4.14.48-yocto-standard/build/tools/objtool/exec-cmd.o
exec-cmd.c: In function 'get_pwd_cwd':
exec-cmd.c:49:4: error: implicit declaration of function 'strlcpy'; did
you mean 'strncpy'? [-Werror=implicit-function-declaration]
     strlcpy(cwd, pwd, PATH_MAX);
     ^~~
     strncpy
exec-cmd.c:49:4: error: nested extern declaration of 'strlcpy'
[-Werror=nested-externs]
cc1: all warnings being treated as errors



Kernel has this function defined in tools/include/linux/string.h so



Well yes, a grep locates that :D



however it has it like this

#if defined(__GLIBC__) && !defined(__UCLIBC__)
extern size_t strlcpy(char *dest, const char *src, size_t size);
#endif

Which means it will not work for musl and we need to ensure that we
use -D_GNU_SOURCE for it to be included from standard musl headers


I've never built or supported musl, and since this is using the
in kernel Makefiles, etc, it isn't something I can patch or fix
outside of linux-yocto.

I'm just worried about the glibc cases, we can do follow up patches
later.

Bruce





Cheers,

Bruce



Cheers,

Richard








--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Khem Raj]

On 7/12/18 7:55 AM, Bruce Ashfield wrote:
> On 2018-07-12 9:53 AM, Richard Purdie wrote:
>> On Thu, 2018-07-12 at 09:49 -0400, Bruce Ashfield wrote:
>>> On 2018-07-10 5:41 PM, Richard Purdie wrote:
 On Tue, 2018-07-10 at 12:38 -0400, Bruce Ashfield wrote:
> On 07/10/2018 06:21 AM, Richard Purdie wrote:
>> On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:
>
> I'll try the other configs, but clearly I have something
> different in
> my x86-64 build.
>
> I can't run the self tests directly, but am copying the files
> onto my
> qemu session and running things myself:
>
> root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make
> ARCH=x86
> scripts prepare
> Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
> install
> libelf-dev, libelf-devel or elfutils-libelf-devel"
>  CHK scripts/mod/devicetable-offsets.h
>  SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
>  SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
>  SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
>  SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
>  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
>  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
>  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
>  HOSTCC  arch/x86/tools/relocs_32.o
>  HOSTCC  arch/x86/tools/relocs_64.o
>  HOSTCC  arch/x86/tools/relocs_common.o
>  HOSTLD  arch/x86/tools/relocs
>  CHK include/config/kernel.release
>  CHK include/generated/uapi/linux/version.h
>  CHK include/generated/utsrelease.h
> Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
> install
> libelf-dev, libelf-devel or elfutils-libelf-devel"
>
> root@qemux86-64:/tmp# make
> make -C /usr/src/kernel M=/tmp modules
> make[1]: Entering directory '/lib/modules/4.14.48-yocto-
> standard/build'
> Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
> install
> libelf-dev, libelf-devel or elfutils-libelf-devel"
>  Building modules, stage 2.
>  MODPOST 1 modules
> make[1]: Leaving directory '/lib/modules/4.14.48-yocto-
> standard/build'
> root@qemux86-64:/tmp# insmod hellomod.ko
> [  419.211616] hellomod: loading out-of-tree module taints
> kernel.
> [  419.212586] Hello world!
>
> This is on the core-image-kernel-dev image, which is the one that
> I created to test all this.
>
> What image is that using ? some sato sdk one ?

 We really need to find you a way to run these. The problem is the
 qemu
 graphics piece? You can't have a dummy vncserver or something to
 direct
 them at?

 The image is a core-image-sato-sdk...
>>>
>>> I was able to trigger the objtool issue with this image, looking at
>>> fixing it and the other arches now.
>>
>> Since its image related, is it a missing package dependency? Just
>> curious why its happening with some images but not your test ones...
> 
> It is. I have extra toolchain elements being installed in
> core-image-kernel-dev
> that hold things together (i.e. the kernel's objtool doesn't need to
> be built).
> 
> The obvious fix is to just add an RDEPENDS, but I'm being stubborn
> and trying to make it build on target, since the build of objtool from
> the kernel source is showing me some sort of include path issue that
> I'd like to sort out:
> 
>   CC /lib/modules/4.14.48-yocto-standard/build/tools/objtool/exec-cmd.o
> exec-cmd.c: In function 'get_pwd_cwd':
> exec-cmd.c:49:4: error: implicit declaration of function 'strlcpy'; did
> you mean 'strncpy'? [-Werror=implicit-function-declaration]
>     strlcpy(cwd, pwd, PATH_MAX);
>     ^~~
>     strncpy
> exec-cmd.c:49:4: error: nested extern declaration of 'strlcpy'
> [-Werror=nested-externs]
> cc1: all warnings being treated as errors
> 

Kernel has this function defined in tools/include/linux/string.h so

however it has it like this

#if defined(__GLIBC__) && !defined(__UCLIBC__)
extern size_t strlcpy(char *dest, const char *src, size_t size);
#endif

Which means it will not work for musl and we need to ensure that we
use -D_GNU_SOURCE for it to be included from standard musl headers


> Cheers,
> 
> Bruce
> 
>>
>> Cheers,
>>
>> Richard
>>
> 




signature.asc
Description: OpenPGP digital signature
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 2018-07-12 9:53 AM, Richard Purdie wrote:

On Thu, 2018-07-12 at 09:49 -0400, Bruce Ashfield wrote:

On 2018-07-10 5:41 PM, Richard Purdie wrote:

On Tue, 2018-07-10 at 12:38 -0400, Bruce Ashfield wrote:

On 07/10/2018 06:21 AM, Richard Purdie wrote:

On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:


I'll try the other configs, but clearly I have something
different in
my x86-64 build.

I can't run the self tests directly, but am copying the files
onto my
qemu session and running things myself:

root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make
ARCH=x86
scripts prepare
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
install
libelf-dev, libelf-devel or elfutils-libelf-devel"
 CHK scripts/mod/devicetable-offsets.h
 SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
 SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
 SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
 SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
 SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
 SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
 SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
 HOSTCC  arch/x86/tools/relocs_32.o
 HOSTCC  arch/x86/tools/relocs_64.o
 HOSTCC  arch/x86/tools/relocs_common.o
 HOSTLD  arch/x86/tools/relocs
 CHK include/config/kernel.release
 CHK include/generated/uapi/linux/version.h
 CHK include/generated/utsrelease.h
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
install
libelf-dev, libelf-devel or elfutils-libelf-devel"

root@qemux86-64:/tmp# make
make -C /usr/src/kernel M=/tmp modules
make[1]: Entering directory '/lib/modules/4.14.48-yocto-
standard/build'
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
install
libelf-dev, libelf-devel or elfutils-libelf-devel"
 Building modules, stage 2.
 MODPOST 1 modules
make[1]: Leaving directory '/lib/modules/4.14.48-yocto-
standard/build'
root@qemux86-64:/tmp# insmod hellomod.ko
[  419.211616] hellomod: loading out-of-tree module taints
kernel.
[  419.212586] Hello world!

This is on the core-image-kernel-dev image, which is the one that
I created to test all this.

What image is that using ? some sato sdk one ?


We really need to find you a way to run these. The problem is the
qemu
graphics piece? You can't have a dummy vncserver or something to
direct
them at?

The image is a core-image-sato-sdk...


I was able to trigger the objtool issue with this image, looking at
fixing it and the other arches now.


Since its image related, is it a missing package dependency? Just
curious why its happening with some images but not your test ones...


It is. I have extra toolchain elements being installed in 
core-image-kernel-dev

that hold things together (i.e. the kernel's objtool doesn't need to
be built).

The obvious fix is to just add an RDEPENDS, but I'm being stubborn
and trying to make it build on target, since the build of objtool from
the kernel source is showing me some sort of include path issue that
I'd like to sort out:

  CC 
/lib/modules/4.14.48-yocto-standard/build/tools/objtool/exec-cmd.o

exec-cmd.c: In function 'get_pwd_cwd':
exec-cmd.c:49:4: error: implicit declaration of function 'strlcpy'; did 
you mean 'strncpy'? [-Werror=implicit-function-declaration]

strlcpy(cwd, pwd, PATH_MAX);
^~~
strncpy
exec-cmd.c:49:4: error: nested extern declaration of 'strlcpy' 
[-Werror=nested-externs]

cc1: all warnings being treated as errors

Cheers,

Bruce



Cheers,

Richard



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Richard Purdie]

On Thu, 2018-07-12 at 09:49 -0400, Bruce Ashfield wrote:
> On 2018-07-10 5:41 PM, Richard Purdie wrote:
> > On Tue, 2018-07-10 at 12:38 -0400, Bruce Ashfield wrote:
> > > On 07/10/2018 06:21 AM, Richard Purdie wrote:
> > > > On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:
> > > 
> > > I'll try the other configs, but clearly I have something
> > > different in
> > > my x86-64 build.
> > > 
> > > I can't run the self tests directly, but am copying the files
> > > onto my
> > > qemu session and running things myself:
> > > 
> > > root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make
> > > ARCH=x86
> > > scripts prepare
> > > Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
> > > install
> > > libelf-dev, libelf-devel or elfutils-libelf-devel"
> > > CHK scripts/mod/devicetable-offsets.h
> > > SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
> > > SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
> > > SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
> > > SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
> > > SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
> > > SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
> > > SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
> > > HOSTCC  arch/x86/tools/relocs_32.o
> > > HOSTCC  arch/x86/tools/relocs_64.o
> > > HOSTCC  arch/x86/tools/relocs_common.o
> > > HOSTLD  arch/x86/tools/relocs
> > > CHK include/config/kernel.release
> > > CHK include/generated/uapi/linux/version.h
> > > CHK include/generated/utsrelease.h
> > > Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
> > > install
> > > libelf-dev, libelf-devel or elfutils-libelf-devel"
> > > 
> > > root@qemux86-64:/tmp# make
> > > make -C /usr/src/kernel M=/tmp modules
> > > make[1]: Entering directory '/lib/modules/4.14.48-yocto-
> > > standard/build'
> > > Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please
> > > install
> > > libelf-dev, libelf-devel or elfutils-libelf-devel"
> > > Building modules, stage 2.
> > > MODPOST 1 modules
> > > make[1]: Leaving directory '/lib/modules/4.14.48-yocto-
> > > standard/build'
> > > root@qemux86-64:/tmp# insmod hellomod.ko
> > > [  419.211616] hellomod: loading out-of-tree module taints
> > > kernel.
> > > [  419.212586] Hello world!
> > > 
> > > This is on the core-image-kernel-dev image, which is the one that
> > > I created to test all this.
> > > 
> > > What image is that using ? some sato sdk one ?
> > 
> > We really need to find you a way to run these. The problem is the
> > qemu
> > graphics piece? You can't have a dummy vncserver or something to
> > direct
> > them at?
> > 
> > The image is a core-image-sato-sdk...
> 
> I was able to trigger the objtool issue with this image, looking at
> fixing it and the other arches now.

Since its image related, is it a missing package dependency? Just
curious why its happening with some images but not your test ones...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 2018-07-10 5:41 PM, Richard Purdie wrote:

On Tue, 2018-07-10 at 12:38 -0400, Bruce Ashfield wrote:

On 07/10/2018 06:21 AM, Richard Purdie wrote:

On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:

I'll try the other configs, but clearly I have something different in
my x86-64 build.

I can't run the self tests directly, but am copying the files onto my
qemu session and running things myself:

root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make
ARCH=x86
scripts prepare
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install
libelf-dev, libelf-devel or elfutils-libelf-devel"
CHK scripts/mod/devicetable-offsets.h
SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
HOSTCC  arch/x86/tools/relocs_32.o
HOSTCC  arch/x86/tools/relocs_64.o
HOSTCC  arch/x86/tools/relocs_common.o
HOSTLD  arch/x86/tools/relocs
CHK include/config/kernel.release
CHK include/generated/uapi/linux/version.h
CHK include/generated/utsrelease.h
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install
libelf-dev, libelf-devel or elfutils-libelf-devel"

root@qemux86-64:/tmp# make
make -C /usr/src/kernel M=/tmp modules
make[1]: Entering directory '/lib/modules/4.14.48-yocto-
standard/build'
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install
libelf-dev, libelf-devel or elfutils-libelf-devel"
Building modules, stage 2.
MODPOST 1 modules
make[1]: Leaving directory '/lib/modules/4.14.48-yocto-
standard/build'
root@qemux86-64:/tmp# insmod hellomod.ko
[  419.211616] hellomod: loading out-of-tree module taints kernel.
[  419.212586] Hello world!

This is on the core-image-kernel-dev image, which is the one that
I created to test all this.

What image is that using ? some sato sdk one ?


We really need to find you a way to run these. The problem is the qemu
graphics piece? You can't have a dummy vncserver or something to direct
them at?

The image is a core-image-sato-sdk...


I was able to trigger the objtool issue with this image, looking at
fixing it and the other arches now.

Bruce



Cheers,

Richard



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 2018-07-10 5:41 PM, Richard Purdie wrote:

On Tue, 2018-07-10 at 12:38 -0400, Bruce Ashfield wrote:

On 07/10/2018 06:21 AM, Richard Purdie wrote:

On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:

I'll try the other configs, but clearly I have something different in
my x86-64 build.

I can't run the self tests directly, but am copying the files onto my
qemu session and running things myself:

root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make
ARCH=x86
scripts prepare
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install
libelf-dev, libelf-devel or elfutils-libelf-devel"
CHK scripts/mod/devicetable-offsets.h
SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
HOSTCC  arch/x86/tools/relocs_32.o
HOSTCC  arch/x86/tools/relocs_64.o
HOSTCC  arch/x86/tools/relocs_common.o
HOSTLD  arch/x86/tools/relocs
CHK include/config/kernel.release
CHK include/generated/uapi/linux/version.h
CHK include/generated/utsrelease.h
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install
libelf-dev, libelf-devel or elfutils-libelf-devel"

root@qemux86-64:/tmp# make
make -C /usr/src/kernel M=/tmp modules
make[1]: Entering directory '/lib/modules/4.14.48-yocto-
standard/build'
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install
libelf-dev, libelf-devel or elfutils-libelf-devel"
Building modules, stage 2.
MODPOST 1 modules
make[1]: Leaving directory '/lib/modules/4.14.48-yocto-
standard/build'
root@qemux86-64:/tmp# insmod hellomod.ko
[  419.211616] hellomod: loading out-of-tree module taints kernel.
[  419.212586] Hello world!

This is on the core-image-kernel-dev image, which is the one that
I created to test all this.

What image is that using ? some sato sdk one ?


We really need to find you a way to run these. The problem is the qemu
graphics piece? You can't have a dummy vncserver or something to direct
them at?


My config never seems to build and run the tests that I need, I had
it work once, and then never again. So I prefer to just build, boot
and run tests manually.

Regardless of the automation, I can easily do "make scripts prepare"
on the kernel-devsrc package, so that is extremely odd that it doesn't
work with that image. I'll try and add devsrc to a sato build and see
of objtool goes nuts.

I'm also unable to build lttng-modules for mips at the moment, so at
least mips is untestable for me.

Bruce



The image is a core-image-sato-sdk...

Cheers,

Richard



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Richard Purdie]

On Tue, 2018-07-10 at 12:38 -0400, Bruce Ashfield wrote:
> On 07/10/2018 06:21 AM, Richard Purdie wrote:
> > On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:
> I'll try the other configs, but clearly I have something different in
> my x86-64 build.
> 
> I can't run the self tests directly, but am copying the files onto my
> qemu session and running things myself:
> 
> root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make
> ARCH=x86 
> scripts prepare
> Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install 
> libelf-dev, libelf-devel or elfutils-libelf-devel"
>CHK scripts/mod/devicetable-offsets.h
>SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
>SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
>SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
>SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
>SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
>SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
>SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
>HOSTCC  arch/x86/tools/relocs_32.o
>HOSTCC  arch/x86/tools/relocs_64.o
>HOSTCC  arch/x86/tools/relocs_common.o
>HOSTLD  arch/x86/tools/relocs
>CHK include/config/kernel.release
>CHK include/generated/uapi/linux/version.h
>CHK include/generated/utsrelease.h
> Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install 
> libelf-dev, libelf-devel or elfutils-libelf-devel"
> 
> root@qemux86-64:/tmp# make
> make -C /usr/src/kernel M=/tmp modules
> make[1]: Entering directory '/lib/modules/4.14.48-yocto-
> standard/build'
> Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install 
> libelf-dev, libelf-devel or elfutils-libelf-devel"
>Building modules, stage 2.
>MODPOST 1 modules
> make[1]: Leaving directory '/lib/modules/4.14.48-yocto-
> standard/build'
> root@qemux86-64:/tmp# insmod hellomod.ko
> [  419.211616] hellomod: loading out-of-tree module taints kernel.
> [  419.212586] Hello world!
> 
> This is on the core-image-kernel-dev image, which is the one that
> I created to test all this.
> 
> What image is that using ? some sato sdk one ?

We really need to find you a way to run these. The problem is the qemu
graphics piece? You can't have a dummy vncserver or something to direct
them at?

The image is a core-image-sato-sdk...

Cheers,

Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 07/10/2018 06:21 AM, Richard Purdie wrote:

On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:

The existing kernel-devsrc package starts with a full copy of the
kernel
source and then starts to strip out elements that are not required.

This results in extra time (I/O) and extra space being taken up in
the
final package. The main purpose of the kernel-devsrc package has been
to
build modules against the running kernel, not to include a full copy
of
the source code for re-building the kernel. The end result was a
600M kernel-devsrc package.

This restructuring of the package uses an approach similar to other
distros, where the kernel-devsrc package is for building against the
running kernel and uses a curated set of copied infrastructure,
versus
a mass copy of the entire kernel.

The differences in this approach versus other is largely due to the
architecture support and the split build/source directory of the
kernel.

The result is a kernel-devsrc package of about 10M, which is capable
of running "make scripts" and compiling kernel modules against the
running kernel.

Along with the changes to the copying of the infrascture, we also
have the following changes:

  - a better/more explicit listing of dependencies for on-target
builds of "make scripts" or "make modules_prepare"

  - The kernel source is installed into /lib/modules//build
and a symlink created from /usr/src/kernel to the new location.
This aligns with the standard location for module support
code

  - There is also a symlink from /lib/modules//source ->
build
to reserve a spot for a new package that is simply the kernel
source. That package is not part of this update.

Signed-off-by: Bruce Ashfield 
---

v2: drop DEPENDS on perf. We no longer need to depend on perf since
the source
 is copied before modification.


It probably won't surprise you to know there were some issues with this
patch, sadly.

The main recurring issue (on all arches and poky+poky-lsb) is failure
of the kernelmodule.KernelModuleTest.test_kernel_module test. The exact
failure varies by arch, for x86-64:

https://autobuilder.yocto.io/builders/nightly-x86-64/builds/1154/steps/Running%20Sanity%20Tests/logs/stdio

|   CHK include/generated/utsrelease.h
|   DESCEND  objtool
| /lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.build:37: 
/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include: No such 
file or directory
| make[4]: *** No rule to make target 
'/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include'.  Stop.
| make[3]: *** [Makefile:43: 
/lib/modules/4.14.48-yocto-standard/build/tools/objtool/fixdep-in.o] Error 2
| make[2]: *** 
[/lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.include:4: 
fixdep] Error 2
| make[1]: *** [Makefile:62: objtool] Error 2
| make: *** [Makefile:1647: tools/objtool] Error 2


I'll try the other configs, but clearly I have something different in
my x86-64 build.

I can't run the self tests directly, but am copying the files onto my
qemu session and running things myself:

root@qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make ARCH=x86 
scripts prepare
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install 
libelf-dev, libelf-devel or elfutils-libelf-devel"

  CHK scripts/mod/devicetable-offsets.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
  HOSTCC  arch/x86/tools/relocs_32.o
  HOSTCC  arch/x86/tools/relocs_64.o
  HOSTCC  arch/x86/tools/relocs_common.o
  HOSTLD  arch/x86/tools/relocs
  CHK include/config/kernel.release
  CHK include/generated/uapi/linux/version.h
  CHK include/generated/utsrelease.h
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install 
libelf-dev, libelf-devel or elfutils-libelf-devel"


root@qemux86-64:/tmp# make
make -C /usr/src/kernel M=/tmp modules
make[1]: Entering directory '/lib/modules/4.14.48-yocto-standard/build'
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install 
libelf-dev, libelf-devel or elfutils-libelf-devel"

  Building modules, stage 2.
  MODPOST 1 modules
make[1]: Leaving directory '/lib/modules/4.14.48-yocto-standard/build'
root@qemux86-64:/tmp# insmod hellomod.ko
[  419.211616] hellomod: loading out-of-tree module taints kernel.
[  419.212586] Hello world!

This is on the core-image-kernel-dev image, which is the one that
I created to test all this.

What image is that using ? some sato sdk one ?

Bruce



For mips:

https://autobuilder.yocto.io/builders/nightly-mips/builds/1115

|   HOSTCC  scripts/sortextable
| make[1]: *** No rule to make target 'arch/mips/boot/tools/relocs_32.c', 
needed 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 07/10/2018 06:21 AM, Richard Purdie wrote:

On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:

The existing kernel-devsrc package starts with a full copy of the
kernel
source and then starts to strip out elements that are not required.

This results in extra time (I/O) and extra space being taken up in
the
final package. The main purpose of the kernel-devsrc package has been
to
build modules against the running kernel, not to include a full copy
of
the source code for re-building the kernel. The end result was a
600M kernel-devsrc package.

This restructuring of the package uses an approach similar to other
distros, where the kernel-devsrc package is for building against the
running kernel and uses a curated set of copied infrastructure,
versus
a mass copy of the entire kernel.

The differences in this approach versus other is largely due to the
architecture support and the split build/source directory of the
kernel.

The result is a kernel-devsrc package of about 10M, which is capable
of running "make scripts" and compiling kernel modules against the
running kernel.

Along with the changes to the copying of the infrascture, we also
have the following changes:

  - a better/more explicit listing of dependencies for on-target
builds of "make scripts" or "make modules_prepare"

  - The kernel source is installed into /lib/modules//build
and a symlink created from /usr/src/kernel to the new location.
This aligns with the standard location for module support
code

  - There is also a symlink from /lib/modules//source ->
build
to reserve a spot for a new package that is simply the kernel
source. That package is not part of this update.

Signed-off-by: Bruce Ashfield 
---

v2: drop DEPENDS on perf. We no longer need to depend on perf since
the source
 is copied before modification.


It probably won't surprise you to know there were some issues with this
patch, sadly.



heh. No problem. These are just things I didn't have the configs
to test.

Outside of the multilib issue, I can fix things up. For the multlib,
I have no idea .. since this:

  file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586 
conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64


is greek to me :D

I'll look at these one by one, and see if anyone with more bitbake
knowledge wants to help out with that last one.

Bruce



The main recurring issue (on all arches and poky+poky-lsb) is failure
of the kernelmodule.KernelModuleTest.test_kernel_module test. The exact
failure varies by arch, for x86-64:

https://autobuilder.yocto.io/builders/nightly-x86-64/builds/1154/steps/Running%20Sanity%20Tests/logs/stdio

|   CHK include/generated/utsrelease.h
|   DESCEND  objtool
| /lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.build:37: 
/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include: No such 
file or directory
| make[4]: *** No rule to make target 
'/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include'.  Stop.
| make[3]: *** [Makefile:43: 
/lib/modules/4.14.48-yocto-standard/build/tools/objtool/fixdep-in.o] Error 2
| make[2]: *** 
[/lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.include:4: 
fixdep] Error 2
| make[1]: *** [Makefile:62: objtool] Error 2
| make: *** [Makefile:1647: tools/objtool] Error 2

For mips:

https://autobuilder.yocto.io/builders/nightly-mips/builds/1115

|   HOSTCC  scripts/sortextable
| make[1]: *** No rule to make target 'arch/mips/boot/tools/relocs_32.c', 
needed by 'arch/mips/boot/tools/relocs_32.o'.  Stop.
| make: *** [arch/mips/Makefile:16: archscripts] Error 2

(same for mips64)

For arm:

https://autobuilder.yocto.io/builders/nightly-arm/builds/1187/

|   HOSTCC  scripts/sortextable
| make[1]: *** No rule to make target 'arch/arm/tools/syscall.tbl', needed by 
'arch/arm/include/generated/uapi/asm/unistd-common.h'.  Stop.
| make: *** [arch/arm/Makefile:319: archheaders] Error 2

For arm64:

https://autobuilder.yocto.io/builders/nightly-arm64/builds/1101/steps/Running%20Sanity%20Tests/logs/stdio

|   CHK include/generated/uapi/linux/version.h
|   CHK include/generated/utsrelease.h
| make[1]: *** No rule to make target 'arch/arm64/kernel/vdso/vdso.lds', needed 
by 'arch/arm64/kernel/vdso/vdso.so.dbg'.  Stop.
| make: *** [arch/arm64/Makefile:160: vdso_prepare] Error 2
  


There was also a failure in building the build-appliance image:

https://autobuilder.yocto.io/builders/build-appliance/builds/1110

and also a failure in one of the multilib builds, probably from
different dependencies pulled in by kernel-devsrc:

https://autobuilder.yocto.io/builders/nightly-multilib/builds/1139/steps/BuildImages_4/logs/stdio

Error: Transaction check error:
   file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586 conflicts with 
file from package lib64-libtool-2.4.6-r0.0.x86_64
   file /usr/bin/libtoolize from install of libtool-2.4.6-r0.0.i586 conflicts 
with file from package 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Richard Purdie]

On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:
> The existing kernel-devsrc package starts with a full copy of the
> kernel
> source and then starts to strip out elements that are not required.
> 
> This results in extra time (I/O) and extra space being taken up in
> the
> final package. The main purpose of the kernel-devsrc package has been
> to
> build modules against the running kernel, not to include a full copy
> of
> the source code for re-building the kernel. The end result was a
> 600M kernel-devsrc package.
> 
> This restructuring of the package uses an approach similar to other
> distros, where the kernel-devsrc package is for building against the
> running kernel and uses a curated set of copied infrastructure,
> versus
> a mass copy of the entire kernel.
> 
> The differences in this approach versus other is largely due to the
> architecture support and the split build/source directory of the
> kernel.
> 
> The result is a kernel-devsrc package of about 10M, which is capable
> of running "make scripts" and compiling kernel modules against the
> running kernel.
> 
> Along with the changes to the copying of the infrascture, we also
> have the following changes:
> 
>  - a better/more explicit listing of dependencies for on-target
>builds of "make scripts" or "make modules_prepare"
> 
>  - The kernel source is installed into /lib/modules//build
>and a symlink created from /usr/src/kernel to the new location.
>This aligns with the standard location for module support
>code
> 
>  - There is also a symlink from /lib/modules//source ->
> build
>to reserve a spot for a new package that is simply the kernel
>source. That package is not part of this update.
> 
> Signed-off-by: Bruce Ashfield 
> ---
> 
> v2: drop DEPENDS on perf. We no longer need to depend on perf since
> the source
> is copied before modification.

It probably won't surprise you to know there were some issues with this
patch, sadly.

The main recurring issue (on all arches and poky+poky-lsb) is failure
of the kernelmodule.KernelModuleTest.test_kernel_module test. The exact
failure varies by arch, for x86-64:

https://autobuilder.yocto.io/builders/nightly-x86-64/builds/1154/steps/Running%20Sanity%20Tests/logs/stdio

|   CHK include/generated/utsrelease.h
|   DESCEND  objtool
| /lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.build:37: 
/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include: No such 
file or directory
| make[4]: *** No rule to make target 
'/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include'.  Stop.
| make[3]: *** [Makefile:43: 
/lib/modules/4.14.48-yocto-standard/build/tools/objtool/fixdep-in.o] Error 2
| make[2]: *** 
[/lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.include:4: 
fixdep] Error 2
| make[1]: *** [Makefile:62: objtool] Error 2
| make: *** [Makefile:1647: tools/objtool] Error 2

For mips:

https://autobuilder.yocto.io/builders/nightly-mips/builds/1115

|   HOSTCC  scripts/sortextable
| make[1]: *** No rule to make target 'arch/mips/boot/tools/relocs_32.c', 
needed by 'arch/mips/boot/tools/relocs_32.o'.  Stop.
| make: *** [arch/mips/Makefile:16: archscripts] Error 2

(same for mips64)

For arm:

https://autobuilder.yocto.io/builders/nightly-arm/builds/1187/

|   HOSTCC  scripts/sortextable
| make[1]: *** No rule to make target 'arch/arm/tools/syscall.tbl', needed by 
'arch/arm/include/generated/uapi/asm/unistd-common.h'.  Stop.
| make: *** [arch/arm/Makefile:319: archheaders] Error 2

For arm64:

https://autobuilder.yocto.io/builders/nightly-arm64/builds/1101/steps/Running%20Sanity%20Tests/logs/stdio

|   CHK include/generated/uapi/linux/version.h
|   CHK include/generated/utsrelease.h
| make[1]: *** No rule to make target 'arch/arm64/kernel/vdso/vdso.lds', needed 
by 'arch/arm64/kernel/vdso/vdso.so.dbg'.  Stop.
| make: *** [arch/arm64/Makefile:160: vdso_prepare] Error 2
 

There was also a failure in building the build-appliance image:

https://autobuilder.yocto.io/builders/build-appliance/builds/1110

and also a failure in one of the multilib builds, probably from
different dependencies pulled in by kernel-devsrc:

https://autobuilder.yocto.io/builders/nightly-multilib/builds/1139/steps/BuildImages_4/logs/stdio

Error: Transaction check error:
  file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586 conflicts with 
file from package lib64-libtool-2.4.6-r0.0.x86_64
  file /usr/bin/libtoolize from install of libtool-2.4.6-r0.0.i586 conflicts 
with file from package lib64-libtool-2.4.6-r0.0.x86_64

I'll retest with this patch removed just so we can unblock the other patches.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

The existing kernel-devsrc package starts with a full copy of the kernel
source and then starts to strip out elements that are not required.

This results in extra time (I/O) and extra space being taken up in the
final package. The main purpose of the kernel-devsrc package has been to
build modules against the running kernel, not to include a full copy of
the source code for re-building the kernel. The end result was a
600M kernel-devsrc package.

This restructuring of the package uses an approach similar to other
distros, where the kernel-devsrc package is for building against the
running kernel and uses a curated set of copied infrastructure, versus
a mass copy of the entire kernel.

The differences in this approach versus other is largely due to the
architecture support and the split build/source directory of the
kernel.

The result is a kernel-devsrc package of about 10M, which is capable
of running "make scripts" and compiling kernel modules against the
running kernel.

Along with the changes to the copying of the infrascture, we also
have the following changes:

 - a better/more explicit listing of dependencies for on-target
   builds of "make scripts" or "make modules_prepare"

 - The kernel source is installed into /lib/modules//build
   and a symlink created from /usr/src/kernel to the new location.
   This aligns with the standard location for module support
   code

 - There is also a symlink from /lib/modules//source -> build
   to reserve a spot for a new package that is simply the kernel
   source. That package is not part of this update.

Signed-off-by: Bruce Ashfield 
---

v2: drop DEPENDS on perf. We no longer need to depend on perf since the source
is copied before modification.

 meta/recipes-kernel/linux/kernel-devsrc.bb | 199 ++---
 1 file changed, 152 insertions(+), 47 deletions(-)

diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb 
b/meta/recipes-kernel/linux/kernel-devsrc.bb
index 8bbfa23e4b86..1bfc011a906d 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -30,57 +30,162 @@ do_populate_sysroot[noexec] = "1"
 S = "${STAGING_KERNEL_DIR}"
 B = "${STAGING_KERNEL_BUILDDIR}"
 
-KERNEL_VERSION = "${@get_kernelversion_headers('${S}')}"
-
 PACKAGE_ARCH = "${MACHINE_ARCH}"
 
+KERNEL_BUILD_ROOT="/lib/modules/"
+
 do_install() {
-kerneldir=${D}${KERNEL_SRC_PATH}
-install -d $kerneldir
-
-#
-# Copy the staging dir source (and module build support) into the 
devsrc structure.
-# We can keep this copy simple and take everything, since a we'll 
clean up any build
-# artifacts afterwards, and the extra i/o is not significant
-#
-cd ${B}
-find . -type d -name '.git*' -prune -o -path '.debug' -prune -o -type 
f -print0 | cpio --null -pdlu $kerneldir
-cd ${S}
-   find . -type d -name '.git*' -prune -o -type d -name '.kernel-meta' 
-prune -o -type f -print0 | cpio --null -pdlu $kerneldir
-
-# Explicitly set KBUILD_OUTPUT to ensure that the image directory is 
cleaned and not
-# The main build artifacts. We clean the directory to avoid QA errors 
on mismatched
-# architecture (since scripts and helpers are native format).
-KBUILD_OUTPUT="$kerneldir"
-oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean 
_mrproper_scripts
-# make clean generates an absolute path symlink called "source"
-# in $kerneldir points to $kerneldir, which doesn't make any
-# sense, so remove it.
-if [ -L $kerneldir/source ]; then
-bbnote "Removing $kerneldir/source symlink"
-rm -f $kerneldir/source
-fi
-
-# As of Linux kernel version 3.0.1, the clean target removes
-# arch/powerpc/lib/crtsavres.o which is present in
-# KBUILD_LDFLAGS_MODULE, making it required to build external modules.
-if [ ${ARCH} = "powerpc" ]; then
-mkdir -p $kerneldir/arch/powerpc/lib/
-cp ${B}/arch/powerpc/lib/crtsavres.o 
$kerneldir/arch/powerpc/lib/crtsavres.o
-fi
-
-# Remove fixdep/objtool as they won't be target binaries
-for i in fixdep objtool; do
-if [ -e $kerneldir/tools/objtool/$i ]; then
-rm -rf $kerneldir/tools/objtool/$i
-fi
-done
-
-chown -R root:root ${D}
+kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
+install -d $kerneldir
+
+# create the directory structure
+rm -f $kerneldir/build
+rm -f $kerneldir/source
+mkdir -p $kerneldir/build
+
+# for compatibility with some older variants of this package, we
+# create  a /usr/src/kernel symlink to /lib/modules//source
+mkdir -p ${D}/usr/src
+(
+   cd ${D}/usr/src
+   ln -s ${KERNEL_BUILD_ROOT}${KERNEL_VERSION}/source kernel
+)
+
+# for on target purposes, we unify build and source
+(
+  

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Burton, Ross]

On 2 March 2018 at 19:29, Bruce Ashfield  wrote:

> Yah. I've already removed that in my updated WIP version of the patch,
> I'm just trying to complete
> the work and get the scripts building on target with musl .. I assume
> I can use the c library variable
> and select the right package ?
>

Until we have a virtual provider to magically pick the right one, this
would be the most correct:

RDEPENDS_append_libc-glibc = " glibc-utils"
RDEPENDS_append_libc-musl = " musl-utils"

I guess you could assume that any existing libc has a utils package and
just use ${TCLIBC}-utils though.

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On Fri, Mar 2, 2018 at 1:16 PM, Khem Raj  wrote:
> On Thu, Mar 1, 2018 at 1:37 AM, Burton, Ross  wrote:
>> Also fails on musl:
>>
>> ERROR: Nothing RPROVIDES 'glibc-utils' (but
>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/kernel-devsrc.bb
>> RDEPENDS on or otherwise requires it)
>>
>
> I sent patches to add musl-utils which should help this case. Perhaps
> adding glibc-utils to
> deps unconditionally is not right thing here. We can use libc-glibc
> and libc-musl overrides
> to select the right utils.

Yah. I've already removed that in my updated WIP version of the patch,
I'm just trying to complete
the work and get the scripts building on target with musl .. I assume
I can use the c library variable
and select the right package ?

Bruce

>
> Other idea is to create virtual/libc-utils but that seems like an overkill.
>
>> Ross
>>
>> On 1 March 2018 at 08:28, Maxin B. John  wrote:
>>>
>>> Hi,
>>>
>>> On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
>>> > The existing kernel-devsrc package starts with a full copy of the kernel
>>> > source and then starts to strip out elements that are not required.
>>> >
>>> > This results in extra time (I/O) and extra space being taken up in the
>>> > final package. The main purpose of the kernel-devsrc package has been to
>>> > build modules against the running kernel, not to include a full copy of
>>> > the source code for re-building the kernel. The end result was a
>>> > 600M kernel-devsrc package.
>>> >
>>> > This restructuring of the package uses an approach similar to other
>>> > distros, where the kernel-devsrc package is for building against the
>>> > running kernel and uses a curated set of copied infrastructure, versus
>>> > a mass copy of the entire kernel.
>>> >
>>> > The differences in this approach versus other is largely due to the
>>> > architecture support and the split build/source directory of the
>>> > kernel.
>>> >
>>> > The result is a kernel-devsrc package of about 10M, which is capable
>>> > of running "make scripts" and compiling kernel modules against the
>>> > running kernel.
>>> >
>>> > Along with the changes to the copying of the infrascture, we also
>>> > have the following changes:
>>> >
>>> >  - a better/more explicit listing of dependencies for on-target
>>> >builds of "make scripts" or "make modules_prepare"
>>> >
>>> >  - The kernel source is installed into /lib/modules//build
>>> >and a symlink created from /usr/src/kernel to the new location.
>>> >This aligns with the standard location for module support
>>> >code
>>> >
>>> >  - There is also a symlink from /lib/modules//source -> build
>>> >to reserve a spot for a new package that is simply the kernel
>>> >source. That package is not part of this update.
>>> >
>>> > [YOCTO #12535]
>>> >
>>> > Signed-off-by: Bruce Ashfield 
>>> > ---
>>> >
>>> >  v2: fix arch/arm devsrc builds
>>>
>>> Noticed one kernel-devsrc build failure in autobuilder (qemuarm) :
>>>
>>>
>>> https://autobuilder.yocto.io/builders/nightly-arm/builds/876/steps/BuildImages/logs/stdio
>>>
>>> ERROR: kernel-devsrc-1.0-r0 do_install: Function failed: do_install (log
>>> file is located at
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822)
>>> ERROR: Logfile of failure stored in:
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822
>>> Log data follows:
>>> | DEBUG: Executing python function extend_recipe_sysroot
>>> | NOTE: Direct dependencies are
>>> ['/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/binutils/binutils-cross_2.30.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-extended/xz/xz_5.2.3.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-support/lzop/lzop_1.03.bb:do_populate_sysroot',
>>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot',
>>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-runtime_7.3.bb:do_populate_sysroot',
>>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-cross_7.
>>>  3.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/bison/bison_3.0.4.bb:do_populate_sysroot',
>>> 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Khem Raj]

On Thu, Mar 1, 2018 at 1:37 AM, Burton, Ross  wrote:
> Also fails on musl:
>
> ERROR: Nothing RPROVIDES 'glibc-utils' (but
> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/kernel-devsrc.bb
> RDEPENDS on or otherwise requires it)
>

I sent patches to add musl-utils which should help this case. Perhaps
adding glibc-utils to
deps unconditionally is not right thing here. We can use libc-glibc
and libc-musl overrides
to select the right utils.

Other idea is to create virtual/libc-utils but that seems like an overkill.

> Ross
>
> On 1 March 2018 at 08:28, Maxin B. John  wrote:
>>
>> Hi,
>>
>> On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
>> > The existing kernel-devsrc package starts with a full copy of the kernel
>> > source and then starts to strip out elements that are not required.
>> >
>> > This results in extra time (I/O) and extra space being taken up in the
>> > final package. The main purpose of the kernel-devsrc package has been to
>> > build modules against the running kernel, not to include a full copy of
>> > the source code for re-building the kernel. The end result was a
>> > 600M kernel-devsrc package.
>> >
>> > This restructuring of the package uses an approach similar to other
>> > distros, where the kernel-devsrc package is for building against the
>> > running kernel and uses a curated set of copied infrastructure, versus
>> > a mass copy of the entire kernel.
>> >
>> > The differences in this approach versus other is largely due to the
>> > architecture support and the split build/source directory of the
>> > kernel.
>> >
>> > The result is a kernel-devsrc package of about 10M, which is capable
>> > of running "make scripts" and compiling kernel modules against the
>> > running kernel.
>> >
>> > Along with the changes to the copying of the infrascture, we also
>> > have the following changes:
>> >
>> >  - a better/more explicit listing of dependencies for on-target
>> >builds of "make scripts" or "make modules_prepare"
>> >
>> >  - The kernel source is installed into /lib/modules//build
>> >and a symlink created from /usr/src/kernel to the new location.
>> >This aligns with the standard location for module support
>> >code
>> >
>> >  - There is also a symlink from /lib/modules//source -> build
>> >to reserve a spot for a new package that is simply the kernel
>> >source. That package is not part of this update.
>> >
>> > [YOCTO #12535]
>> >
>> > Signed-off-by: Bruce Ashfield 
>> > ---
>> >
>> >  v2: fix arch/arm devsrc builds
>>
>> Noticed one kernel-devsrc build failure in autobuilder (qemuarm) :
>>
>>
>> https://autobuilder.yocto.io/builders/nightly-arm/builds/876/steps/BuildImages/logs/stdio
>>
>> ERROR: kernel-devsrc-1.0-r0 do_install: Function failed: do_install (log
>> file is located at
>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822)
>> ERROR: Logfile of failure stored in:
>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822
>> Log data follows:
>> | DEBUG: Executing python function extend_recipe_sysroot
>> | NOTE: Direct dependencies are
>> ['/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/binutils/binutils-cross_2.30.bb:do_populate_sysroot',
>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-extended/xz/xz_5.2.3.bb:do_populate_sysroot',
>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot',
>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-support/lzop/lzop_1.03.bb:do_populate_sysroot',
>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot',
>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-runtime_7.3.bb:do_populate_sysroot',
>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-cross_7.
>>  3.bb:do_populate_sysroot',
>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/bison/bison_3.0.4.bb:do_populate_sysroot',
>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/pseudo/pseudo_1.8.2.bb:do_populate_sysroot',
>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-core/glibc/glibc_2.26.bb:do_populate_sysroot',
>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/kmod/kmod-native_git.bb:do_populate_sysroot',
>> 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Khem Raj]

On Fri, Mar 2, 2018 at 6:26 AM, Burton, Ross  wrote:
> Alpine Linux has hit this and they just build a custom getconf binary as
> part of their musl build;
>
> https://git.alpinelinux.org/cgit/aports/tree/main/musl
>
> (also getent)
>

I have long thought for adding musl-utils recipes for compat issues

we should probably enhance bsd-headers to include
https://github.com/boltlinux/musl-utils
and also rename the recipe to be called glibc-compat or musl-utils

> Ross
>
> On 2 March 2018 at 14:23, Bruce Ashfield 
> wrote:
>>
>> I worked through some problems to test with musl:
>>
>>   - uprev'd linux-libc-headers --> refresh / change of the musl ethhdr
>> patch
>>
>>   - worked around the binutils link issue and libsystemd
>>
>> So I was able to assemble kernel-devsrc on an image with musl, and now
>> I'm back to the problem that lead me to add glibc-utils to the RDEPENDS
>> of kernel-devsrc:
>>
>> -
>>
>> root@qemux86-64:/lib/modules/4.14.19-yocto-standard/build# make ARCH=x86
>> scripts
>> make: getconf: Command not found
>> make: getconf: Command not found
>> make: getconf: Command not found
>>   HOSTCC  scripts/basic/fixdep
>>   HOSTCC  scripts/basic/bin2c
>> make[1]: getconf: Command not found
>> make[1]: getconf: Command not found
>> make[1]: getconf: Command not found
>>   HOSTCC  scripts/kconfig/conf.o
>>   SHIPPED scripts/kconfig/zconf.tab.c
>>   SHIPPED scripts/kconfig/zconf.lex.c
>>   HOSTCC  scripts/kconfig/zconf.tab.o
>>   HOSTLD  scripts/kconfig/conf
>> scripts/kconfig/conf  --silentoldconfig Kconfig
>>   WRAParch/x86/include/generated/asm/clkdev.h
>>   WRAParch/x86/include/generated/asm/dma-contiguous.h
>>   WRAParch/x86/include/generated/asm/early_ioremap.h
>>   WRAParch/x86/include/generated/asm/mcs_spinlock.h
>>   WRAParch/x86/include/generated/asm/mm-arch-hooks.h
>>   CC  scripts/mod/empty.o
>>   HOSTCC  scripts/mod/mk_elfconfig
>>   MKELF   scripts/mod/elfconfig.h
>>   HOSTCC  scripts/mod/modpost.o
>>   CC  scripts/mod/devicetable-offsets.s
>>   CHK scripts/mod/devicetable-offsets.h
>>   UPD scripts/mod/devicetable-offsets.h
>>   HOSTCC  scripts/mod/file2alias.o
>>   HOSTCC  scripts/mod/sumversion.o
>>   HOSTLD  scripts/mod/modpost
>>   HOSTCC  scripts/kallsyms
>>   HOSTCC  scripts/conmakehash
>>   HOSTCC  scripts/recordmcount
>>   HOSTCC  scripts/sortextable
>> root@qemux86-64:/lib/modules/4.14.19-yocto-standard/build#
>>
>> 
>>
>> I've added Khem to the cc .. Khem: what in the muslc world would
>> provide 'getconf' ? I didn't see it when glancing through the packaging.
>>
>> Bruce
>>
>> On 2018-03-01 4:37 AM, Burton, Ross wrote:
>>>
>>> Also fails on musl:
>>>
>>> ERROR: Nothing RPROVIDES 'glibc-utils' (but
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/kernel-devsrc.bb
>>>  RDEPENDS on or otherwise requires it)
>>>
>>> Ross
>>>
>>>
>>> On 1 March 2018 at 08:28, Maxin B. John >> > wrote:
>>>
>>> Hi,
>>>
>>> On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
>>>  > The existing kernel-devsrc package starts with a full copy of the
>>> kernel
>>>  > source and then starts to strip out elements that are not
>>> required.
>>>  >
>>>  > This results in extra time (I/O) and extra space being taken up
>>> in the
>>>  > final package. The main purpose of the kernel-devsrc package has
>>> been to
>>>  > build modules against the running kernel, not to include a full
>>> copy of
>>>  > the source code for re-building the kernel. The end result was a
>>>  > 600M kernel-devsrc package.
>>>  >
>>>  > This restructuring of the package uses an approach similar to
>>> other
>>>  > distros, where the kernel-devsrc package is for building against
>>> the
>>>  > running kernel and uses a curated set of copied infrastructure,
>>> versus
>>>  > a mass copy of the entire kernel.
>>>  >
>>>  > The differences in this approach versus other is largely due to
>>> the
>>>  > architecture support and the split build/source directory of the
>>>  > kernel.
>>>  >
>>>  > The result is a kernel-devsrc package of about 10M, which is
>>> capable
>>>  > of running "make scripts" and compiling kernel modules against the
>>>  > running kernel.
>>>  >
>>>  > Along with the changes to the copying of the infrascture, we also
>>>  > have the following changes:
>>>  >
>>>  >  - a better/more explicit listing of dependencies for on-target
>>>  >builds of "make scripts" or "make modules_prepare"
>>>  >
>>>  >  - The kernel source is installed into
>>> /lib/modules//build
>>>  >and a symlink created from /usr/src/kernel to the new location.
>>>  >This aligns with the standard location for module support
>>>  >

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Burton, Ross]

Alpine Linux has hit this and they just build a custom getconf binary as
part of their musl build;

https://git.alpinelinux.org/cgit/aports/tree/main/musl

(also getent)

Ross

On 2 March 2018 at 14:23, Bruce Ashfield 
wrote:

> I worked through some problems to test with musl:
>
>   - uprev'd linux-libc-headers --> refresh / change of the musl ethhdr
> patch
>
>   - worked around the binutils link issue and libsystemd
>
> So I was able to assemble kernel-devsrc on an image with musl, and now
> I'm back to the problem that lead me to add glibc-utils to the RDEPENDS
> of kernel-devsrc:
>
> -
>
> root@qemux86-64:/lib/modules/4.14.19-yocto-standard/build# make ARCH=x86
> scripts
> make: getconf: Command not found
> make: getconf: Command not found
> make: getconf: Command not found
>   HOSTCC  scripts/basic/fixdep
>   HOSTCC  scripts/basic/bin2c
> make[1]: getconf: Command not found
> make[1]: getconf: Command not found
> make[1]: getconf: Command not found
>   HOSTCC  scripts/kconfig/conf.o
>   SHIPPED scripts/kconfig/zconf.tab.c
>   SHIPPED scripts/kconfig/zconf.lex.c
>   HOSTCC  scripts/kconfig/zconf.tab.o
>   HOSTLD  scripts/kconfig/conf
> scripts/kconfig/conf  --silentoldconfig Kconfig
>   WRAParch/x86/include/generated/asm/clkdev.h
>   WRAParch/x86/include/generated/asm/dma-contiguous.h
>   WRAParch/x86/include/generated/asm/early_ioremap.h
>   WRAParch/x86/include/generated/asm/mcs_spinlock.h
>   WRAParch/x86/include/generated/asm/mm-arch-hooks.h
>   CC  scripts/mod/empty.o
>   HOSTCC  scripts/mod/mk_elfconfig
>   MKELF   scripts/mod/elfconfig.h
>   HOSTCC  scripts/mod/modpost.o
>   CC  scripts/mod/devicetable-offsets.s
>   CHK scripts/mod/devicetable-offsets.h
>   UPD scripts/mod/devicetable-offsets.h
>   HOSTCC  scripts/mod/file2alias.o
>   HOSTCC  scripts/mod/sumversion.o
>   HOSTLD  scripts/mod/modpost
>   HOSTCC  scripts/kallsyms
>   HOSTCC  scripts/conmakehash
>   HOSTCC  scripts/recordmcount
>   HOSTCC  scripts/sortextable
> root@qemux86-64:/lib/modules/4.14.19-yocto-standard/build#
>
> 
>
> I've added Khem to the cc .. Khem: what in the muslc world would
> provide 'getconf' ? I didn't see it when glancing through the packaging.
>
> Bruce
>
> On 2018-03-01 4:37 AM, Burton, Ross wrote:
>
>> Also fails on musl:
>>
>> ERROR: Nothing RPROVIDES 'glibc-utils' (but /home/pokybuild/yocto-autobuil
>> der/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/
>> kernel-devsrc.bb  RDEPENDS on or otherwise
>> requires it)
>>
>> Ross
>>
>>
>> On 1 March 2018 at 08:28, Maxin B. John  maxin.j...@intel.com>> wrote:
>>
>> Hi,
>>
>> On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
>>  > The existing kernel-devsrc package starts with a full copy of the
>> kernel
>>  > source and then starts to strip out elements that are not required.
>>  >
>>  > This results in extra time (I/O) and extra space being taken up
>> in the
>>  > final package. The main purpose of the kernel-devsrc package has
>> been to
>>  > build modules against the running kernel, not to include a full
>> copy of
>>  > the source code for re-building the kernel. The end result was a
>>  > 600M kernel-devsrc package.
>>  >
>>  > This restructuring of the package uses an approach similar to other
>>  > distros, where the kernel-devsrc package is for building against
>> the
>>  > running kernel and uses a curated set of copied infrastructure,
>> versus
>>  > a mass copy of the entire kernel.
>>  >
>>  > The differences in this approach versus other is largely due to the
>>  > architecture support and the split build/source directory of the
>>  > kernel.
>>  >
>>  > The result is a kernel-devsrc package of about 10M, which is
>> capable
>>  > of running "make scripts" and compiling kernel modules against the
>>  > running kernel.
>>  >
>>  > Along with the changes to the copying of the infrascture, we also
>>  > have the following changes:
>>  >
>>  >  - a better/more explicit listing of dependencies for on-target
>>  >builds of "make scripts" or "make modules_prepare"
>>  >
>>  >  - The kernel source is installed into /lib/modules//build
>>  >and a symlink created from /usr/src/kernel to the new location.
>>  >This aligns with the standard location for module support
>>  >code
>>  >
>>  >  - There is also a symlink from /lib/modules//source ->
>> build
>>  >to reserve a spot for a new package that is simply the kernel
>>  >source. That package is not part of this update.
>>  >
>>  > [YOCTO #12535]
>>  >
>>  > Signed-off-by: Bruce Ashfield > >
>>
>>  > ---
>>  >
>>  >  v2: fix arch/arm devsrc builds

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

I worked through some problems to test with musl:

  - uprev'd linux-libc-headers --> refresh / change of the musl ethhdr
patch

  - worked around the binutils link issue and libsystemd

So I was able to assemble kernel-devsrc on an image with musl, and now
I'm back to the problem that lead me to add glibc-utils to the RDEPENDS
of kernel-devsrc:

-

root@qemux86-64:/lib/modules/4.14.19-yocto-standard/build# make ARCH=x86 
scripts

make: getconf: Command not found
make: getconf: Command not found
make: getconf: Command not found
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/basic/bin2c
make[1]: getconf: Command not found
make[1]: getconf: Command not found
make[1]: getconf: Command not found
  HOSTCC  scripts/kconfig/conf.o
  SHIPPED scripts/kconfig/zconf.tab.c
  SHIPPED scripts/kconfig/zconf.lex.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf
scripts/kconfig/conf  --silentoldconfig Kconfig
  WRAParch/x86/include/generated/asm/clkdev.h
  WRAParch/x86/include/generated/asm/dma-contiguous.h
  WRAParch/x86/include/generated/asm/early_ioremap.h
  WRAParch/x86/include/generated/asm/mcs_spinlock.h
  WRAParch/x86/include/generated/asm/mm-arch-hooks.h
  CC  scripts/mod/empty.o
  HOSTCC  scripts/mod/mk_elfconfig
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  CC  scripts/mod/devicetable-offsets.s
  CHK scripts/mod/devicetable-offsets.h
  UPD scripts/mod/devicetable-offsets.h
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTLD  scripts/mod/modpost
  HOSTCC  scripts/kallsyms
  HOSTCC  scripts/conmakehash
  HOSTCC  scripts/recordmcount
  HOSTCC  scripts/sortextable
root@qemux86-64:/lib/modules/4.14.19-yocto-standard/build#



I've added Khem to the cc .. Khem: what in the muslc world would
provide 'getconf' ? I didn't see it when glancing through the packaging.

Bruce

On 2018-03-01 4:37 AM, Burton, Ross wrote:

Also fails on musl:

ERROR: Nothing RPROVIDES 'glibc-utils' (but 
/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/kernel-devsrc.bb 
 RDEPENDS on or otherwise requires it)


Ross

On 1 March 2018 at 08:28, Maxin B. John > wrote:


Hi,

On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
 > The existing kernel-devsrc package starts with a full copy of the
kernel
 > source and then starts to strip out elements that are not required.
 >
 > This results in extra time (I/O) and extra space being taken up
in the
 > final package. The main purpose of the kernel-devsrc package has
been to
 > build modules against the running kernel, not to include a full
copy of
 > the source code for re-building the kernel. The end result was a
 > 600M kernel-devsrc package.
 >
 > This restructuring of the package uses an approach similar to other
 > distros, where the kernel-devsrc package is for building against the
 > running kernel and uses a curated set of copied infrastructure,
versus
 > a mass copy of the entire kernel.
 >
 > The differences in this approach versus other is largely due to the
 > architecture support and the split build/source directory of the
 > kernel.
 >
 > The result is a kernel-devsrc package of about 10M, which is capable
 > of running "make scripts" and compiling kernel modules against the
 > running kernel.
 >
 > Along with the changes to the copying of the infrascture, we also
 > have the following changes:
 >
 >  - a better/more explicit listing of dependencies for on-target
 >    builds of "make scripts" or "make modules_prepare"
 >
 >  - The kernel source is installed into /lib/modules//build
 >    and a symlink created from /usr/src/kernel to the new location.
 >    This aligns with the standard location for module support
 >    code
 >
 >  - There is also a symlink from /lib/modules//source ->
build
 >    to reserve a spot for a new package that is simply the kernel
 >    source. That package is not part of this update.
 >
 > [YOCTO #12535]
 >
 > Signed-off-by: Bruce Ashfield >
 > ---
 >
 >  v2: fix arch/arm devsrc builds

Noticed one kernel-devsrc build failure in autobuilder (qemuarm) :


https://autobuilder.yocto.io/builders/nightly-arm/builds/876/steps/BuildImages/logs/stdio



ERROR: kernel-devsrc-1.0-r0 do_install: Function failed: do_install
(log file is located at

/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822)
ERROR: Logfile of 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Burton, Ross]

Ah ok, I misunderstood.

Ross

On 1 March 2018 at 14:45, Bruce Ashfield 
wrote:

> On 2018-03-01 9:22 AM, Burton, Ross wrote:
>
>> Swore I picked the v2 patch for the second run, I've fired a mini test
>> build to see.
>>
>
> The other failures reported were valid and not addressed by v2. So only
> 32 bit ARM would have been fixed (packaging wise), the rest were ok.
>
> I build and deployed kernel-devsrc for ARM multiple times, so if it does
> actually fail again .. I'm *very* interested to hear.
>
> Bruce
>
>
>> Ross
>>
>> On 1 March 2018 at 13:59, Bruce Ashfield > > wrote:
>>
>> On 2018-03-01 5:58 AM, Maxin B. John wrote:
>>
>>
>> Also fails on musl:
>>
>> ERROR: Nothing RPROVIDES 'glibc-utils' (but
>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/
>> build/meta/recipes-kernel/linux/kernel-devsrc.bb
>>  RDEPENDS on or otherwise requires
>> it)
>>
>> Ross
>>
>> On 1 March 2018 at 08:28, Maxin B. John
>> > wrote:
>>
>>
>>>On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield
>> wrote:
>>>> The existing kernel-devsrc package starts with a full copy
>> of the kernel
>>>> source and then starts to strip out elements that are not
>> required.
>>>>
>>>> This results in extra time (I/O) and extra space being
>> taken up in the
>>>> final package. The main purpose of the kernel-devsrc
>> package has been to
>>>> build modules against the running kernel, not to include a
>> full copy of
>>>> the source code for re-building the kernel. The end result
>> was a
>>>> 600M kernel-devsrc package.
>>
>>Other failures are listed below:
>>  arm64:
>> https://autobuilder.yocto.io/builders/nightly-arm64/builds/7
>> 90/steps/Running%20Sanity%20Tests/logs/stdio
>> > 790/steps/Running%20Sanity%20Tests/logs/stdio>
>>
>>RESULTS - kernelmodule.KernelModuleTest.test_kernel_module -
>> Testcase 1541: FAILED
>>
>> HOSTCC  scripts/sortextable
>>scripts/sortextable.c:31:10: fatal error:
>> tools/be_byteshift.h: No such file or directory
>> #include 
>> ^~
>> compilation terminated.
>> make[1]: *** [scripts/Makefile.host:102: scripts/sortextable]
>> Error 1
>> make: *** [Makefile:557: scripts] Error 2
>> 
>> --
>>
>> Similar on qemuarm - poky-lsb :
>>
>> https://autobuilder.yocto.io/builders/nightly-arm-lsb/builds
>> /812/steps/Running%20Sanity%20Tests/logs/stdio
>> > s/812/steps/Running%20Sanity%20Tests/logs/stdio>
>>
>> HOSTCC  scripts/recordmcount
>> HOSTCC  scripts/sortextable
>> scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h:
>> No such file or directory
>>   #include 
>> ^~
>> compilation terminated.
>> make[1]: *** [scripts/Makefile.host:102:
>> scripts/sortextable] Error 1
>> make: *** [Makefile:557: scripts] Error 2
>> 
>> -
>>
>> Also on PowerPC:
>>
>> https://autobuilder.yocto.io/builders/nightly-ppc-lsb/builds
>> /814/steps/Running%20Sanity%20Tests/logs/stdio
>> > s/814/steps/Running%20Sanity%20Tests/logs/stdio>
>>
>> scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h:
>> No such file or directory
>>#include 
>>  ^~
>> compilation terminated.
>> make[1]: *** [scripts/Makefile.host:102: scripts/sortextable]
>> Error 1
>> make: *** [Makefile:557: scripts] Error 2
>>
>>
>> I have all these fixed now.
>>
>> Will have a look at musl next.
>>
>> Cheers,
>>
>> Bruce
>>
>>
>> Best Regards,
>> Maxin
>>
>>
>>
>>
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 2018-03-01 9:22 AM, Burton, Ross wrote:
Swore I picked the v2 patch for the second run, I've fired a mini test 
build to see.


The other failures reported were valid and not addressed by v2. So only
32 bit ARM would have been fixed (packaging wise), the rest were ok.

I build and deployed kernel-devsrc for ARM multiple times, so if it does
actually fail again .. I'm *very* interested to hear.

Bruce



Ross

On 1 March 2018 at 13:59, Bruce Ashfield > wrote:


On 2018-03-01 5:58 AM, Maxin B. John wrote:


Also fails on musl:

ERROR: Nothing RPROVIDES 'glibc-utils' (but

/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/kernel-devsrc.bb
 RDEPENDS on or otherwise requires it)

Ross

On 1 March 2018 at 08:28, Maxin B. John
> wrote:


   >On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
   >> The existing kernel-devsrc package starts with a full copy
of the kernel
   >> source and then starts to strip out elements that are not
required.
   >>
   >> This results in extra time (I/O) and extra space being
taken up in the
   >> final package. The main purpose of the kernel-devsrc
package has been to
   >> build modules against the running kernel, not to include a
full copy of
   >> the source code for re-building the kernel. The end result
was a
   >> 600M kernel-devsrc package.

   Other failures are listed below:
     arm64:

https://autobuilder.yocto.io/builders/nightly-arm64/builds/790/steps/Running%20Sanity%20Tests/logs/stdio



   RESULTS - kernelmodule.KernelModuleTest.test_kernel_module -
Testcase 1541: FAILED

HOSTCC  scripts/sortextable
   scripts/sortextable.c:31:10: fatal error:
tools/be_byteshift.h: No such file or directory
#include 
    ^~
compilation terminated.
make[1]: *** [scripts/Makefile.host:102: scripts/sortextable]
Error 1
make: *** [Makefile:557: scripts] Error 2
--

Similar on qemuarm - poky-lsb :


https://autobuilder.yocto.io/builders/nightly-arm-lsb/builds/812/steps/Running%20Sanity%20Tests/logs/stdio



HOSTCC  scripts/recordmcount
HOSTCC  scripts/sortextable
scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h:
No such file or directory
      #include 
                ^~
                compilation terminated.
                make[1]: *** [scripts/Makefile.host:102:
scripts/sortextable] Error 1
                make: *** [Makefile:557: scripts] Error 2

-

Also on PowerPC:


https://autobuilder.yocto.io/builders/nightly-ppc-lsb/builds/814/steps/Running%20Sanity%20Tests/logs/stdio



scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h:
No such file or directory
   #include 
             ^~
compilation terminated.
make[1]: *** [scripts/Makefile.host:102: scripts/sortextable]
Error 1
make: *** [Makefile:557: scripts] Error 2


I have all these fixed now.

Will have a look at musl next.

Cheers,

Bruce


Best Regards,
Maxin





--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Burton, Ross]

Swore I picked the v2 patch for the second run, I've fired a mini test
build to see.

Ross

On 1 March 2018 at 13:59, Bruce Ashfield 
wrote:

> On 2018-03-01 5:58 AM, Maxin B. John wrote:
>
>>
>> Also fails on musl:
>>>
>>> ERROR: Nothing RPROVIDES 'glibc-utils' (but
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/
>>> build/meta/recipes-kernel/linux/kernel-devsrc.bb RDEPENDS on or
>>> otherwise requires it)
>>>
>>> Ross
>>>
>>> On 1 March 2018 at 08:28, Maxin B. John  wrote:
>>>
>>
>>   >On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
>>   >> The existing kernel-devsrc package starts with a full copy of the
>> kernel
>>   >> source and then starts to strip out elements that are not required.
>>   >>
>>   >> This results in extra time (I/O) and extra space being taken up in
>> the
>>   >> final package. The main purpose of the kernel-devsrc package has
>> been to
>>   >> build modules against the running kernel, not to include a full copy
>> of
>>   >> the source code for re-building the kernel. The end result was a
>>   >> 600M kernel-devsrc package.
>>
>>   Other failures are listed below:
>> arm64:
>>   https://autobuilder.yocto.io/builders/nightly-arm64/builds/7
>> 90/steps/Running%20Sanity%20Tests/logs/stdio
>>
>>   RESULTS - kernelmodule.KernelModuleTest.test_kernel_module - Testcase
>> 1541: FAILED
>>
>> HOSTCC  scripts/sortextable
>>   scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such
>> file or directory
>> #include 
>>^~
>> compilation terminated.
>> make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] Error 1
>> make: *** [Makefile:557: scripts] Error 2
>> --
>>
>> Similar on qemuarm - poky-lsb :
>>
>> https://autobuilder.yocto.io/builders/nightly-arm-lsb/builds
>> /812/steps/Running%20Sanity%20Tests/logs/stdio
>>
>> HOSTCC  scripts/recordmcount
>> HOSTCC  scripts/sortextable
>> scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such
>> file or directory
>>  #include 
>>^~
>>compilation terminated.
>>make[1]: *** [scripts/Makefile.host:102:
>> scripts/sortextable] Error 1
>>make: *** [Makefile:557: scripts] Error 2
>> -
>>
>> Also on PowerPC:
>>
>> https://autobuilder.yocto.io/builders/nightly-ppc-lsb/builds
>> /814/steps/Running%20Sanity%20Tests/logs/stdio
>>
>> scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such
>> file or directory
>>   #include 
>> ^~
>> compilation terminated.
>> make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] Error 1
>> make: *** [Makefile:557: scripts] Error 2
>>
>
> I have all these fixed now.
>
> Will have a look at musl next.
>
> Cheers,
>
> Bruce
>
>
>> Best Regards,
>> Maxin
>>
>>
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 2018-03-01 5:58 AM, Maxin B. John wrote:



Also fails on musl:

ERROR: Nothing RPROVIDES 'glibc-utils' (but 
/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/kernel-devsrc.bb
 RDEPENDS on or otherwise requires it)

Ross

On 1 March 2018 at 08:28, Maxin B. John  wrote:


  >On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
  >> The existing kernel-devsrc package starts with a full copy of the kernel
  >> source and then starts to strip out elements that are not required.
  >>
  >> This results in extra time (I/O) and extra space being taken up in the
  >> final package. The main purpose of the kernel-devsrc package has been to
  >> build modules against the running kernel, not to include a full copy of
  >> the source code for re-building the kernel. The end result was a
  >> 600M kernel-devsrc package.

  Other failures are listed below:
  
  arm64:

  
https://autobuilder.yocto.io/builders/nightly-arm64/builds/790/steps/Running%20Sanity%20Tests/logs/stdio

  RESULTS - kernelmodule.KernelModuleTest.test_kernel_module - Testcase 1541: 
FAILED

HOSTCC  scripts/sortextable
  scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such file 
or directory
#include 
   ^~
compilation terminated.
make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] Error 1
make: *** [Makefile:557: scripts] Error 2
--

Similar on qemuarm - poky-lsb :

https://autobuilder.yocto.io/builders/nightly-arm-lsb/builds/812/steps/Running%20Sanity%20Tests/logs/stdio

HOSTCC  scripts/recordmcount
HOSTCC  scripts/sortextable
scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such file or 
directory
 #include 
   ^~
   compilation terminated.
   make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] 
Error 1
   make: *** [Makefile:557: scripts] Error 2
-

Also on PowerPC:

https://autobuilder.yocto.io/builders/nightly-ppc-lsb/builds/814/steps/Running%20Sanity%20Tests/logs/stdio

scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such file or 
directory
  #include 
^~
compilation terminated.
make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] Error 1
make: *** [Makefile:557: scripts] Error 2


I have all these fixed now.

Will have a look at musl next.

Cheers,

Bruce



Best Regards,
Maxin



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

Interesting. I'll look at the errors and send a v3 next week.

I unfortunately have no capability to run the oe-selftests on my
builder, so I can only fix these
by inspection.

Cheers,

Bruce

On Thu, Mar 1, 2018 at 5:58 AM, Maxin B. John  wrote:
>
>>Also fails on musl:
>>
>>ERROR: Nothing RPROVIDES 'glibc-utils' (but 
>>/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/kernel-devsrc.bb
>> RDEPENDS on or otherwise requires it)
>>
>>Ross
>>
>>On 1 March 2018 at 08:28, Maxin B. John  wrote:
>
>  >On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
>  >> The existing kernel-devsrc package starts with a full copy of the kernel
>  >> source and then starts to strip out elements that are not required.
>  >>
>  >> This results in extra time (I/O) and extra space being taken up in the
>  >> final package. The main purpose of the kernel-devsrc package has been to
>  >> build modules against the running kernel, not to include a full copy of
>  >> the source code for re-building the kernel. The end result was a
>  >> 600M kernel-devsrc package.
>
>  Other failures are listed below:
>
>  arm64:
>  
> https://autobuilder.yocto.io/builders/nightly-arm64/builds/790/steps/Running%20Sanity%20Tests/logs/stdio
>
>  RESULTS - kernelmodule.KernelModuleTest.test_kernel_module - Testcase 1541: 
> FAILED
>
> HOSTCC  scripts/sortextable
>  scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such file 
> or directory
> #include 
>   ^~
> compilation terminated.
> make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] Error 1
> make: *** [Makefile:557: scripts] Error 2
> --
>
> Similar on qemuarm - poky-lsb :
>
> https://autobuilder.yocto.io/builders/nightly-arm-lsb/builds/812/steps/Running%20Sanity%20Tests/logs/stdio
>
> HOSTCC  scripts/recordmcount
> HOSTCC  scripts/sortextable
> scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such file 
> or directory
> #include 
>   ^~
>   compilation terminated.
>   make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] 
> Error 1
>   make: *** [Makefile:557: scripts] Error 2
> -
>
> Also on PowerPC:
>
> https://autobuilder.yocto.io/builders/nightly-ppc-lsb/builds/814/steps/Running%20Sanity%20Tests/logs/stdio
>
> scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such file 
> or directory
>  #include 
>^~
> compilation terminated.
> make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] Error 1
> make: *** [Makefile:557: scripts] Error 2
>
> Best Regards,
> Maxin
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On Thu, Mar 1, 2018 at 3:28 AM, Maxin B. John  wrote:
> Hi,
>
> On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
>> The existing kernel-devsrc package starts with a full copy of the kernel
>> source and then starts to strip out elements that are not required.
>>
>> This results in extra time (I/O) and extra space being taken up in the
>> final package. The main purpose of the kernel-devsrc package has been to
>> build modules against the running kernel, not to include a full copy of
>> the source code for re-building the kernel. The end result was a
>> 600M kernel-devsrc package.
>>
>> This restructuring of the package uses an approach similar to other
>> distros, where the kernel-devsrc package is for building against the
>> running kernel and uses a curated set of copied infrastructure, versus
>> a mass copy of the entire kernel.
>>
>> The differences in this approach versus other is largely due to the
>> architecture support and the split build/source directory of the
>> kernel.
>>
>> The result is a kernel-devsrc package of about 10M, which is capable
>> of running "make scripts" and compiling kernel modules against the
>> running kernel.
>>
>> Along with the changes to the copying of the infrascture, we also
>> have the following changes:
>>
>>  - a better/more explicit listing of dependencies for on-target
>>builds of "make scripts" or "make modules_prepare"
>>
>>  - The kernel source is installed into /lib/modules//build
>>and a symlink created from /usr/src/kernel to the new location.
>>This aligns with the standard location for module support
>>code
>>
>>  - There is also a symlink from /lib/modules//source -> build
>>to reserve a spot for a new package that is simply the kernel
>>source. That package is not part of this update.
>>
>> [YOCTO #12535]
>>
>> Signed-off-by: Bruce Ashfield 
>> ---
>>
>>  v2: fix arch/arm devsrc builds
>
> Noticed one kernel-devsrc build failure in autobuilder (qemuarm) :
>
> https://autobuilder.yocto.io/builders/nightly-arm/builds/876/steps/BuildImages/logs/stdio

This failure actually can't be from v2, it is the v1 issue I fixed.
The offending lines of
code are completely gone.

Bruce

>
> ERROR: kernel-devsrc-1.0-r0 do_install: Function failed: do_install (log file 
> is located at 
> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822)
> ERROR: Logfile of failure stored in: 
> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822
> Log data follows:
> | DEBUG: Executing python function extend_recipe_sysroot
> | NOTE: Direct dependencies are 
> ['/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/binutils/binutils-cross_2.30.bb:do_populate_sysroot',
>  
> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-extended/xz/xz_5.2.3.bb:do_populate_sysroot',
>  
> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot',
>  
> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-support/lzop/lzop_1.03.bb:do_populate_sysroot',
>  
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot',
>  
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-runtime_7.3.bb:do_populate_sysroot',
>  
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-cross_
 7.
>  3.bb:do_populate_sysroot', 
> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/bison/bison_3.0.4.bb:do_populate_sysroot',
>  
> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/pseudo/pseudo_1.8.2.bb:do_populate_sysroot',
>  
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-core/glibc/glibc_2.26.bb:do_populate_sysroot',
>  
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/kmod/kmod-native_git.bb:do_populate_sysroot',
>  
> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-extended/bc/bc_1.06.bb:do_populate_sysroot',
>  
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/quilt/quilt-native_0.65.bb:do_populate_sysroot',
>  
> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-connectivity/openssl/openssl_1.0.2n.bb
 :d
>  o_populate_sysroot']
> | NOTE: Installed into sysroot: ['pkgconfig-native', 'lzop-native', 
> 'kern-tools-native', 'pseudo-native', 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Maxin B. John]

>Also fails on musl:
>
>ERROR: Nothing RPROVIDES 'glibc-utils' (but 
>/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/kernel-devsrc.bb
> RDEPENDS on or otherwise requires it)
>
>Ross
>
>On 1 March 2018 at 08:28, Maxin B. John  wrote:

 >On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
 >> The existing kernel-devsrc package starts with a full copy of the kernel
 >> source and then starts to strip out elements that are not required.
 >>
 >> This results in extra time (I/O) and extra space being taken up in the
 >> final package. The main purpose of the kernel-devsrc package has been to
 >> build modules against the running kernel, not to include a full copy of
 >> the source code for re-building the kernel. The end result was a
 >> 600M kernel-devsrc package.

 Other failures are listed below:
 
 arm64:
 
https://autobuilder.yocto.io/builders/nightly-arm64/builds/790/steps/Running%20Sanity%20Tests/logs/stdio

 RESULTS - kernelmodule.KernelModuleTest.test_kernel_module - Testcase 1541: 
FAILED

HOSTCC  scripts/sortextable
 scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such file 
or directory
#include 
  ^~
compilation terminated.
make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] Error 1
make: *** [Makefile:557: scripts] Error 2
--

Similar on qemuarm - poky-lsb :

https://autobuilder.yocto.io/builders/nightly-arm-lsb/builds/812/steps/Running%20Sanity%20Tests/logs/stdio

HOSTCC  scripts/recordmcount
HOSTCC  scripts/sortextable
scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such file or 
directory
#include 
  ^~
  compilation terminated.
  make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] 
Error 1
  make: *** [Makefile:557: scripts] Error 2
-

Also on PowerPC:

https://autobuilder.yocto.io/builders/nightly-ppc-lsb/builds/814/steps/Running%20Sanity%20Tests/logs/stdio

scripts/sortextable.c:31:10: fatal error: tools/be_byteshift.h: No such file or 
directory
 #include 
   ^~
compilation terminated.
make[1]: *** [scripts/Makefile.host:102: scripts/sortextable] Error 1
make: *** [Makefile:557: scripts] Error 2

Best Regards,
Maxin
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Burton, Ross]

Also fails on musl:

ERROR: Nothing RPROVIDES 'glibc-utils' (but
/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/
kernel-devsrc.bb RDEPENDS on or otherwise requires it)

Ross

On 1 March 2018 at 08:28, Maxin B. John  wrote:

> Hi,
>
> On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
> > The existing kernel-devsrc package starts with a full copy of the kernel
> > source and then starts to strip out elements that are not required.
> >
> > This results in extra time (I/O) and extra space being taken up in the
> > final package. The main purpose of the kernel-devsrc package has been to
> > build modules against the running kernel, not to include a full copy of
> > the source code for re-building the kernel. The end result was a
> > 600M kernel-devsrc package.
> >
> > This restructuring of the package uses an approach similar to other
> > distros, where the kernel-devsrc package is for building against the
> > running kernel and uses a curated set of copied infrastructure, versus
> > a mass copy of the entire kernel.
> >
> > The differences in this approach versus other is largely due to the
> > architecture support and the split build/source directory of the
> > kernel.
> >
> > The result is a kernel-devsrc package of about 10M, which is capable
> > of running "make scripts" and compiling kernel modules against the
> > running kernel.
> >
> > Along with the changes to the copying of the infrascture, we also
> > have the following changes:
> >
> >  - a better/more explicit listing of dependencies for on-target
> >builds of "make scripts" or "make modules_prepare"
> >
> >  - The kernel source is installed into /lib/modules//build
> >and a symlink created from /usr/src/kernel to the new location.
> >This aligns with the standard location for module support
> >code
> >
> >  - There is also a symlink from /lib/modules//source -> build
> >to reserve a spot for a new package that is simply the kernel
> >source. That package is not part of this update.
> >
> > [YOCTO #12535]
> >
> > Signed-off-by: Bruce Ashfield 
> > ---
> >
> >  v2: fix arch/arm devsrc builds
>
> Noticed one kernel-devsrc build failure in autobuilder (qemuarm) :
>
> https://autobuilder.yocto.io/builders/nightly-arm/builds/
> 876/steps/BuildImages/logs/stdio
>
> ERROR: kernel-devsrc-1.0-r0 do_install: Function failed: do_install (log
> file is located at /home/pokybuild/yocto-autobuilder/yocto-worker/
> nightly-arm/build/build/tmp/work/qemuarm-poky-linux-
> gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822)
> ERROR: Logfile of failure stored in: /home/pokybuild/yocto-
> autobuilder/yocto-worker/nightly-arm/build/build/tmp/
> work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/
> temp/log.do_install.25822
> Log data follows:
> | DEBUG: Executing python function extend_recipe_sysroot
> | NOTE: Direct dependencies are ['/home/pokybuild/yocto-
> autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/binutils/
> binutils-cross_2.30.bb:do_populate_sysroot', 'virtual:native:/home/
> pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/
> build/meta/recipes-extended/xz/xz_5.2.3.bb:do_populate_sysroot',
> 'virtual:native:/home/pokybuild/yocto-autobuilder/
> yocto-worker/nightly-arm/build/meta/recipes-devtools/
> pkgconfig/pkgconfig_git.bb:do_populate_sysroot', 'virtual:native:/home/
> pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/
> build/meta/recipes-support/lzop/lzop_1.03.bb:do_populate_sysroot',
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/
> recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot',
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/
> recipes-devtools/gcc/gcc-runtime_7.3.bb:do_populate_sysroot',
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/
> recipes-devtools/gcc/gcc-cross_7.
>  3.bb:do_populate_sysroot', 'virtual:native:/home/
> pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/
> build/meta/recipes-devtools/bison/bison_3.0.4.bb:do_populate_sysroot',
> 'virtual:native:/home/pokybuild/yocto-autobuilder/
> yocto-worker/nightly-arm/build/meta/recipes-devtools/
> pseudo/pseudo_1.8.2.bb:do_populate_sysroot', '/home/pokybuild/yocto-
> autobuilder/yocto-worker/nightly-arm/build/meta/
> recipes-core/glibc/glibc_2.26.bb:do_populate_sysroot',
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/
> recipes-kernel/kmod/kmod-native_git.bb:do_populate_sysroot',
> 'virtual:native:/home/pokybuild/yocto-autobuilder/
> yocto-worker/nightly-arm/build/meta/recipes-extended/bc/bc_1.06.bb:
> do_populate_sysroot', '/home/pokybuild/yocto-autobuilder/yocto-worker/
> nightly-arm/build/meta/recipes-devtools/quilt/quilt-native_0.65.bb:
> do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-autobuilder/
> yocto-worker/nightly-arm/build/meta/recipes-connectivity/openssl/openssl_
> 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Maxin B. John]

Hi,

On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
> The existing kernel-devsrc package starts with a full copy of the kernel
> source and then starts to strip out elements that are not required.
> 
> This results in extra time (I/O) and extra space being taken up in the
> final package. The main purpose of the kernel-devsrc package has been to
> build modules against the running kernel, not to include a full copy of
> the source code for re-building the kernel. The end result was a
> 600M kernel-devsrc package.
> 
> This restructuring of the package uses an approach similar to other
> distros, where the kernel-devsrc package is for building against the
> running kernel and uses a curated set of copied infrastructure, versus
> a mass copy of the entire kernel.
> 
> The differences in this approach versus other is largely due to the
> architecture support and the split build/source directory of the
> kernel.
> 
> The result is a kernel-devsrc package of about 10M, which is capable
> of running "make scripts" and compiling kernel modules against the
> running kernel.
> 
> Along with the changes to the copying of the infrascture, we also
> have the following changes:
> 
>  - a better/more explicit listing of dependencies for on-target
>builds of "make scripts" or "make modules_prepare"
> 
>  - The kernel source is installed into /lib/modules//build
>and a symlink created from /usr/src/kernel to the new location.
>This aligns with the standard location for module support
>code
> 
>  - There is also a symlink from /lib/modules//source -> build
>to reserve a spot for a new package that is simply the kernel
>source. That package is not part of this update.
> 
> [YOCTO #12535]
> 
> Signed-off-by: Bruce Ashfield 
> ---
> 
>  v2: fix arch/arm devsrc builds

Noticed one kernel-devsrc build failure in autobuilder (qemuarm) :

https://autobuilder.yocto.io/builders/nightly-arm/builds/876/steps/BuildImages/logs/stdio

ERROR: kernel-devsrc-1.0-r0 do_install: Function failed: do_install (log file 
is located at 
/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822)
ERROR: Logfile of failure stored in: 
/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822
Log data follows:
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are 
['/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/binutils/binutils-cross_2.30.bb:do_populate_sysroot',
 
'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-extended/xz/xz_5.2.3.bb:do_populate_sysroot',
 
'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot',
 
'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-support/lzop/lzop_1.03.bb:do_populate_sysroot',
 
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot',
 
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-runtime_7.3.bb:do_populate_sysroot',
 
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-cross_7.
 3.bb:do_populate_sysroot', 
'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/bison/bison_3.0.4.bb:do_populate_sysroot',
 
'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/pseudo/pseudo_1.8.2.bb:do_populate_sysroot',
 
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-core/glibc/glibc_2.26.bb:do_populate_sysroot',
 
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/kmod/kmod-native_git.bb:do_populate_sysroot',
 
'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-extended/bc/bc_1.06.bb:do_populate_sysroot',
 
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/quilt/quilt-native_0.65.bb:do_populate_sysroot',
 
'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-connectivity/openssl/openssl_1.0.2n.bb:d
 o_populate_sysroot']
| NOTE: Installed into sysroot: ['pkgconfig-native', 'lzop-native', 
'kern-tools-native', 'pseudo-native', 'kmod-native', 'bc-native', 
'openssl-native', 'gtk-doc-native', 'lzo-native', 'makedepend-native', 
'cryptodev-linux-native', 'util-macros-native', 'xproto-native']
| NOTE: Skipping as already exists in sysroot: ['binutils-cross-arm', 
'xz-native', 'gcc-runtime', 'gcc-cross-arm', 'bison-native', 'glibc', 
'quilt-native', 

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

On 2018-02-28 4:10 PM, Mark Hatle wrote:

One question, below inline:

On 2/28/18 1:20 PM, Bruce Ashfield wrote:

The existing kernel-devsrc package starts with a full copy of the kernel
source and then starts to strip out elements that are not required.

This results in extra time (I/O) and extra space being taken up in the
final package. The main purpose of the kernel-devsrc package has been to
build modules against the running kernel, not to include a full copy of
the source code for re-building the kernel. The end result was a
600M kernel-devsrc package.

This restructuring of the package uses an approach similar to other
distros, where the kernel-devsrc package is for building against the
running kernel and uses a curated set of copied infrastructure, versus
a mass copy of the entire kernel.

The differences in this approach versus other is largely due to the
architecture support and the split build/source directory of the
kernel.

The result is a kernel-devsrc package of about 10M, which is capable
of running "make scripts" and compiling kernel modules against the
running kernel.

Along with the changes to the copying of the infrascture, we also
have the following changes:

  - a better/more explicit listing of dependencies for on-target
builds of "make scripts" or "make modules_prepare"

  - The kernel source is installed into /lib/modules//build
and a symlink created from /usr/src/kernel to the new location.
This aligns with the standard location for module support
code

  - There is also a symlink from /lib/modules//source -> build
to reserve a spot for a new package that is simply the kernel
source. That package is not part of this update.

[YOCTO #12535]

Signed-off-by: Bruce Ashfield 
---

  v2: fix arch/arm devsrc builds

  The arch conditional code for ARM was using invalid variables for the
  current version of the recipe. Those are now fixed, and we can package
  for ARM.

  meta/recipes-kernel/linux/kernel-devsrc.bb | 178 ++---
  1 file changed, 138 insertions(+), 40 deletions(-)

diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb 
b/meta/recipes-kernel/linux/kernel-devsrc.bb
index c1b5b7786d7c..738290ac2dc3 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -30,50 +30,148 @@ do_populate_sysroot[noexec] = "1"
  S = "${STAGING_KERNEL_DIR}"
  B = "${STAGING_KERNEL_BUILDDIR}"
  
-KERNEL_VERSION = "${@get_kernelversion_headers('${S}')}"

-


I see this is removed, but later items reference KERNEL_VERSION.  Is that
defined elsewhere?


Yup. It comes from the base classes. That variable in the current
form, is actually empty.

Bruce



--Mark


  PACKAGE_ARCH = "${MACHINE_ARCH}"
  
+KERNEL_BUILD_ROOT="/lib/modules/"

+
  do_install() {
-kerneldir=${D}${KERNEL_SRC_PATH}
-install -d $kerneldir
-
-#
-# Copy the staging dir source (and module build support) into the 
devsrc structure.
-# We can keep this copy simple and take everything, since a we'll 
clean up any build
-# artifacts afterwards, and the extra i/o is not significant
-#
-cd ${B}
-find . -type d -name '.git*' -prune -o -path '.debug' -prune -o -type 
f -print0 | cpio --null -pdlu $kerneldir
-cd ${S}
-   find . -type d -name '.git*' -prune -o -type d -name '.kernel-meta' 
-prune -o -type f -print0 | cpio --null -pdlu $kerneldir
-
-# Explicitly set KBUILD_OUTPUT to ensure that the image directory is 
cleaned and not
-# The main build artifacts. We clean the directory to avoid QA errors 
on mismatched
-# architecture (since scripts and helpers are native format).
-KBUILD_OUTPUT="$kerneldir"
-oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean 
_mrproper_scripts
-# make clean generates an absolute path symlink called "source"
-# in $kerneldir points to $kerneldir, which doesn't make any
-# sense, so remove it.
-if [ -L $kerneldir/source ]; then
-bbnote "Removing $kerneldir/source symlink"
-rm -f $kerneldir/source
-fi
-
-# As of Linux kernel version 3.0.1, the clean target removes
-# arch/powerpc/lib/crtsavres.o which is present in
-# KBUILD_LDFLAGS_MODULE, making it required to build external modules.
-if [ ${ARCH} = "powerpc" ]; then
-mkdir -p $kerneldir/arch/powerpc/lib/
-cp ${B}/arch/powerpc/lib/crtsavres.o 
$kerneldir/arch/powerpc/lib/crtsavres.o
-fi
-
-chown -R root:root ${D}
+kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
+install -d $kerneldir
+
+# create the directory structure
+rm -f $kerneldir/build
+rm -f $kerneldir/source
+mkdir -p $kerneldir/build
+
+# for compatibility with some older variants of this package, we
+# create  a /usr/src/kernel symlink to /lib/modules//source

Re: [OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Mark Hatle]

One question, below inline:

On 2/28/18 1:20 PM, Bruce Ashfield wrote:
> The existing kernel-devsrc package starts with a full copy of the kernel
> source and then starts to strip out elements that are not required.
> 
> This results in extra time (I/O) and extra space being taken up in the
> final package. The main purpose of the kernel-devsrc package has been to
> build modules against the running kernel, not to include a full copy of
> the source code for re-building the kernel. The end result was a
> 600M kernel-devsrc package.
> 
> This restructuring of the package uses an approach similar to other
> distros, where the kernel-devsrc package is for building against the
> running kernel and uses a curated set of copied infrastructure, versus
> a mass copy of the entire kernel.
> 
> The differences in this approach versus other is largely due to the
> architecture support and the split build/source directory of the
> kernel.
> 
> The result is a kernel-devsrc package of about 10M, which is capable
> of running "make scripts" and compiling kernel modules against the
> running kernel.
> 
> Along with the changes to the copying of the infrascture, we also
> have the following changes:
> 
>  - a better/more explicit listing of dependencies for on-target
>builds of "make scripts" or "make modules_prepare"
> 
>  - The kernel source is installed into /lib/modules//build
>and a symlink created from /usr/src/kernel to the new location.
>This aligns with the standard location for module support
>code
> 
>  - There is also a symlink from /lib/modules//source -> build
>to reserve a spot for a new package that is simply the kernel
>source. That package is not part of this update.
> 
> [YOCTO #12535]
> 
> Signed-off-by: Bruce Ashfield 
> ---
> 
>  v2: fix arch/arm devsrc builds
> 
>  The arch conditional code for ARM was using invalid variables for the
>  current version of the recipe. Those are now fixed, and we can package
>  for ARM.
> 
>  meta/recipes-kernel/linux/kernel-devsrc.bb | 178 
> ++---
>  1 file changed, 138 insertions(+), 40 deletions(-)
> 
> diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb 
> b/meta/recipes-kernel/linux/kernel-devsrc.bb
> index c1b5b7786d7c..738290ac2dc3 100644
> --- a/meta/recipes-kernel/linux/kernel-devsrc.bb
> +++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
> @@ -30,50 +30,148 @@ do_populate_sysroot[noexec] = "1"
>  S = "${STAGING_KERNEL_DIR}"
>  B = "${STAGING_KERNEL_BUILDDIR}"
>  
> -KERNEL_VERSION = "${@get_kernelversion_headers('${S}')}"
> -

I see this is removed, but later items reference KERNEL_VERSION.  Is that
defined elsewhere?

--Mark

>  PACKAGE_ARCH = "${MACHINE_ARCH}"
>  
> +KERNEL_BUILD_ROOT="/lib/modules/"
> +
>  do_install() {
> -kerneldir=${D}${KERNEL_SRC_PATH}
> -install -d $kerneldir
> -
> -#
> -# Copy the staging dir source (and module build support) into the 
> devsrc structure.
> -# We can keep this copy simple and take everything, since a we'll 
> clean up any build
> -# artifacts afterwards, and the extra i/o is not significant
> -#
> -cd ${B}
> -find . -type d -name '.git*' -prune -o -path '.debug' -prune -o 
> -type f -print0 | cpio --null -pdlu $kerneldir
> -cd ${S}
> - find . -type d -name '.git*' -prune -o -type d -name '.kernel-meta' 
> -prune -o -type f -print0 | cpio --null -pdlu $kerneldir
> -
> -# Explicitly set KBUILD_OUTPUT to ensure that the image directory is 
> cleaned and not
> -# The main build artifacts. We clean the directory to avoid QA 
> errors on mismatched
> -# architecture (since scripts and helpers are native format).
> -KBUILD_OUTPUT="$kerneldir"
> -oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean 
> _mrproper_scripts
> -# make clean generates an absolute path symlink called "source"
> -# in $kerneldir points to $kerneldir, which doesn't make any
> -# sense, so remove it.
> -if [ -L $kerneldir/source ]; then
> -bbnote "Removing $kerneldir/source symlink"
> -rm -f $kerneldir/source
> -fi
> -
> -# As of Linux kernel version 3.0.1, the clean target removes
> -# arch/powerpc/lib/crtsavres.o which is present in
> -# KBUILD_LDFLAGS_MODULE, making it required to build external 
> modules.
> -if [ ${ARCH} = "powerpc" ]; then
> -mkdir -p $kerneldir/arch/powerpc/lib/
> -cp ${B}/arch/powerpc/lib/crtsavres.o 
> $kerneldir/arch/powerpc/lib/crtsavres.o
> -fi
> -
> -chown -R root:root ${D}
> +kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
> +install -d $kerneldir
> +
> +# create the directory structure
> +rm -f $kerneldir/build
> +rm -f $kerneldir/source
> +mkdir -p $kerneldir/build
> +
> +# for compatibility with some older variants of this 

[OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

[Bruce Ashfield]

The existing kernel-devsrc package starts with a full copy of the kernel
source and then starts to strip out elements that are not required.

This results in extra time (I/O) and extra space being taken up in the
final package. The main purpose of the kernel-devsrc package has been to
build modules against the running kernel, not to include a full copy of
the source code for re-building the kernel. The end result was a
600M kernel-devsrc package.

This restructuring of the package uses an approach similar to other
distros, where the kernel-devsrc package is for building against the
running kernel and uses a curated set of copied infrastructure, versus
a mass copy of the entire kernel.

The differences in this approach versus other is largely due to the
architecture support and the split build/source directory of the
kernel.

The result is a kernel-devsrc package of about 10M, which is capable
of running "make scripts" and compiling kernel modules against the
running kernel.

Along with the changes to the copying of the infrascture, we also
have the following changes:

 - a better/more explicit listing of dependencies for on-target
   builds of "make scripts" or "make modules_prepare"

 - The kernel source is installed into /lib/modules//build
   and a symlink created from /usr/src/kernel to the new location.
   This aligns with the standard location for module support
   code

 - There is also a symlink from /lib/modules//source -> build
   to reserve a spot for a new package that is simply the kernel
   source. That package is not part of this update.

[YOCTO #12535]

Signed-off-by: Bruce Ashfield 
---

 v2: fix arch/arm devsrc builds

 The arch conditional code for ARM was using invalid variables for the
 current version of the recipe. Those are now fixed, and we can package
 for ARM.

 meta/recipes-kernel/linux/kernel-devsrc.bb | 178 ++---
 1 file changed, 138 insertions(+), 40 deletions(-)

diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb 
b/meta/recipes-kernel/linux/kernel-devsrc.bb
index c1b5b7786d7c..738290ac2dc3 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -30,50 +30,148 @@ do_populate_sysroot[noexec] = "1"
 S = "${STAGING_KERNEL_DIR}"
 B = "${STAGING_KERNEL_BUILDDIR}"
 
-KERNEL_VERSION = "${@get_kernelversion_headers('${S}')}"
-
 PACKAGE_ARCH = "${MACHINE_ARCH}"
 
+KERNEL_BUILD_ROOT="/lib/modules/"
+
 do_install() {
-kerneldir=${D}${KERNEL_SRC_PATH}
-install -d $kerneldir
-
-#
-# Copy the staging dir source (and module build support) into the 
devsrc structure.
-# We can keep this copy simple and take everything, since a we'll 
clean up any build
-# artifacts afterwards, and the extra i/o is not significant
-#
-cd ${B}
-find . -type d -name '.git*' -prune -o -path '.debug' -prune -o -type 
f -print0 | cpio --null -pdlu $kerneldir
-cd ${S}
-   find . -type d -name '.git*' -prune -o -type d -name '.kernel-meta' 
-prune -o -type f -print0 | cpio --null -pdlu $kerneldir
-
-# Explicitly set KBUILD_OUTPUT to ensure that the image directory is 
cleaned and not
-# The main build artifacts. We clean the directory to avoid QA errors 
on mismatched
-# architecture (since scripts and helpers are native format).
-KBUILD_OUTPUT="$kerneldir"
-oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean 
_mrproper_scripts
-# make clean generates an absolute path symlink called "source"
-# in $kerneldir points to $kerneldir, which doesn't make any
-# sense, so remove it.
-if [ -L $kerneldir/source ]; then
-bbnote "Removing $kerneldir/source symlink"
-rm -f $kerneldir/source
-fi
-
-# As of Linux kernel version 3.0.1, the clean target removes
-# arch/powerpc/lib/crtsavres.o which is present in
-# KBUILD_LDFLAGS_MODULE, making it required to build external modules.
-if [ ${ARCH} = "powerpc" ]; then
-mkdir -p $kerneldir/arch/powerpc/lib/
-cp ${B}/arch/powerpc/lib/crtsavres.o 
$kerneldir/arch/powerpc/lib/crtsavres.o
-fi
-
-chown -R root:root ${D}
+kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
+install -d $kerneldir
+
+# create the directory structure
+rm -f $kerneldir/build
+rm -f $kerneldir/source
+mkdir -p $kerneldir/build
+
+# for compatibility with some older variants of this package, we
+# create  a /usr/src/kernel symlink to /lib/modules//source
+mkdir -p ${D}/usr/src
+(
+   cd ${D}/usr/src
+   ln -s ${KERNEL_BUILD_ROOT}${KERNEL_VERSION}/source kernel
+)
+
+# for on target purposes, we unify build and source
+(
+   cd $kerneldir
+   ln -s build source
+)
+
+# first copy everything
+(
+   cd ${S}
+   cp --parents $(find  -type f