Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
Hi Bruce, > On 08/22/2018 10:47 AM, Lukasz Majewski wrote: > > On Wed, 22 Aug 2018 10:44:08 -0400 > > Bruce Ashfield wrote: > > > >> On 08/22/2018 10:20 AM, Lukasz Majewski wrote: > >>> On Wed, 22 Aug 2018 10:13:33 -0400 > >>> Bruce Ashfield wrote: > >>> > On 08/22/2018 10:05 AM, Lukasz Majewski wrote: > > Hi Bruce, > > > >> On 08/22/2018 09:40 AM, Lukasz Majewski wrote: > >>> Without this patch it happens that do_populate_recipe_sysroot > >>> is called just before do_compile (on multi core build > >>> machines). This is way too late as the .config generated in > >>> do_kernel_configme() is already broken. > >>> > >>> The problem is that do_kernel_configme() calls native's > >>> merge_config.sh script which may call bison and > >>> arm-linux-*-gcc to compile kconfig infrastructure to create > >>> base .config (scripts/kconfig/conf --allnoconfig Kconfig.) > >>> > >>> It turns out that we only got above binaries after > >>> do_prepare_recipe_sysroot() is called. > >>> If those binaries are not available, the merge_config.sh do > >>> not generate base .config and without any error produces > >>> malfunctioned .config. The build process continues and as a > >>> result broken kernel is compiled. > >>> > >>> To reproduce: > >>> > >>> bitbake -c cleansstate virtual/kernel > >>> bitbake -c kernel_metadata -v virtual/kernel > >>> bitbake -c do_kernel_configme -v virtual/kernel > >>> (one shall see broken .config in ${B}/.config) > >>> bitbake -c do_populate_sysroot -v virtual/kernel > >>> bitbake -c do_compile -v virtual/kernel > >>> > >>> (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d > >>> > >>> Signed-off-by: Lukasz Majewski > >>> --- > >>> meta/classes/kernel-yocto.bbclass | 1 + > >>> 1 file changed, 1 insertion(+) > >>> > >>> diff --git a/meta/classes/kernel-yocto.bbclass > >>> b/meta/classes/kernel-yocto.bbclass index > >>> 4ac3a39e47..19a3f2bc46 100644 --- > >>> a/meta/classes/kernel-yocto.bbclass +++ > >>> b/meta/classes/kernel-yocto.bbclass @@ -276,6 +276,7 @@ > >>> do_kernel_metadata[depends] = > >>> "kern-tools-native:do_populate_sysroot" > >>> do_validate_branches[depends] = > >>> "kern-tools-native:do_populate_sysroot" > >>> do_kernel_configme[dirs] += "${S} ${B}" > >>> +do_kernel_configme[depends] += > >>> "virtual/kernel:do_prepare_recipe_sysroot" > >> > >> On a closer look .. what branch is this from ? We already have > >> a change that triggered breakage on 4.17+ that should fix this. > >> > >> i.e. commit ff1bdd75d50f0ebac3d599e461685ace29559a82 in > >> oe-core, adds: > >> > >> -- > >> > >> do_kernel_metadata[depends] = > >> "kern-tools-native:do_populate_sysroot" > >> do_validate_branches[depends] = > >> "kern-tools-native:do_populate_sysroot" > >> > >> +do_kernel_configme[depends] += > >> "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot" > >> +do_kernel_configme[depends] += > >> "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot" > >> +do_kernel_configme[depends] += "bc-native:do_populate_sysroot > >> bison-native:do_populate_sysroot" > >> do_kernel_configme[dirs] += "${S} ${B}" > >> do_kernel_configme() { > >> > >> -- > >> > >> Which I don't see in the context of your patch. > >> > > > > I do use poky as in the commit message. > > Yes, I checked that .. but that isn't a useful part of a patch > to oe-core. > > The kernel-yocto bbclass in that context is not the one in > master, and if I run a --contains on that commit ID, I don't get > an active branch. > >>> > >>> I might have misunderstood something... apparently. > >>> > >>> The changes which I did were to: > >>> > >>> poky/meta/classes/kernel-yocto.bbclass > >>> > >>> Is this a wrong place to add the fix? > >> > >> It is the right place to add the fix, but the tree that you are > >> using is not up to date with the current poky or oe-core master. > >> > >> There are other changes in place that address a similar failure. > >> > >> For us to look at this change, it needs to be rebased on top of > >> master (or master-next). > > > > Now I got it. I shall sent the fix against cutting edge -master. > > > > Shall I re-send the fix, of will you first check if patches in the > > queue doesn't fix it already? > > I ran the steps yesterday and wasn't able to trigger the issue > here. > > so if you try against the current master of poky/oe-core, I'd be > interested to hear if you can make the problem happen (and I can > re-try as required). The upstream patch: "kernel: specify dependencies for compilation for config tasks" SHA1: 81e8a52e8e40e47c34f900db5d73e69ffc25f5d0 Fixes the problem, which I've
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
On 08/22/2018 10:47 AM, Lukasz Majewski wrote: On Wed, 22 Aug 2018 10:44:08 -0400 Bruce Ashfield wrote: On 08/22/2018 10:20 AM, Lukasz Majewski wrote: On Wed, 22 Aug 2018 10:13:33 -0400 Bruce Ashfield wrote: On 08/22/2018 10:05 AM, Lukasz Majewski wrote: Hi Bruce, On 08/22/2018 09:40 AM, Lukasz Majewski wrote: Without this patch it happens that do_populate_recipe_sysroot is called just before do_compile (on multi core build machines). This is way too late as the .config generated in do_kernel_configme() is already broken. The problem is that do_kernel_configme() calls native's merge_config.sh script which may call bison and arm-linux-*-gcc to compile kconfig infrastructure to create base .config (scripts/kconfig/conf --allnoconfig Kconfig.) It turns out that we only got above binaries after do_prepare_recipe_sysroot() is called. If those binaries are not available, the merge_config.sh do not generate base .config and without any error produces malfunctioned .config. The build process continues and as a result broken kernel is compiled. To reproduce: bitbake -c cleansstate virtual/kernel bitbake -c kernel_metadata -v virtual/kernel bitbake -c do_kernel_configme -v virtual/kernel (one shall see broken .config in ${B}/.config) bitbake -c do_populate_sysroot -v virtual/kernel bitbake -c do_compile -v virtual/kernel (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d Signed-off-by: Lukasz Majewski --- meta/classes/kernel-yocto.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 100644 --- a/meta/classes/kernel-yocto.bbclass +++ b/meta/classes/kernel-yocto.bbclass @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" +do_kernel_configme[depends] += "virtual/kernel:do_prepare_recipe_sysroot" On a closer look .. what branch is this from ? We already have a change that triggered breakage on 4.17+ that should fix this. i.e. commit ff1bdd75d50f0ebac3d599e461685ace29559a82 in oe-core, adds: -- do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot" +do_kernel_configme[depends] += "bc-native:do_populate_sysroot bison-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" do_kernel_configme() { -- Which I don't see in the context of your patch. I do use poky as in the commit message. Yes, I checked that .. but that isn't a useful part of a patch to oe-core. The kernel-yocto bbclass in that context is not the one in master, and if I run a --contains on that commit ID, I don't get an active branch. I might have misunderstood something... apparently. The changes which I did were to: poky/meta/classes/kernel-yocto.bbclass Is this a wrong place to add the fix? It is the right place to add the fix, but the tree that you are using is not up to date with the current poky or oe-core master. There are other changes in place that address a similar failure. For us to look at this change, it needs to be rebased on top of master (or master-next). Now I got it. I shall sent the fix against cutting edge -master. Shall I re-send the fix, of will you first check if patches in the queue doesn't fix it already? I ran the steps yesterday and wasn't able to trigger the issue here. so if you try against the current master of poky/oe-core, I'd be interested to hear if you can make the problem happen (and I can re-try as required). Bruce Bruce So the patch as it stands, cannot be applied to master (and I also checked master-next in case something snuck in that I missed) of oe-core or poky, so there's no way to work with it. Bruce Regarding the kernel - I do have my own recipe; inherit kernel require recipes-kernel/linux/linux-yocto.inc And then I do have: COMPATIBLE_MACHINE_display5 = "display5" KBUILD_DEFCONFIG_pn-linux-yocto-custom_display5 = "imx6q_display5_defconfig" As the board is in mainline I just point to Linus tree. Bruce do_kernel_configme() { set +e Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email:
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
On 08/22/2018 10:47 AM, Lukasz Majewski wrote: On Wed, 22 Aug 2018 10:44:08 -0400 Bruce Ashfield wrote: On 08/22/2018 10:20 AM, Lukasz Majewski wrote: On Wed, 22 Aug 2018 10:13:33 -0400 Bruce Ashfield wrote: On 08/22/2018 10:05 AM, Lukasz Majewski wrote: Hi Bruce, On 08/22/2018 09:40 AM, Lukasz Majewski wrote: Without this patch it happens that do_populate_recipe_sysroot is called just before do_compile (on multi core build machines). This is way too late as the .config generated in do_kernel_configme() is already broken. The problem is that do_kernel_configme() calls native's merge_config.sh script which may call bison and arm-linux-*-gcc to compile kconfig infrastructure to create base .config (scripts/kconfig/conf --allnoconfig Kconfig.) It turns out that we only got above binaries after do_prepare_recipe_sysroot() is called. If those binaries are not available, the merge_config.sh do not generate base .config and without any error produces malfunctioned .config. The build process continues and as a result broken kernel is compiled. To reproduce: bitbake -c cleansstate virtual/kernel bitbake -c kernel_metadata -v virtual/kernel bitbake -c do_kernel_configme -v virtual/kernel (one shall see broken .config in ${B}/.config) bitbake -c do_populate_sysroot -v virtual/kernel bitbake -c do_compile -v virtual/kernel (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d Signed-off-by: Lukasz Majewski --- meta/classes/kernel-yocto.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 100644 --- a/meta/classes/kernel-yocto.bbclass +++ b/meta/classes/kernel-yocto.bbclass @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" +do_kernel_configme[depends] += "virtual/kernel:do_prepare_recipe_sysroot" On a closer look .. what branch is this from ? We already have a change that triggered breakage on 4.17+ that should fix this. i.e. commit ff1bdd75d50f0ebac3d599e461685ace29559a82 in oe-core, adds: -- do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot" +do_kernel_configme[depends] += "bc-native:do_populate_sysroot bison-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" do_kernel_configme() { -- Which I don't see in the context of your patch. I do use poky as in the commit message. Yes, I checked that .. but that isn't a useful part of a patch to oe-core. The kernel-yocto bbclass in that context is not the one in master, and if I run a --contains on that commit ID, I don't get an active branch. I might have misunderstood something... apparently. The changes which I did were to: poky/meta/classes/kernel-yocto.bbclass Is this a wrong place to add the fix? It is the right place to add the fix, but the tree that you are using is not up to date with the current poky or oe-core master. There are other changes in place that address a similar failure. For us to look at this change, it needs to be rebased on top of master (or master-next). Now I got it. I shall sent the fix against cutting edge -master. Shall I re-send the fix, of will you first check if patches in the queue doesn't fix it already? I'll run the steps in your reproducer and let you know if it works here. Bruce Bruce So the patch as it stands, cannot be applied to master (and I also checked master-next in case something snuck in that I missed) of oe-core or poky, so there's no way to work with it. Bruce Regarding the kernel - I do have my own recipe; inherit kernel require recipes-kernel/linux/linux-yocto.inc And then I do have: COMPATIBLE_MACHINE_display5 = "display5" KBUILD_DEFCONFIG_pn-linux-yocto-custom_display5 = "imx6q_display5_defconfig" As the board is in mainline I just point to Linus tree. Bruce do_kernel_configme() { set +e Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office:
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
On Wed, 22 Aug 2018 10:44:08 -0400 Bruce Ashfield wrote: > On 08/22/2018 10:20 AM, Lukasz Majewski wrote: > > On Wed, 22 Aug 2018 10:13:33 -0400 > > Bruce Ashfield wrote: > > > >> On 08/22/2018 10:05 AM, Lukasz Majewski wrote: > >>> Hi Bruce, > >>> > On 08/22/2018 09:40 AM, Lukasz Majewski wrote: > > Without this patch it happens that do_populate_recipe_sysroot is > > called just before do_compile (on multi core build machines). > > This is way too late as the .config generated in > > do_kernel_configme() is already broken. > > > > The problem is that do_kernel_configme() calls native's > > merge_config.sh script which may call bison and arm-linux-*-gcc > > to compile kconfig infrastructure to create base .config > > (scripts/kconfig/conf --allnoconfig Kconfig.) > > > > It turns out that we only got above binaries after > > do_prepare_recipe_sysroot() is called. > > If those binaries are not available, the merge_config.sh do not > > generate base .config and without any error produces > > malfunctioned .config. The build process continues and as a > > result broken kernel is compiled. > > > > To reproduce: > > > > bitbake -c cleansstate virtual/kernel > > bitbake -c kernel_metadata -v virtual/kernel > > bitbake -c do_kernel_configme -v virtual/kernel > > (one shall see broken .config in ${B}/.config) > > bitbake -c do_populate_sysroot -v virtual/kernel > > bitbake -c do_compile -v virtual/kernel > > > > (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d > > > > Signed-off-by: Lukasz Majewski > > --- > > meta/classes/kernel-yocto.bbclass | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/meta/classes/kernel-yocto.bbclass > > b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 > > 100644 --- a/meta/classes/kernel-yocto.bbclass > > +++ b/meta/classes/kernel-yocto.bbclass > > @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = > > "kern-tools-native:do_populate_sysroot" > > do_validate_branches[depends] = > > "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] > > += "${S} ${B}" +do_kernel_configme[depends] += > > "virtual/kernel:do_prepare_recipe_sysroot" > > On a closer look .. what branch is this from ? We already have a > change that triggered breakage on 4.17+ that should fix this. > > i.e. commit ff1bdd75d50f0ebac3d599e461685ace29559a82 in oe-core, > adds: > > -- > > do_kernel_metadata[depends] = > "kern-tools-native:do_populate_sysroot" > do_validate_branches[depends] = > "kern-tools-native:do_populate_sysroot" > > +do_kernel_configme[depends] += > "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot" > +do_kernel_configme[depends] += > "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot" > +do_kernel_configme[depends] += "bc-native:do_populate_sysroot > bison-native:do_populate_sysroot" > do_kernel_configme[dirs] += "${S} ${B}" > do_kernel_configme() { > > -- > > Which I don't see in the context of your patch. > > >>> > >>> I do use poky as in the commit message. > >> > >> Yes, I checked that .. but that isn't a useful part of a patch > >> to oe-core. > >> > >> The kernel-yocto bbclass in that context is not the one in master, > >> and if I run a --contains on that commit ID, I don't get an active > >> branch. > > > > I might have misunderstood something... apparently. > > > > The changes which I did were to: > > > > poky/meta/classes/kernel-yocto.bbclass > > > > Is this a wrong place to add the fix? > > It is the right place to add the fix, but the tree that you are > using is not up to date with the current poky or oe-core master. > > There are other changes in place that address a similar failure. > > For us to look at this change, it needs to be rebased on top of > master (or master-next). Now I got it. I shall sent the fix against cutting edge -master. Shall I re-send the fix, of will you first check if patches in the queue doesn't fix it already? > > Bruce > > > > >> > >> So the patch as it stands, cannot be applied to master (and I > >> also checked master-next in case something snuck in that I > >> missed) of oe-core or poky, so there's no way to work with it. > >> > >> Bruce > >> > >>> > >>> Regarding the kernel - I do have my own recipe; > >>> inherit kernel > >>> > >>> require > >>> recipes-kernel/linux/linux-yocto.inc > >>> > >>> > >>> And then I do have: > >>> COMPATIBLE_MACHINE_display5 = "display5" > >>> > >>> KBUILD_DEFCONFIG_pn-linux-yocto-custom_display5 = > >>> "imx6q_display5_defconfig" > >>> > >>> As the board is in mainline I just point to Linus tree. > >>> > Bruce > > > do_kernel_configme() { > > set +e > >
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
On 08/22/2018 10:20 AM, Lukasz Majewski wrote: On Wed, 22 Aug 2018 10:13:33 -0400 Bruce Ashfield wrote: On 08/22/2018 10:05 AM, Lukasz Majewski wrote: Hi Bruce, On 08/22/2018 09:40 AM, Lukasz Majewski wrote: Without this patch it happens that do_populate_recipe_sysroot is called just before do_compile (on multi core build machines). This is way too late as the .config generated in do_kernel_configme() is already broken. The problem is that do_kernel_configme() calls native's merge_config.sh script which may call bison and arm-linux-*-gcc to compile kconfig infrastructure to create base .config (scripts/kconfig/conf --allnoconfig Kconfig.) It turns out that we only got above binaries after do_prepare_recipe_sysroot() is called. If those binaries are not available, the merge_config.sh do not generate base .config and without any error produces malfunctioned .config. The build process continues and as a result broken kernel is compiled. To reproduce: bitbake -c cleansstate virtual/kernel bitbake -c kernel_metadata -v virtual/kernel bitbake -c do_kernel_configme -v virtual/kernel (one shall see broken .config in ${B}/.config) bitbake -c do_populate_sysroot -v virtual/kernel bitbake -c do_compile -v virtual/kernel (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d Signed-off-by: Lukasz Majewski --- meta/classes/kernel-yocto.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 100644 --- a/meta/classes/kernel-yocto.bbclass +++ b/meta/classes/kernel-yocto.bbclass @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" +do_kernel_configme[depends] += "virtual/kernel:do_prepare_recipe_sysroot" On a closer look .. what branch is this from ? We already have a change that triggered breakage on 4.17+ that should fix this. i.e. commit ff1bdd75d50f0ebac3d599e461685ace29559a82 in oe-core, adds: -- do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot" +do_kernel_configme[depends] += "bc-native:do_populate_sysroot bison-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" do_kernel_configme() { -- Which I don't see in the context of your patch. I do use poky as in the commit message. Yes, I checked that .. but that isn't a useful part of a patch to oe-core. The kernel-yocto bbclass in that context is not the one in master, and if I run a --contains on that commit ID, I don't get an active branch. I might have misunderstood something... apparently. The changes which I did were to: poky/meta/classes/kernel-yocto.bbclass Is this a wrong place to add the fix? It is the right place to add the fix, but the tree that you are using is not up to date with the current poky or oe-core master. There are other changes in place that address a similar failure. For us to look at this change, it needs to be rebased on top of master (or master-next). Bruce So the patch as it stands, cannot be applied to master (and I also checked master-next in case something snuck in that I missed) of oe-core or poky, so there's no way to work with it. Bruce Regarding the kernel - I do have my own recipe; inherit kernel require recipes-kernel/linux/linux-yocto.inc And then I do have: COMPATIBLE_MACHINE_display5 = "display5" KBUILD_DEFCONFIG_pn-linux-yocto-custom_display5 = "imx6q_display5_defconfig" As the board is in mainline I just point to Linus tree. Bruce do_kernel_configme() { set +e Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
On Wed, 22 Aug 2018 10:13:33 -0400 Bruce Ashfield wrote: > On 08/22/2018 10:05 AM, Lukasz Majewski wrote: > > Hi Bruce, > > > >> On 08/22/2018 09:40 AM, Lukasz Majewski wrote: > >>> Without this patch it happens that do_populate_recipe_sysroot is > >>> called just before do_compile (on multi core build machines). > >>> This is way too late as the .config generated in > >>> do_kernel_configme() is already broken. > >>> > >>> The problem is that do_kernel_configme() calls native's > >>> merge_config.sh script which may call bison and arm-linux-*-gcc to > >>> compile kconfig infrastructure to create base .config > >>> (scripts/kconfig/conf --allnoconfig Kconfig.) > >>> > >>> It turns out that we only got above binaries after > >>> do_prepare_recipe_sysroot() is called. > >>> If those binaries are not available, the merge_config.sh do not > >>> generate base .config and without any error produces > >>> malfunctioned .config. The build process continues and as a result > >>> broken kernel is compiled. > >>> > >>> To reproduce: > >>> > >>> bitbake -c cleansstate virtual/kernel > >>> bitbake -c kernel_metadata -v virtual/kernel > >>> bitbake -c do_kernel_configme -v virtual/kernel > >>> (one shall see broken .config in ${B}/.config) > >>> bitbake -c do_populate_sysroot -v virtual/kernel > >>> bitbake -c do_compile -v virtual/kernel > >>> > >>> (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d > >>> > >>> Signed-off-by: Lukasz Majewski > >>> --- > >>>meta/classes/kernel-yocto.bbclass | 1 + > >>>1 file changed, 1 insertion(+) > >>> > >>> diff --git a/meta/classes/kernel-yocto.bbclass > >>> b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 > >>> 100644 --- a/meta/classes/kernel-yocto.bbclass > >>> +++ b/meta/classes/kernel-yocto.bbclass > >>> @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = > >>> "kern-tools-native:do_populate_sysroot" > >>> do_validate_branches[depends] = > >>> "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] > >>> += "${S} ${B}" +do_kernel_configme[depends] += > >>> "virtual/kernel:do_prepare_recipe_sysroot" > >> > >> On a closer look .. what branch is this from ? We already have a > >> change that triggered breakage on 4.17+ that should fix this. > >> > >> i.e. commit ff1bdd75d50f0ebac3d599e461685ace29559a82 in oe-core, > >> adds: > >> > >> -- > >> > >>do_kernel_metadata[depends] = > >> "kern-tools-native:do_populate_sysroot" > >> do_validate_branches[depends] = > >> "kern-tools-native:do_populate_sysroot" > >> > >> +do_kernel_configme[depends] += > >> "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot" > >> +do_kernel_configme[depends] += > >> "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot" > >> +do_kernel_configme[depends] += "bc-native:do_populate_sysroot > >> bison-native:do_populate_sysroot" > >>do_kernel_configme[dirs] += "${S} ${B}" > >>do_kernel_configme() { > >> > >> -- > >> > >> Which I don't see in the context of your patch. > >> > > > > I do use poky as in the commit message. > > Yes, I checked that .. but that isn't a useful part of a patch > to oe-core. > > The kernel-yocto bbclass in that context is not the one in master, and > if I run a --contains on that commit ID, I don't get an active > branch. I might have misunderstood something... apparently. The changes which I did were to: poky/meta/classes/kernel-yocto.bbclass Is this a wrong place to add the fix? > > So the patch as it stands, cannot be applied to master (and I > also checked master-next in case something snuck in that I > missed) of oe-core or poky, so there's no way to work with it. > > Bruce > > > > > Regarding the kernel - I do have my own recipe; > > inherit kernel > > > > require > > recipes-kernel/linux/linux-yocto.inc > > > > > > And then I do have: > > COMPATIBLE_MACHINE_display5 = "display5" > > > > KBUILD_DEFCONFIG_pn-linux-yocto-custom_display5 = > > "imx6q_display5_defconfig" > > > > As the board is in mainline I just point to Linus tree. > > > >> Bruce > >> > >>>do_kernel_configme() { > >>> set +e > >>> > >>> > >> > > > > > > > > > > Best regards, > > > > Lukasz Majewski > > > > -- > > > > DENX Software Engineering GmbH, Managing Director: Wolfgang > > Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, > > Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: > > w...@denx.de > Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de pgpKUFCqSg4k9.pgp Description: OpenPGP digital signature -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
On 08/22/2018 10:05 AM, Lukasz Majewski wrote: Hi Bruce, On 08/22/2018 09:40 AM, Lukasz Majewski wrote: Without this patch it happens that do_populate_recipe_sysroot is called just before do_compile (on multi core build machines). This is way too late as the .config generated in do_kernel_configme() is already broken. The problem is that do_kernel_configme() calls native's merge_config.sh script which may call bison and arm-linux-*-gcc to compile kconfig infrastructure to create base .config (scripts/kconfig/conf --allnoconfig Kconfig.) It turns out that we only got above binaries after do_prepare_recipe_sysroot() is called. If those binaries are not available, the merge_config.sh do not generate base .config and without any error produces malfunctioned .config. The build process continues and as a result broken kernel is compiled. To reproduce: bitbake -c cleansstate virtual/kernel bitbake -c kernel_metadata -v virtual/kernel bitbake -c do_kernel_configme -v virtual/kernel (one shall see broken .config in ${B}/.config) bitbake -c do_populate_sysroot -v virtual/kernel bitbake -c do_compile -v virtual/kernel (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d Signed-off-by: Lukasz Majewski --- meta/classes/kernel-yocto.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 100644 --- a/meta/classes/kernel-yocto.bbclass +++ b/meta/classes/kernel-yocto.bbclass @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" +do_kernel_configme[depends] += "virtual/kernel:do_prepare_recipe_sysroot" On a closer look .. what branch is this from ? We already have a change that triggered breakage on 4.17+ that should fix this. i.e. commit ff1bdd75d50f0ebac3d599e461685ace29559a82 in oe-core, adds: -- do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot" +do_kernel_configme[depends] += "bc-native:do_populate_sysroot bison-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" do_kernel_configme() { -- Which I don't see in the context of your patch. I do use poky as in the commit message. Yes, I checked that .. but that isn't a useful part of a patch to oe-core. The kernel-yocto bbclass in that context is not the one in master, and if I run a --contains on that commit ID, I don't get an active branch. So the patch as it stands, cannot be applied to master (and I also checked master-next in case something snuck in that I missed) of oe-core or poky, so there's no way to work with it. Bruce Regarding the kernel - I do have my own recipe; inherit kernel require recipes-kernel/linux/linux-yocto.inc And then I do have: COMPATIBLE_MACHINE_display5 = "display5" KBUILD_DEFCONFIG_pn-linux-yocto-custom_display5 = "imx6q_display5_defconfig" As the board is in mainline I just point to Linus tree. Bruce do_kernel_configme() { set +e Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
Hi Bruce, > On 08/22/2018 09:40 AM, Lukasz Majewski wrote: > > Without this patch it happens that do_populate_recipe_sysroot is > > called just before do_compile (on multi core build machines). > > This is way too late as the .config generated in > > do_kernel_configme() is already broken. > > > > The problem is that do_kernel_configme() calls native's > > merge_config.sh script which may call bison and arm-linux-*-gcc to > > compile kconfig infrastructure to create base .config > > (scripts/kconfig/conf --allnoconfig Kconfig.) > > > > It turns out that we only got above binaries after > > do_prepare_recipe_sysroot() is called. > > If those binaries are not available, the merge_config.sh do not > > generate base .config and without any error produces > > malfunctioned .config. The build process continues and as a result > > broken kernel is compiled. > > > > To reproduce: > > > > bitbake -c cleansstate virtual/kernel > > bitbake -c kernel_metadata -v virtual/kernel > > bitbake -c do_kernel_configme -v virtual/kernel > > (one shall see broken .config in ${B}/.config) > > bitbake -c do_populate_sysroot -v virtual/kernel > > bitbake -c do_compile -v virtual/kernel > > > > (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d > > > > Signed-off-by: Lukasz Majewski > > --- > > meta/classes/kernel-yocto.bbclass | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/meta/classes/kernel-yocto.bbclass > > b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 > > 100644 --- a/meta/classes/kernel-yocto.bbclass > > +++ b/meta/classes/kernel-yocto.bbclass > > @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = > > "kern-tools-native:do_populate_sysroot" > > do_validate_branches[depends] = > > "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] += > > "${S} ${B}" +do_kernel_configme[depends] += > > "virtual/kernel:do_prepare_recipe_sysroot" > > On a closer look .. what branch is this from ? We already have a > change that triggered breakage on 4.17+ that should fix this. > > i.e. commit ff1bdd75d50f0ebac3d599e461685ace29559a82 in oe-core, adds: > > -- > > do_kernel_metadata[depends] = > "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] > = "kern-tools-native:do_populate_sysroot" > > +do_kernel_configme[depends] += > "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot" > +do_kernel_configme[depends] += > "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot" > +do_kernel_configme[depends] += "bc-native:do_populate_sysroot > bison-native:do_populate_sysroot" > do_kernel_configme[dirs] += "${S} ${B}" > do_kernel_configme() { > > -- > > Which I don't see in the context of your patch. > I do use poky as in the commit message. Regarding the kernel - I do have my own recipe; inherit kernel require recipes-kernel/linux/linux-yocto.inc And then I do have: COMPATIBLE_MACHINE_display5 = "display5" KBUILD_DEFCONFIG_pn-linux-yocto-custom_display5 = "imx6q_display5_defconfig" As the board is in mainline I just point to Linus tree. > Bruce > > > do_kernel_configme() { > > set +e > > > > > Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de pgppJVKheNOBq.pgp Description: OpenPGP digital signature -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
Hi Bruce, > On 08/22/2018 09:40 AM, Lukasz Majewski wrote: > > Without this patch it happens that do_populate_recipe_sysroot is > > called just before do_compile (on multi core build machines). > > This is way too late as the .config generated in > > do_kernel_configme() is already broken. > > > > The problem is that do_kernel_configme() calls native's > > merge_config.sh script which may call bison and arm-linux-*-gcc to > > compile kconfig infrastructure to create base .config > > (scripts/kconfig/conf --allnoconfig Kconfig.) > > > > It turns out that we only got above binaries after > > do_prepare_recipe_sysroot() is called. > > If those binaries are not available, the merge_config.sh do not > > generate base .config and without any error produces > > malfunctioned .config. The build process continues and as a result > > broken kernel is compiled. > > > > To reproduce: > > > > bitbake -c cleansstate virtual/kernel > > bitbake -c kernel_metadata -v virtual/kernel > > bitbake -c do_kernel_configme -v virtual/kernel > > (one shall see broken .config in ${B}/.config) > > bitbake -c do_populate_sysroot -v virtual/kernel > > bitbake -c do_compile -v virtual/kernel > > > > I already have a queue of changes for the 4.18 kernel that has a > similar tweak. > > I'll double check the use case above, and if it isn't already > fixed, I'll add this to my queue. One more strange observation the merge_config.sh shall use -native binaries (like bison, gcc, etc). Unfortunately, in my case it depends on target ones (as we do have CFLAGS = ARCH=arm ... merge_configs.sh). The bitbake -g -u taskexp virtual/kernel shows however that we do depend on native bison. To provide target ones - I do need the do_prepare_recipe_sysroot() to call before do_configure. > > Bruce > > > (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d > > > > Signed-off-by: Lukasz Majewski > > --- > > meta/classes/kernel-yocto.bbclass | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/meta/classes/kernel-yocto.bbclass > > b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 > > 100644 --- a/meta/classes/kernel-yocto.bbclass > > +++ b/meta/classes/kernel-yocto.bbclass > > @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = > > "kern-tools-native:do_populate_sysroot" > > do_validate_branches[depends] = > > "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] += > > "${S} ${B}" +do_kernel_configme[depends] += > > "virtual/kernel:do_prepare_recipe_sysroot" do_kernel_configme() { > > set +e > > > > > Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de pgpvXHLp3o_p4.pgp Description: OpenPGP digital signature -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
On 08/22/2018 09:40 AM, Lukasz Majewski wrote: Without this patch it happens that do_populate_recipe_sysroot is called just before do_compile (on multi core build machines). This is way too late as the .config generated in do_kernel_configme() is already broken. The problem is that do_kernel_configme() calls native's merge_config.sh script which may call bison and arm-linux-*-gcc to compile kconfig infrastructure to create base .config (scripts/kconfig/conf --allnoconfig Kconfig.) It turns out that we only got above binaries after do_prepare_recipe_sysroot() is called. If those binaries are not available, the merge_config.sh do not generate base .config and without any error produces malfunctioned .config. The build process continues and as a result broken kernel is compiled. To reproduce: bitbake -c cleansstate virtual/kernel bitbake -c kernel_metadata -v virtual/kernel bitbake -c do_kernel_configme -v virtual/kernel (one shall see broken .config in ${B}/.config) bitbake -c do_populate_sysroot -v virtual/kernel bitbake -c do_compile -v virtual/kernel (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d Signed-off-by: Lukasz Majewski --- meta/classes/kernel-yocto.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 100644 --- a/meta/classes/kernel-yocto.bbclass +++ b/meta/classes/kernel-yocto.bbclass @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" +do_kernel_configme[depends] += "virtual/kernel:do_prepare_recipe_sysroot" On a closer look .. what branch is this from ? We already have a change that triggered breakage on 4.17+ that should fix this. i.e. commit ff1bdd75d50f0ebac3d599e461685ace29559a82 in oe-core, adds: -- do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}binutils:do_populate_sysroot" +do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot" +do_kernel_configme[depends] += "bc-native:do_populate_sysroot bison-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" do_kernel_configme() { -- Which I don't see in the context of your patch. Bruce do_kernel_configme() { set +e -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel: yocto: Add dependency on do_prepare_recipe_sysroot
On 08/22/2018 09:40 AM, Lukasz Majewski wrote: Without this patch it happens that do_populate_recipe_sysroot is called just before do_compile (on multi core build machines). This is way too late as the .config generated in do_kernel_configme() is already broken. The problem is that do_kernel_configme() calls native's merge_config.sh script which may call bison and arm-linux-*-gcc to compile kconfig infrastructure to create base .config (scripts/kconfig/conf --allnoconfig Kconfig.) It turns out that we only got above binaries after do_prepare_recipe_sysroot() is called. If those binaries are not available, the merge_config.sh do not generate base .config and without any error produces malfunctioned .config. The build process continues and as a result broken kernel is compiled. To reproduce: bitbake -c cleansstate virtual/kernel bitbake -c kernel_metadata -v virtual/kernel bitbake -c do_kernel_configme -v virtual/kernel (one shall see broken .config in ${B}/.config) bitbake -c do_populate_sysroot -v virtual/kernel bitbake -c do_compile -v virtual/kernel I already have a queue of changes for the 4.18 kernel that has a similar tweak. I'll double check the use case above, and if it isn't already fixed, I'll add this to my queue. Bruce (Poky) SHA1: bb91b2ae3ee5cf108aa2f9b78abb14d5aa00831d Signed-off-by: Lukasz Majewski --- meta/classes/kernel-yocto.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass index 4ac3a39e47..19a3f2bc46 100644 --- a/meta/classes/kernel-yocto.bbclass +++ b/meta/classes/kernel-yocto.bbclass @@ -276,6 +276,7 @@ do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot" do_validate_branches[depends] = "kern-tools-native:do_populate_sysroot" do_kernel_configme[dirs] += "${S} ${B}" +do_kernel_configme[depends] += "virtual/kernel:do_prepare_recipe_sysroot" do_kernel_configme() { set +e -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core