Re: [OE-core] [PATCH 2/2] multiconfig: Enable multiconfig dependencies on oe-core

2018-08-01 Thread Alejandro Enedino Hernandez Samaniego 

Hey Khem, Richard,


On 08/01/2018 10:44 AM, Khem Raj wrote:



On Tue, Jul 31, 2018 at 4:59 AM Richard Purdie 
> wrote:


On Mon, 2018-07-30 at 15:05 -0700, Alejandro Enedino Hernandez
Samaniego wrote:
> On 07/26/2018 08:36 PM, Khem Raj wrote:
> >
> > On 7/25/18 9:05 AM, Alejandro Enedino Hernandez Samaniego wrote:
> > > This patch enables multiconfig dependencies (mcdepends) to be
> > > used on
> > > recipes using the following format:
> > >
> > > task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-
> > > on"
> > >
> > > For the sake of simplicity consider the following example:
> > >
> > > Assuming we have set up multiconfig builds, one for qemux86 and
> > > one for
> > > qemuarm, named x86 and arm respectively.
> > >
> > > Adding the following line to an image recipe (core-image-sato):
> > > do_image[mcdepends] = "multiconfig:x86:arm:core-image-
> > > minimal:do_rootfs"
> > >
> >
> > Do we really need to add multiconfig to keyword namespace ? can we
> > deduce this from "arch1:arch2:..."
>
> Bitbake already uses this and I believed adopting it would make it
> easier for the user to understand its usage.
>
> For example, since the multiconfig names are also provided during
> setup, if you assume that someone set up the build and a different
> user is going through the recipe, arch1:arch2 wouldn't mean a
lot for
> that user because they may not map 1:1 to MACHINE names.

It is consistent with the rest of the system. I'm still torn on
whether
we should shorten "multiconfig:" to "mc:" but we need to be
consistent.


Given a choice I would suggest for using multiconfig for lesser chance 
of conflicts

I also agree here that its better to use multiconfig.



> > secondy, this need to be well documented, with examples for users
>
> I agree we could add more documentation to it, where do you
think it
> should go?

I'm going to queue the patches for testing in -next, they look good to
me just reading the patches.

Could you start to work with Scott (cc'd) to get this new dependency
type listed in the bitbake manual and anywhere else it needs to be
documented?


Sure, I'll sync with Scott

Thanks!,

Alejandro



Its great to see this finally working!

Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] multiconfig: Enable multiconfig dependencies on oe-core

2018-08-01 Thread Khem Raj 
On Tue, Jul 31, 2018 at 4:59 AM Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> On Mon, 2018-07-30 at 15:05 -0700, Alejandro Enedino Hernandez
> Samaniego wrote:
> > On 07/26/2018 08:36 PM, Khem Raj wrote:
> > >
> > > On 7/25/18 9:05 AM, Alejandro Enedino Hernandez Samaniego wrote:
> > > > This patch enables multiconfig dependencies (mcdepends) to be
> > > > used on
> > > > recipes using the following format:
> > > >
> > > > task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-
> > > > on"
> > > >
> > > > For the sake of simplicity consider the following example:
> > > >
> > > > Assuming we have set up multiconfig builds, one for qemux86 and
> > > > one for
> > > > qemuarm, named x86 and arm respectively.
> > > >
> > > > Adding the following line to an image recipe (core-image-sato):
> > > > do_image[mcdepends] = "multiconfig:x86:arm:core-image-
> > > > minimal:do_rootfs"
> > > >
> > >
> > > Do we really need to add multiconfig to keyword namespace ? can we
> > > deduce this from "arch1:arch2:..."
> >
> > Bitbake already uses this and I believed adopting it would make it
> > easier for the user to understand its usage.
> >
> > For example, since the multiconfig names are also provided during
> > setup, if you assume that someone set up the build and a different
> > user is going through the recipe, arch1:arch2 wouldn't mean a lot for
> > that user because they may not map 1:1 to MACHINE names.
>
> It is consistent with the rest of the system. I'm still torn on whether
> we should shorten "multiconfig:" to "mc:" but we need to be consistent.
>

Given a choice I would suggest for using multiconfig for lesser chance of
conflicts

>
> > > secondy, this need to be well documented, with examples for users
> >
> > I agree we could add more documentation to it, where do you think it
> > should go?
>
> I'm going to queue the patches for testing in -next, they look good to
> me just reading the patches.
>
> Could you start to work with Scott (cc'd) to get this new dependency
> type listed in the bitbake manual and anywhere else it needs to be
> documented?
>
> Its great to see this finally working!
>
> Cheers,
>
> Richard
>
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] multiconfig: Enable multiconfig dependencies on oe-core

2018-07-31 Thread Richard Purdie 
On Mon, 2018-07-30 at 15:05 -0700, Alejandro Enedino Hernandez
Samaniego wrote:
> On 07/26/2018 08:36 PM, Khem Raj wrote:
> > 
> > On 7/25/18 9:05 AM, Alejandro Enedino Hernandez Samaniego wrote:
> > > This patch enables multiconfig dependencies (mcdepends) to be
> > > used on
> > > recipes using the following format:
> > > 
> > > task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-
> > > on"
> > > 
> > > For the sake of simplicity consider the following example:
> > > 
> > > Assuming we have set up multiconfig builds, one for qemux86 and
> > > one for
> > > qemuarm, named x86 and arm respectively.
> > > 
> > > Adding the following line to an image recipe (core-image-sato):
> > > do_image[mcdepends] = "multiconfig:x86:arm:core-image-
> > > minimal:do_rootfs"
> > > 
> > 
> > Do we really need to add multiconfig to keyword namespace ? can we
> > deduce this from "arch1:arch2:..."
> 
> Bitbake already uses this and I believed adopting it would make it 
> easier for the user to understand its usage.
>
> For example, since the multiconfig names are also provided during
> setup, if you assume that someone set up the build and a different
> user is going through the recipe, arch1:arch2 wouldn't mean a lot for
> that user because they may not map 1:1 to MACHINE names.

It is consistent with the rest of the system. I'm still torn on whether
we should shorten "multiconfig:" to "mc:" but we need to be consistent.

> > secondy, this need to be well documented, with examples for users
> 
> I agree we could add more documentation to it, where do you think it 
> should go?

I'm going to queue the patches for testing in -next, they look good to
me just reading the patches.

Could you start to work with Scott (cc'd) to get this new dependency
type listed in the bitbake manual and anywhere else it needs to be
documented?

Its great to see this finally working!

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] multiconfig: Enable multiconfig dependencies on oe-core

2018-07-30 Thread Alejandro Enedino Hernandez Samaniego 

Hey Khem,


On 07/26/2018 08:36 PM, Khem Raj wrote:


On 7/25/18 9:05 AM, Alejandro Enedino Hernandez Samaniego wrote:

This patch enables multiconfig dependencies (mcdepends) to be used on
recipes using the following format:

task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on"

For the sake of simplicity consider the following example:

Assuming we have set up multiconfig builds, one for qemux86 and one for
qemuarm, named x86 and arm respectively.

Adding the following line to an image recipe (core-image-sato):
do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs"


Do we really need to add multiconfig to keyword namespace ? can we
deduce this from "arch1:arch2:..."
Bitbake already uses this and I believed adopting it would make it 
easier for the user to understand its usage.


For example, since the multiconfig names are also provided during setup, 
if you assume that someone set up
the build and a different user is going through the recipe, arch1:arch2 
wouldn't mean a lot for that user

because they may not map 1:1 to MACHINE names.



secondy, this need to be well documented, with examples for users
I agree we could add more documentation to it, where do you think it 
should go?



Would state that core-image-sato:do_image from x86 will depend on
core-image-minimal:do_rootfs from arm so it can be executed.

This patch makes modifications to bitbake.conf to enable mcdepends, and
to sstatesig and staging.bbclass to avoid conflicts between packages from
different multiconfigs.

[YOCTO #10681]

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
  meta/classes/staging.bbclass | 8 +++-
  meta/conf/bitbake.conf   | 2 +-
  meta/lib/oe/sstatesig.py | 9 -
  3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index 41df883..27b012e 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -256,7 +256,7 @@ python extend_recipe_sysroot() {
  workdir = d.getVar("WORKDIR")
  #bb.warn(str(taskdepdata))
  pn = d.getVar("PN")
-
+mc = d.getVar("BB_CURRENT_MC")
  stagingdir = d.getVar("STAGING_DIR")
  sharedmanifests = d.getVar("COMPONENTS_DIR") + "/manifests"
  recipesysroot = d.getVar("RECIPE_SYSROOT")
@@ -443,7 +443,13 @@ python extend_recipe_sysroot() {
  
  msg_exists = []

  msg_adding = []
+
  for dep in configuredeps:
+if mc != 'default':
+# We should not care about other multiconfigs
+depmc = dep.split(':')[1]
+if depmc != mc:
+continue
  c = setscenedeps[dep][0]
  if c not in installed:
  continue
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index e28f3c7..f68954c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -877,7 +877,7 @@ BB_SIGNATURE_EXCLUDE_FLAGS ?= "doc deps depends \
  file-checksums python func task export unexport noexec nostamp dirs 
cleandirs \
  sstate-lockfile-shared prefuncs postfuncs export_func deptask rdeptask \
  recrdeptask nodeprrecs stamp-extra-info sstate-outputdirs filename lineno 
\
-progress"
+progress mcdepends"
  
  MLPREFIX ??= ""

  MULTILIB_VARIANTS ??= ""
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 5dcc2f5..18c5a35 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -150,16 +150,23 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
  if recipename in self.unlockedrecipes:
  unlocked = True
  else:
+def get_mc(tid):
+tid = tid.rsplit('.', 1)[0]
+if tid.startswith('multiconfig:'):
+elems = tid.split(':')
+return elems[1]
  def recipename_from_dep(dep):
  # The dep entry will look something like
  # /path/path/recipename.bb.task, 
virtual:native:/p/foo.bb.task,
  # ...
+
  fn = dep.rsplit('.', 1)[0]
  return dataCache.pkg_fn[fn]
  
+mc = get_mc(fn)

  # If any unlocked recipe is in the direct dependencies then the
  # current recipe should be unlocked as well.
-depnames = [ recipename_from_dep(x) for x in deps ]
+depnames = [ recipename_from_dep(x) for x in deps if mc == 
get_mc(x)]
  if any(x in y for y in depnames for x in self.unlockedrecipes):
  self.unlockedrecipes[recipename] = ''
  unlocked = True


Cheers,

Alejandro
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] multiconfig: Enable multiconfig dependencies on oe-core

2018-07-26 Thread Khem Raj 


On 7/25/18 9:05 AM, Alejandro Enedino Hernandez Samaniego wrote:
> This patch enables multiconfig dependencies (mcdepends) to be used on
> recipes using the following format:
> 
> task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on"
> 
> For the sake of simplicity consider the following example:
> 
> Assuming we have set up multiconfig builds, one for qemux86 and one for
> qemuarm, named x86 and arm respectively.
> 
> Adding the following line to an image recipe (core-image-sato):
> do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs"
> 

Do we really need to add multiconfig to keyword namespace ? can we
deduce this from "arch1:arch2:..."

secondy, this need to be well documented, with examples for users

> Would state that core-image-sato:do_image from x86 will depend on
> core-image-minimal:do_rootfs from arm so it can be executed.
> 
> This patch makes modifications to bitbake.conf to enable mcdepends, and
> to sstatesig and staging.bbclass to avoid conflicts between packages from
> different multiconfigs.
> 
> [YOCTO #10681]
> 
> Signed-off-by: Alejandro Enedino Hernandez Samaniego 
> ---
>  meta/classes/staging.bbclass | 8 +++-
>  meta/conf/bitbake.conf   | 2 +-
>  meta/lib/oe/sstatesig.py | 9 -
>  3 files changed, 16 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
> index 41df883..27b012e 100644
> --- a/meta/classes/staging.bbclass
> +++ b/meta/classes/staging.bbclass
> @@ -256,7 +256,7 @@ python extend_recipe_sysroot() {
>  workdir = d.getVar("WORKDIR")
>  #bb.warn(str(taskdepdata))
>  pn = d.getVar("PN")
> -
> +mc = d.getVar("BB_CURRENT_MC")
>  stagingdir = d.getVar("STAGING_DIR")
>  sharedmanifests = d.getVar("COMPONENTS_DIR") + "/manifests"
>  recipesysroot = d.getVar("RECIPE_SYSROOT")
> @@ -443,7 +443,13 @@ python extend_recipe_sysroot() {
>  
>  msg_exists = []
>  msg_adding = []
> +
>  for dep in configuredeps:
> +if mc != 'default':
> +# We should not care about other multiconfigs
> +depmc = dep.split(':')[1]
> +if depmc != mc:
> +continue
>  c = setscenedeps[dep][0]
>  if c not in installed:
>  continue
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index e28f3c7..f68954c 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -877,7 +877,7 @@ BB_SIGNATURE_EXCLUDE_FLAGS ?= "doc deps depends \
>  file-checksums python func task export unexport noexec nostamp dirs 
> cleandirs \
>  sstate-lockfile-shared prefuncs postfuncs export_func deptask rdeptask \
>  recrdeptask nodeprrecs stamp-extra-info sstate-outputdirs filename 
> lineno \
> -progress"
> +progress mcdepends"
>  
>  MLPREFIX ??= ""
>  MULTILIB_VARIANTS ??= ""
> diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
> index 5dcc2f5..18c5a35 100644
> --- a/meta/lib/oe/sstatesig.py
> +++ b/meta/lib/oe/sstatesig.py
> @@ -150,16 +150,23 @@ class 
> SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
>  if recipename in self.unlockedrecipes:
>  unlocked = True
>  else:
> +def get_mc(tid):
> +tid = tid.rsplit('.', 1)[0]
> +if tid.startswith('multiconfig:'):
> +elems = tid.split(':')
> +return elems[1]
>  def recipename_from_dep(dep):
>  # The dep entry will look something like
>  # /path/path/recipename.bb.task, 
> virtual:native:/p/foo.bb.task,
>  # ...
> +
>  fn = dep.rsplit('.', 1)[0]
>  return dataCache.pkg_fn[fn]
>  
> +mc = get_mc(fn)
>  # If any unlocked recipe is in the direct dependencies then the
>  # current recipe should be unlocked as well.
> -depnames = [ recipename_from_dep(x) for x in deps ]
> +depnames = [ recipename_from_dep(x) for x in deps if mc == 
> get_mc(x)]
>  if any(x in y for y in depnames for x in self.unlockedrecipes):
>  self.unlockedrecipes[recipename] = ''
>  unlocked = True
> 



signature.asc
Description: OpenPGP digital signature
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] multiconfig: Enable multiconfig dependencies on oe-core

2018-07-25 Thread Alejandro Enedino Hernandez Samaniego 
This patch enables multiconfig dependencies (mcdepends) to be used on
recipes using the following format:

task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on"

For the sake of simplicity consider the following example:

Assuming we have set up multiconfig builds, one for qemux86 and one for
qemuarm, named x86 and arm respectively.

Adding the following line to an image recipe (core-image-sato):
do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs"

Would state that core-image-sato:do_image from x86 will depend on
core-image-minimal:do_rootfs from arm so it can be executed.

This patch makes modifications to bitbake.conf to enable mcdepends, and
to sstatesig and staging.bbclass to avoid conflicts between packages from
different multiconfigs.

[YOCTO #10681]

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/classes/staging.bbclass | 8 +++-
 meta/conf/bitbake.conf   | 2 +-
 meta/lib/oe/sstatesig.py | 9 -
 3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index 41df883..27b012e 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -256,7 +256,7 @@ python extend_recipe_sysroot() {
 workdir = d.getVar("WORKDIR")
 #bb.warn(str(taskdepdata))
 pn = d.getVar("PN")
-
+mc = d.getVar("BB_CURRENT_MC")
 stagingdir = d.getVar("STAGING_DIR")
 sharedmanifests = d.getVar("COMPONENTS_DIR") + "/manifests"
 recipesysroot = d.getVar("RECIPE_SYSROOT")
@@ -443,7 +443,13 @@ python extend_recipe_sysroot() {
 
 msg_exists = []
 msg_adding = []
+
 for dep in configuredeps:
+if mc != 'default':
+# We should not care about other multiconfigs
+depmc = dep.split(':')[1]
+if depmc != mc:
+continue
 c = setscenedeps[dep][0]
 if c not in installed:
 continue
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index e28f3c7..f68954c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -877,7 +877,7 @@ BB_SIGNATURE_EXCLUDE_FLAGS ?= "doc deps depends \
 file-checksums python func task export unexport noexec nostamp dirs 
cleandirs \
 sstate-lockfile-shared prefuncs postfuncs export_func deptask rdeptask \
 recrdeptask nodeprrecs stamp-extra-info sstate-outputdirs filename lineno \
-progress"
+progress mcdepends"
 
 MLPREFIX ??= ""
 MULTILIB_VARIANTS ??= ""
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 5dcc2f5..18c5a35 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -150,16 +150,23 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
 if recipename in self.unlockedrecipes:
 unlocked = True
 else:
+def get_mc(tid):
+tid = tid.rsplit('.', 1)[0]
+if tid.startswith('multiconfig:'):
+elems = tid.split(':')
+return elems[1]
 def recipename_from_dep(dep):
 # The dep entry will look something like
 # /path/path/recipename.bb.task, virtual:native:/p/foo.bb.task,
 # ...
+
 fn = dep.rsplit('.', 1)[0]
 return dataCache.pkg_fn[fn]
 
+mc = get_mc(fn)
 # If any unlocked recipe is in the direct dependencies then the
 # current recipe should be unlocked as well.
-depnames = [ recipename_from_dep(x) for x in deps ]
+depnames = [ recipename_from_dep(x) for x in deps if mc == 
get_mc(x)]
 if any(x in y for y in depnames for x in self.unlockedrecipes):
 self.unlockedrecipes[recipename] = ''
 unlocked = True
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core