Re: [OE-core] [PATCH v2] kernel: Add support for multiple kernel packages

2016-02-03 Thread Diego Sueiro 
Haris,

On 2 February 2016 at 20:39, Haris Okanovic  wrote:

> Here's a simple recipe to make a debug version linux-yocto. It includes
> the real recipe then overrides KERNEL_PACKAGE_NAME and PROVIDES so that two
> sets of kernel* packages can co-exist in the feed. The other file is a
> kernel config snippet which enables some extra debug options.
>
> [linux-yocto-debug_3.19.bb]
> include linux-yocto_3.19.bb
> KERNEL_PACKAGE_NAME = "kernel-debug"
> PROVIDES = ""
> SRC_URI += "file://debug.cfg"
>
> [files/debug.cfg]
> CONFIG_DEBUG_INFO=y
> CONFIG_FRAME_POINTER=y
> CONFIG_LOCKUP_DETECTOR=y
> CONFIG_DEBUG_PREEMPT=y
> CONFIG_KGDB=y
> CONFIG_KGDB_KDB=y
> CONFIG_EARLY_PRINTK=y
>

Thanks for the info.

Nice job, hope that it can be up-streamed since this use case is quite
common in a product development life cycle.

Regards,

--
*dS
Diego Sueiro

Administrador do Embarcados
www.embarcados.com.br


/*long live rock 'n roll*/
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel: Add support for multiple kernel packages

2016-02-02 Thread Haris Okanovic 

Hi Diego,

On 02/02/2016 10:12 AM, Diego Sueiro wrote:

Haris,


On 19 January 2016 at 22:40, Haris Okanovic > wrote:

Any concerns pulling this into master as-is?

We briefly discussed the possibility of building out-of-tree modules
against multiple kernels back in December, right before this thread
died. This change certainly doesn't preclude that improvement if someone
would like to implement it, nor does it change current behavior.
More importantly, it provides a useful function as-is in our opinion --
E.g. to provide a debug kernel in a feed for testing/development
activities.

Thread:
http://thread.gmane.org/gmane.comp.handhelds.openembedded.core/72908



Could you please give examples how kernel recipes look like?



Here's a simple recipe to make a debug version linux-yocto. It includes 
the real recipe then overrides KERNEL_PACKAGE_NAME and PROVIDES so that 
two sets of kernel* packages can co-exist in the feed. The other file is 
a kernel config snippet which enables some extra debug options.


[linux-yocto-debug_3.19.bb]
include linux-yocto_3.19.bb
KERNEL_PACKAGE_NAME = "kernel-debug"
PROVIDES = ""
SRC_URI += "file://debug.cfg"

[files/debug.cfg]
CONFIG_DEBUG_INFO=y
CONFIG_FRAME_POINTER=y
CONFIG_LOCKUP_DETECTOR=y
CONFIG_DEBUG_PREEMPT=y
CONFIG_KGDB=y
CONFIG_KGDB_KDB=y
CONFIG_EARLY_PRINTK=y


I'm particularly interested in this feature.


Regards,

--
*dS
Diego Sueiro

Administrador do Embarcados
www.embarcados.com.br


/*long live rock 'n roll*/

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel: Add support for multiple kernel packages

2016-02-02 Thread Diego Sueiro 
Haris,


On 19 January 2016 at 22:40, Haris Okanovic  wrote:

> Any concerns pulling this into master as-is?
>
> We briefly discussed the possibility of building out-of-tree modules
> against multiple kernels back in December, right before this thread
> died. This change certainly doesn't preclude that improvement if someone
> would like to implement it, nor does it change current behavior.
> More importantly, it provides a useful function as-is in our opinion --
> E.g. to provide a debug kernel in a feed for testing/development
> activities.
>
> Thread:
> http://thread.gmane.org/gmane.comp.handhelds.openembedded.core/72908
>


Could you please give examples how kernel recipes look like?

I'm particularly interested in this feature.


Regards,

--
*dS
Diego Sueiro

Administrador do Embarcados
www.embarcados.com.br


/*long live rock 'n roll*/
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel: Add support for multiple kernel packages

2016-01-19 Thread Haris Okanovic 

Any concerns pulling this into master as-is?

We briefly discussed the possibility of building out-of-tree modules
against multiple kernels back in December, right before this thread
died. This change certainly doesn't preclude that improvement if someone
would like to implement it, nor does it change current behavior.
More importantly, it provides a useful function as-is in our opinion --
E.g. to provide a debug kernel in a feed for testing/development activities.

Thread: http://thread.gmane.org/gmane.comp.handhelds.openembedded.core/72908

Thanks,
Haris


On 12/14/2015 04:35 PM, Haris Okanovic wrote:

From: Gratian Crisan 

This change allows distributions to provide multiple builds of kernel
and module packages. For example, a distro may want provide an alternate
debug kernel in a package feed for debug/testing.

Templetize kernel package name:

Add a "weak" variable KERNEL_PACKAGE_NAME used as the base name for kernel
packages. It defaults to the old hard-coded name 'kernel' but it can be
redefined by recipes that provide additional kernel packages in order
to avoid build conflicts.

Change hard-coded 'kernel' references to KERNEL_PACKAGE_NAME in
kernel bbclass-es.

Build alternate kernels from WORKDIR instead of STAGING_KERNEL_DIR:

Prior to this change, kernel recipes would all fetch source to
STAGING_KERNEL_DIR as defined by bitbake/distro confs. This broke
parallel builds when more than one kernel recipes are defined in
the distribution, since they all attempted to fetch() and patch()
in a shared source dir.

With this change, alternate kernel recipes fetch source into their
${WORKDIR} so that they may build in parallel to each other and the
default kernel recipe, which still fetches to STAGING_KERNEL_DIR.

Testing:

Built linux-yocto-4.1.13 for qemux86 and verified it produced kernel
image and modules packages. Added kernel recipe with non-default
KERNEL_PACKAGE_NAME and verified it produces kernel image and modules
packages with alternate name next to default kernel.

Signed-off-by: Gratian Crisan 
Signed-off-by: Haris Okanovic 
Coauthored-by: Haris Okanovic 
Coauthored-by: Josh Hernstrom 
Natinst-ReviewBoard-ID: 120348
Natinst-ReviewBoard-ID: 120447
---
  meta/classes/kernel-module-split.bbclass |  9 ++--
  meta/classes/kernel.bbclass  | 76 
  meta/conf/documentation.conf |  1 +
  meta/recipes-kernel/linux/linux-dtb.inc  |  2 +-
  4 files changed, 55 insertions(+), 33 deletions(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index e1a70e6..7415ec8 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -28,7 +28,7 @@ do_install_append() {

  PACKAGESPLITFUNCS_prepend = "split_kernel_module_packages "

-KERNEL_MODULES_META_PACKAGE ?= "kernel-modules"
+KERNEL_MODULES_META_PACKAGE ?= "${KERNEL_PACKAGE_NAME}-modules"

  python split_kernel_module_packages () {
  import re
@@ -179,14 +179,17 @@ python split_kernel_module_packages () {
  # Avoid automatic -dev recommendations for modules ending with -dev.
  d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1)

+kernel_package_name = d.getVar("KERNEL_PACKAGE_NAME", True)
+kernel_version = d.getVar("KERNEL_VERSION", True)
+
  module_deps = parse_depmod()
  module_regex = '^(.*)\.k?o$'
-module_pattern = 'kernel-module-%s'
+module_pattern = '%s-module-%%s' % kernel_package_name

  postinst = d.getVar('pkg_postinst_modules', True)
  postrm = d.getVar('pkg_postrm_modules', True)

-modules = do_split_packages(d, root='/lib/modules', file_regex=module_regex, 
output_pattern=module_pattern, description='%s kernel module', postinst=postinst, 
postrm=postrm, recursive=True, hook=frob_metadata, extra_depends='kernel-%s' % 
(d.getVar("KERNEL_VERSION", True)))
+modules = do_split_packages(d, root='/lib/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='%s-%s' % (kernel_package_name, kernel_version))
  if modules:
  metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE', True)
  d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules))
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index b75a462..281519e 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -1,5 +1,23 @@
  inherit linux-kernel-base kernel-module-split

+KERNEL_PACKAGE_NAME ??= "kernel"
+
+# The default kernel recipe builds in a shared location defined by
+# bitbake/distro confs: STAGING_KERNEL_DIR and STAGING_KERNEL_BUILDDIR.
+# Set these variables to directories under ${WORKDIR} in alternate
+# kernel recipes (I.e. where KERNEL_PACKAGE_NAME != kernel) so that they
+# may build

Re: [OE-core] [PATCH v2] kernel: Add support for multiple kernel packages

2015-12-15 Thread Haris Okanovic 

Hi Khem,

On 12/14/2015 06:33 PM, Khem Raj wrote:



On Dec 14, 2015, at 2:35 PM, Haris Okanovic  wrote:

From: Gratian Crisan 

This change allows distributions to provide multiple builds of kernel
and module packages. For example, a distro may want provide an alternate
debug kernel in a package feed for debug/testing.

Templetize kernel package name:

Add a "weak" variable KERNEL_PACKAGE_NAME used as the base name for kernel
packages. It defaults to the old hard-coded name 'kernel' but it can be
redefined by recipes that provide additional kernel packages in order
to avoid build conflicts.

Change hard-coded 'kernel' references to KERNEL_PACKAGE_NAME in
kernel bbclass-es.

Build alternate kernels from WORKDIR instead of STAGING_KERNEL_DIR:

Prior to this change, kernel recipes would all fetch source to
STAGING_KERNEL_DIR as defined by bitbake/distro confs. This broke
parallel builds when more than one kernel recipes are defined in
the distribution, since they all attempted to fetch() and patch()
in a shared source dir.

With this change, alternate kernel recipes fetch source into their
${WORKDIR} so that they may build in parallel to each other and the
default kernel recipe, which still fetches to STAGING_KERNEL_DIR.

Testing:

Built linux-yocto-4.1.13 for qemux86 and verified it produced kernel
image and modules packages. Added kernel recipe with non-default
KERNEL_PACKAGE_NAME and verified it produces kernel image and modules
packages with alternate name next to default kernel.


is it all targeted at just kernel package ? can you cite some use cases which 
benefit from
such a thing and what are current alternatives ?



The ultimate goal is to improve developer efficiency:

We're looking to provide an alternate debug kernel+modules in our feed, 
which our developers could easily install to diagnose problems or run 
tests without having to build kernels on their desktop. E.g. Our debug 
kernel would have options like CONFIG_DEBUG_INFO, CONFIG_DEBUG_PREEMPT, 
and CONFIG_FRAME_POINTER enabled. We may provide more variants in the 
future tuned to specific use cases.


We need all kernel+module variants to co-exist both in the feed and 
on-target, so that users may quickly install different variants and 
toggle between them as needed. The only solution we have to that end is 
to change the hard-coded package prefix "kernel" throughout 
kernel.bbclass & friends. The proposed change makes that prefix 
configurable by the recipe.


I'm not sure how else we could do this without rewriting a lot of 
kernel.bbclass functionality. We're certainly open to suggestions.



Denys Dmytriyenko also brought up an interesting point in a separate 
email: "how can you build/rebuild and package out-of-tree modules 
against all the alternatives?"


kernel.bbclass builds under work-shared, which is what I presume enables 
building out-of-tree modules against the default distro kernel. Please 
correct me if I'm wrong on that point. This change overrides the build 
location of alternate kernel recipes
(where KERNEL_PACKAGE_NAME != kernel) to WORKDIR, which likely makes it 
impossible to build against the alternatives.


This is not something we've considered in this change since we build our 
out-of-tree modules outside of OE entirely. I'm not sure what it would 
take to make this happen, but I'd prefer to do it in a separate change, 
so long this change doesn't preclude that eventual improvement.



-- Haris




Signed-off-by: Gratian Crisan 
Signed-off-by: Haris Okanovic 
Coauthored-by: Haris Okanovic 
Coauthored-by: Josh Hernstrom 
Natinst-ReviewBoard-ID: 120348
Natinst-ReviewBoard-ID: 120447
---
meta/classes/kernel-module-split.bbclass |  9 ++--
meta/classes/kernel.bbclass  | 76 
meta/conf/documentation.conf |  1 +
meta/recipes-kernel/linux/linux-dtb.inc  |  2 +-
4 files changed, 55 insertions(+), 33 deletions(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index e1a70e6..7415ec8 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -28,7 +28,7 @@ do_install_append() {

PACKAGESPLITFUNCS_prepend = "split_kernel_module_packages "

-KERNEL_MODULES_META_PACKAGE ?= "kernel-modules"
+KERNEL_MODULES_META_PACKAGE ?= "${KERNEL_PACKAGE_NAME}-modules"

python split_kernel_module_packages () {
 import re
@@ -179,14 +179,17 @@ python split_kernel_module_packages () {
 # Avoid automatic -dev recommendations for modules ending with -dev.
 d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1)

+kernel_package_name = d.getVar("KERNEL_PACKAGE_NAME", True)
+kernel_version = d.getVar("KERNEL_VERSION", True)
+
 module_deps = parse_depmod()
 module_regex = '^(.*)\.k?o$'
-module_pattern = 'kernel-module-%s'
+

Re: [OE-core] [PATCH v2] kernel: Add support for multiple kernel packages

2015-12-14 Thread Khem Raj 

> On Dec 14, 2015, at 2:35 PM, Haris Okanovic  wrote:
> 
> From: Gratian Crisan 
> 
> This change allows distributions to provide multiple builds of kernel
> and module packages. For example, a distro may want provide an alternate
> debug kernel in a package feed for debug/testing.
> 
> Templetize kernel package name:
> 
> Add a "weak" variable KERNEL_PACKAGE_NAME used as the base name for kernel
> packages. It defaults to the old hard-coded name 'kernel' but it can be
> redefined by recipes that provide additional kernel packages in order
> to avoid build conflicts.
> 
> Change hard-coded 'kernel' references to KERNEL_PACKAGE_NAME in
> kernel bbclass-es.
> 
> Build alternate kernels from WORKDIR instead of STAGING_KERNEL_DIR:
> 
> Prior to this change, kernel recipes would all fetch source to
> STAGING_KERNEL_DIR as defined by bitbake/distro confs. This broke
> parallel builds when more than one kernel recipes are defined in
> the distribution, since they all attempted to fetch() and patch()
> in a shared source dir.
> 
> With this change, alternate kernel recipes fetch source into their
> ${WORKDIR} so that they may build in parallel to each other and the
> default kernel recipe, which still fetches to STAGING_KERNEL_DIR.
> 
> Testing:
> 
> Built linux-yocto-4.1.13 for qemux86 and verified it produced kernel
> image and modules packages. Added kernel recipe with non-default
> KERNEL_PACKAGE_NAME and verified it produces kernel image and modules
> packages with alternate name next to default kernel.

is it all targeted at just kernel package ? can you cite some use cases which 
benefit from
such a thing and what are current alternatives ?

> 
> Signed-off-by: Gratian Crisan 
> Signed-off-by: Haris Okanovic 
> Coauthored-by: Haris Okanovic 
> Coauthored-by: Josh Hernstrom 
> Natinst-ReviewBoard-ID: 120348
> Natinst-ReviewBoard-ID: 120447
> ---
> meta/classes/kernel-module-split.bbclass |  9 ++--
> meta/classes/kernel.bbclass  | 76 
> meta/conf/documentation.conf |  1 +
> meta/recipes-kernel/linux/linux-dtb.inc  |  2 +-
> 4 files changed, 55 insertions(+), 33 deletions(-)
> 
> diff --git a/meta/classes/kernel-module-split.bbclass 
> b/meta/classes/kernel-module-split.bbclass
> index e1a70e6..7415ec8 100644
> --- a/meta/classes/kernel-module-split.bbclass
> +++ b/meta/classes/kernel-module-split.bbclass
> @@ -28,7 +28,7 @@ do_install_append() {
> 
> PACKAGESPLITFUNCS_prepend = "split_kernel_module_packages "
> 
> -KERNEL_MODULES_META_PACKAGE ?= "kernel-modules"
> +KERNEL_MODULES_META_PACKAGE ?= "${KERNEL_PACKAGE_NAME}-modules"
> 
> python split_kernel_module_packages () {
> import re
> @@ -179,14 +179,17 @@ python split_kernel_module_packages () {
> # Avoid automatic -dev recommendations for modules ending with -dev.
> d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1)
> 
> +kernel_package_name = d.getVar("KERNEL_PACKAGE_NAME", True)
> +kernel_version = d.getVar("KERNEL_VERSION", True)
> +
> module_deps = parse_depmod()
> module_regex = '^(.*)\.k?o$'
> -module_pattern = 'kernel-module-%s'
> +module_pattern = '%s-module-%%s' % kernel_package_name
> 
> postinst = d.getVar('pkg_postinst_modules', True)
> postrm = d.getVar('pkg_postrm_modules', True)
> 
> -modules = do_split_packages(d, root='/lib/modules', 
> file_regex=module_regex, output_pattern=module_pattern, description='%s 
> kernel module', postinst=postinst, postrm=postrm, recursive=True, 
> hook=frob_metadata, extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION", 
> True)))
> +modules = do_split_packages(d, root='/lib/modules', 
> file_regex=module_regex, output_pattern=module_pattern, description='%s 
> kernel module', postinst=postinst, postrm=postrm, recursive=True, 
> hook=frob_metadata, extra_depends='%s-%s' % (kernel_package_name, 
> kernel_version))
> if modules:
> metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE', True)
> d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules))
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index b75a462..281519e 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -1,5 +1,23 @@
> inherit linux-kernel-base kernel-module-split
> 
> +KERNEL_PACKAGE_NAME ??= "kernel"
> +
> +# The default kernel recipe builds in a shared location defined by
> +# bitbake/distro confs: STAGING_KERNEL_DIR and STAGING_KERNEL_BUILDDIR.
> +# Set these variables to directories under ${WORKDIR} in alternate
> +# kernel recipes (I.e. where KERNEL_PACKAGE_NAME != kernel) so that they
> +# may build in parallel with the default kernel without clobbering.
> +python __anonymous () {
> +if d.getVar("KERNEL_PACKAGE_NAME", True) != "kernel":
> +workdir = d.getVar("WORKDIR", True)
> +
> +