[OE-core] [PATCH v4] kernel-module-split: Append KERNEL_VERSION string to kernel module name

[ola . redell]

From: Ola Redell <ola.red...@retotech.se>

The KERNEL_VERSION string is added to kernel module package names in order
to make the kernel modules for different kernel versions distinct packages
instead of different versions of the same package. With this change, when
a new kernel is installed together with its kernel modules (e.g. by upgrade
of the packages kernel and kernel-modules) using some package manager such
as apt-get or rpm, the kernel modules for the older kernel will not be
removed. This enables a fall back to the older kernel if the new one fails.

Also, for backwards compatibility and to enable kernel version agnostic
dependencies to kernel modules, create a virtual package with the old
(shorter) kernel module package name using RPROVIDES.

Signed-off-by: Ola Redell <ola.red...@retotech.se>
---
 meta/classes/kernel-module-split.bbclass | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index efe1b42..742320c 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -31,6 +31,7 @@ PACKAGESPLITFUNCS_prepend = "split_kernel_module_packages "
 KERNEL_MODULES_META_PACKAGE ?= "kernel-modules"
 
 KERNEL_MODULE_PACKAGE_PREFIX ?= ""
+KERNEL_MODULE_PROVIDE_VIRTUAL ?= "1"
 
 python split_kernel_module_packages () {
 import re
@@ -119,10 +120,16 @@ python split_kernel_module_packages () {
 # Avoid automatic -dev recommendations for modules ending with -dev.
 d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1)
 
+# Provide virtual package without postfix
+providevirt = d.getVar('KERNEL_MODULE_PROVIDE_VIRTUAL', True)
+if providevirt == "1":
+   postfix = format.split('%s')[1]
+   d.setVar('RPROVIDES_' + pkg, pkg.replace(postfix, ''))
+
 module_regex = '^(.*)\.k?o$'
 
 module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
-module_pattern = module_pattern_prefix + 'kernel-module-%s'
+module_pattern = module_pattern_prefix + 'kernel-module-%s-' + 
d.getVar("KERNEL_VERSION", True)
 
 postinst = d.getVar('pkg_postinst_modules')
 postrm = d.getVar('pkg_postrm_modules')
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] kernel-module-split: Append KERNEL_VERSION string to kernel module name

[Ola Redell]

On 01/12/2017 02:44 PM, Bruce Ashfield wrote:



Interesting. I think we've probably fallen off the radar of the casual 
reader that may have
more bitbake knowledge to explain why the virtual package 
(kernel-modules) provides
works with the same name and different versions, while a package with 
the version postfix

that rprovides the same name doesn't work with different versions.

I think I have a clue on this so let me try to straight it out as much 
as I can in a somewhat more structured manner than my earlier attempt, 
and hopefully with a readable formatting this time... I'll do a 
background and overview too to let new readers in. I am sorry about this 
being lengthy. I'll post a patch v4 based on the end conclusion.


Anyone with a better knowledge in package management than me (that means 
very many of you) might want to fill in or correct the below. And Bruce, 
I kept #1 and #3 separate since I see quite a difference between "meta 
packages" and "virtual packages".


In my current system I use deb packages and apt-get as package manager 
so the discussion below is apt-get centric. The *assumption* here is 
that package managers in general behave in a similar way to apt-get, 
when it comes to the details below.



Background

My use case is that I want a kernel module packaging that allows me to 
upgrade my kernel and kernel modules using a package manager such as 
apt-get without the kernel modules for the older kernel being removed. 
When I perform an upgrade today (either dist-upgrade or upgrade on the 
package "kernel-modules") the old modules are removed. My motivation is 
that I want to be able to fall back to the older kernel and its modules 
if necessary.


The "kernel-modules" package is called a "meta package" in the code. 
Even though it is an empty package (therefore "meta"), it is a package 
in its own right with a package version based on the kernel version used 
to build it. It has dependencies to all kernel module packages (e.g. 
kernel-module-foo) that are built by bitbake. Hence when the 
"kernel-modules" package is installed or upgraded it pulls in all those 
modules that it depends on.


Side note: If apt-get finds many "kernel-modules" packages available it 
simply picks the one with the latest version (unless given a specific 
version). So having many "meta packages" of different versions available 
at the same time is not a problem - apt-get can distinguish between them.


Today, the kernel module packages like kernel-module-foo are named the 
same regardless of what kernel they belong to. Hence kernel-module-foo 
for kernel-x is the same package as kernel-module-foo for kernel-y, just 
with another version. Therefore, when the "kernel-modules" package is 
upgraded, the new version of kernel-module-foo is installed while the 
older version is removed (you cannot have two versions of the same 
package installed at once). That is the problem.



Alternative solutions

My original suggestion (#0) was to just postfix the kernel module 
package names with a kernel version string (creating 
kernel-module-foo-) just like what is done with the kernel 
itself already today. Note that this  has very little to do 
with the actual version of the package.  is just 
KERNEL_VERSION, nothing else. This solves my use case but causes other 
problems as Bruce pointed out. Existing recipes with dependencies to 
kernel-module-foo will break, and even worse, will have to be updated 
for every new kernel version.


One alternative (#1) would be to add the postfix and also add empty 
"meta packages" for all kernel-module packages, such that the meta 
package "kernel-module-foo" would depend on 
"kernel-module-foo-". That would work great (presumably - 
untested), just like the "kernel-modules" meta package, but would add a 
lot of new packages to the already time consuming build. It would also 
make the kernel-module-split class even more complex.


Another alternative (#2) is to simply make the postfix configurable, on 
or off. This would work fine but it would also add to the complexity of 
the kernel-module-split code as well as adding another (rarely used) 
code path, with the effects that has on stability.


A third alternative (#3) is to add the postfix and let the kernel module 
packages RPROVIDE the shorter named package. Hence 
kernel-module-foo- would then provide kernel-module-foo such 
that recipes can still depend on the kernel-module-foo which is now a 
"virtual package". A "virtual package" is more like an alias for the 
real package and is not a package in its own right (unlike the meta 
package). With this solution one would also be able to install 
kernel-module-foo using "apt-get install kernel-module-foo" but only if 
there is only one provider available. Hence, it would not be possible to 
"apt-get upgrade kernel-module-foo" to get 
kernel-module-foo- since the original package 
(kernel-module-foo-) is always available in an OE system(?) 
(afaik). This would not be a 

Re: [OE-core] [PATCH v3] kernel-module-split: Append KERNEL_VERSION string to kernel module name

[Ola Redell]

2017-01-10 15:45 GMT+01:00 Bruce Ashfield :

>
>
>>
>> So basically, as I understand it, this may be a blocker for this change.
>> Alternatives I can come up with are:
>>
>> 1) For all packaged kernel modules, add meta packages that depend on the
>> correct version of the kernel module itself. Then "kernel-module-foo" would
>> depend on "kernel-module-foo-". The package "kernel-modules" would
>> then still be made depending on "kernel-module-foo-" and the likes.
>>
>
>
> Wouldn't this break your primary use case ? i.e. if you had two kernels in
> the image or
> when you install the 2nd newer kernel, the meta packages would conflict
> and hence couldn't
> be around at the same time ?
>
>
No, it would not break as far as my tests show. I use the meta package
"kernel-modules" today and since it is versioned (not the version string we
are talking about adding here - but regular package version) it all works
fine. apt-get knows about the versions and installs the latest one, with
dependencies, if I don't say anything else. I don't have any problems
having many kernel-modules packages available for installation in my repos
simultaneously. The fine thing with this is that I can perform an "apt-get
dist-upgrade" (or if I'd prefer apt-get upgrade kernel-modules) to install
all new versions of the packages. When I have done that, apt-get finds out
that the older packages are no longer needed and allows me to remove them
(when I want to remove them - i.e. when I know my new kernel works fine)
using "apt-get autoremove". (A note here: After my first kernel upgrade
this is not the case because apt-get considers the original kernel and all
original kernel-modules to not having been installed "automatically", that
is not as dependencies to other packages. But after my second dist-upgrade
the autoremove option allows me to remove unused packages)


> Rather than a meta-package, I think it would be feasible to just have the
> generated split
> modules RPROVIDE both a PN- and PN. i.e. kernel-module-foo and
> kernel-module-foo-
> would be RPROVIDED by the packages. I'd still consider this the same
> solution as #1, just
> with a different implementation, but one that wouldn't generate more
> packages and slow
> the install/generation phase even more.
>


>
> I'm not sure what happens if two packages can have a different on-disk
> name (i.e. kernel-module-foo-1.2
> and kernel-module-foo-1.3) and both RPROVIDE "kernel-module-foo". Only one
> would be installed,
> but if someone did an apt-get or a smart install of kernel-module-foo ..
> chaos may erupt.
>
>
That is an interesting alternative and I looked in to that a bit by doing
the necessary hack and testing it out. Your earlier concerns are solved
with this - you can do RDEPENDS_x = "kernel-module-foo" and I can install
that virtual package using "apt-get install kernel-module-foo" as long as I
only have one package providing that virtual package available.  But as
soon as I have more than one package providing that virtual package,
apt-get refuses to install it without me stating exactly what package I
want. Hence I then need to point out kernel-module-foo-. The
reason for this seems to be that virtual packages are not versioned (again
package version - not the postfix string we discuss here). This is not a
huge problem for me with my current use case, but if I'd only install a
couple of kernel modules with bitbake and then later would want to upgrade
them using e.g. "apt-get dist-upgrade" or "apt-get upgrade", that would be
a problem.

I agree adding a huge number of more meta packages should be avoided if
possible.


>
>
>>
>> 2) Make the version postfix configurable with a
>> KERNEL_MODULE_PACKAGE_POSTFIX, exactly like the prefix (which was added
>> recently). The default would be an empty postfix, but it could be used to
>> achieve this functionality for those who need it.
>>
>> No. 2 is a very simple change. No. 1 is more uncertain (to me at least).
>>
>
> I was discussing this change with a few people in person this morning and
> we came up with making this
> optional as well (your #2), which would ensure that existing use cases are
> not broken, but on the
> flip side it makes already complex code more complex and would leave some
> paths more tested
> than others.
>
> If #1 is feasible, I wonder if a variant of both 1 an 2 are viable ? i.e.
> make the generation of the
> packages always have a version in the name, but make the generation of a
> "versionless" meta
> package be optional (or a versionless RPROVIDE). In the default case
> they'd be enabled, and no
> one would notice that anything had changed but the module-split code would
> have a single path.
> In a case where multiple kernels are to be installed, you'd have to
> disable the versionless package
> generation.
>

Given my tests described above, this may be the best solution, yes.


>
> I mentioned wildcards above, I'm not sure if anyone else reading this
> thread knows if 

Re: [OE-core] [PATCH v3] kernel-module-split: Append KERNEL_VERSION string to kernel module name

[Ola Redell]

On 01/09/2017 06:20 PM, Bruce Ashfield wrote:


The functionality you describe is a good thing, but I have a few 
questions to make
sure I understand the change, and confirm that existing functionality 
isn't broken ..
and that the internals of the package management do the right thing 
(I'm not an

expert on those internals .. so bear with me).
I am not an expert either so I thought I'd suggest this since it works 
for me and see how it is received.


With this change, do you know if RDEPENDS="kernel-module-foo" still 
works ? or

does it now have to be RDEPENDS="kernel-module-foo-" ?

No, the old writing would not work. You would need to add the version as 
in your second example. That will break things, true.
On a similar thought, "smart install kernel-module-foo" would also 
work ? or does it

now require a version ?

Also in this example you would need to add the version, even though I 
have only used apt-get and not smart.


The meta package "kernel-modules" will work as before, without a version 
postfix.


So basically, as I understand it, this may be a blocker for this change. 
Alternatives I can come up with are:


1) For all packaged kernel modules, add meta packages that depend on the 
correct version of the kernel module itself. Then "kernel-module-foo" 
would depend on "kernel-module-foo-". The package 
"kernel-modules" would then still be made depending on 
"kernel-module-foo-" and the likes.


2) Make the version postfix configurable with a 
KERNEL_MODULE_PACKAGE_POSTFIX, exactly like the prefix (which was added 
recently). The default would be an empty postfix, but it could be used 
to achieve this functionality for those who need it.


No. 2 is a very simple change. No. 1 is more uncertain (to me at least).

/Ola

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [jethro][PATCH] kernel.bbclass: use correct path in postrm's call to update-alternatives

[ola . redell]

From: Ola Redell <ola.red...@retotech.se>

The postrm script for the kernel-image package needs the same path
when calling 'update-alternatives --remove', as the one given to
'update-alternatives --install' in the postinst script. Without this
change, the postrm script fails to remove the alternative.

This applies to [jethro], [master] already has a fix for this.

Signed-off-by: Ola Redell <ola.red...@retotech.se>
---
 meta/classes/kernel.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 5e8b6cf..0db621e 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -390,7 +390,7 @@ pkg_postinst_kernel-image () {
 }
 
 pkg_postrm_kernel-image () {
-   update-alternatives --remove ${KERNEL_IMAGETYPE} 
${KERNEL_IMAGETYPE}-${KERNEL_VERSION} || true
+   update-alternatives --remove ${KERNEL_IMAGETYPE} 
/${KERNEL_IMAGEDEST}/${KERNEL_IMAGETYPE}-${KERNEL_VERSION} || true
 }
 
 PACKAGESPLITFUNCS_prepend = "split_kernel_packages "
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3] kernel-module-split: Append KERNEL_VERSION string to kernel module name

[ola . redell]

From: Ola Redell <ola.red...@retotech.se>

The KERNEL_VERSION string is added to kernel module package names in order to
make the kernel modules for different kernel versions distinct packages instead
of different versions of the same package. With this change, when a new kernel
is installed together with its kernel modules (e.g. by upgrade of the packages
kernel and kernel-modules) using some package manager such as apt-get or rpm,
the kernel modules for the older kernel will not be removed. This enables a
fall back to the older kernel if the new one fails.

Signed-off-by: Ola Redell <ola.red...@retotech.se>
---
 meta/classes/kernel-module-split.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index efe1b42..13cff51 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -122,7 +122,7 @@ python split_kernel_module_packages () {
 module_regex = '^(.*)\.k?o$'
 
 module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
-module_pattern = module_pattern_prefix + 'kernel-module-%s'
+module_pattern = module_pattern_prefix + 'kernel-module-%s-' + 
d.getVar("KERNEL_VERSION", True)
 
 postinst = d.getVar('pkg_postinst_modules')
 postrm = d.getVar('pkg_postrm_modules')
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] questions on patching

[Ola Redell]

Hi all,

I am a bit new to sending in patches to a list like this and 
specifically to OE-core, so I need a bit of guidance.


I have a fix for a bug I found in Jethro in the kernel.bbclass and I 
have tested my fix there (update-alternatives needs a full path also 
when removing an alternative). Now, the bug is around in many different 
branches (laverne, krogoth and master for sure) but that particular 
patch will only apply to Jethro.


I can make a patch for master but would have problems testing it 
thoroughly there without spending quite a bit of time (which I don't 
have at the moment).


Easy questions:
How should I proceed?
How do I know what branches are actively maintained and for which 
patches are welcome?


I should have asked these questions before sending my earlier patch for 
the kernel-module-split.bbclass since this holds for that too.


Thanks,
Ola



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] Append KERNEL_VERSION string to kernel module package names

[ola . redell]

From: Ola Redell <ola.red...@retotech.se>

The KERNEL_VERSION string is added to kernel module package names in order to
make the kernel modules for different kernel versions distinct packages instead
of different versions of the same package. With this change, when a new kernel
is installed together with its kernel modules (e.g. by upgrade of the packages
kernel and kernel-modules) using some package manager such as apt-get or rpm,
the kernel modules for the older kernel will not be removed. This enables a
fall back to the older kernel if the new one fails.

Signed-off-by: Ola Redell <ola.red...@retotech.se>
---
 meta/classes/kernel-module-split.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index efe1b42..13cff51 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -122,7 +122,7 @@ python split_kernel_module_packages () {
 module_regex = '^(.*)\.k?o$'
 
 module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
-module_pattern = module_pattern_prefix + 'kernel-module-%s'
+module_pattern = module_pattern_prefix + 'kernel-module-%s-' + 
d.getVar("KERNEL_VERSION", True)
 
 postinst = d.getVar('pkg_postinst_modules')
 postrm = d.getVar('pkg_postrm_modules')
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] Append KERNEL_VERSION string to kernel module package names

[Ola Redell]

Thanks, that was a mistake. I did this change in Jethro originally where 
the prefix does not appear. I will prepare a v2.


Ola


On 2017-01-03 14:56, Mats Karrman wrote:

-module_pattern = module_pattern_prefix + 'kernel-module-%s'
+module_pattern = 'kernel-module-%s-' + 
d.getVar("KERNEL_VERSION", True)

Why did you drop 'module_pattern_prefix'?

postinst = d.getVar('pkg_postinst_modules')
  postrm = d.getVar('pkg_postrm_modules')




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] Append KERNEL_VERSION string to kernel module package names

[ola . redell]

From: Ola Redell <ola.red...@retotech.se>

The KERNEL_VERSION string is added to kernel module package names in order to
make the kernel modules for different kernel versions distinct packages instead
of different versions of the same package. With this change, when a new kernel
is installed together with its kernel modules (e.g. by upgrade of the packages
kernel and kernel-modules) using some package manager such as apt-get or rpm,
the kernel modules for the older kernel will not be removed. This enables a
fall back to the older kernel if the new one fails.

Signed-off-by: Ola Redell <ola.red...@retotech.se>
---
 meta/classes/kernel-module-split.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index efe1b42..8365448 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -122,7 +122,7 @@ python split_kernel_module_packages () {
 module_regex = '^(.*)\.k?o$'
 
 module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
-module_pattern = module_pattern_prefix + 'kernel-module-%s'
+module_pattern = 'kernel-module-%s-' + d.getVar("KERNEL_VERSION", True)
 
 postinst = d.getVar('pkg_postinst_modules')
 postrm = d.getVar('pkg_postrm_modules')
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] Kernel module package naming

[ola . redell]

From: Ola Redell <ola.red...@retotech.se>

Hi,

I have been working on a problem with safe kernel and kernel-module upgrade 
on a system using .deb packages and apt-get as package manager.

My problem has been that I need to be able to fall back to an earlier kernel
if a newly installed kernel fails to boot for some reason. The basic support
for this is handled in U-boot, but my problem has been with the upgrade of 
kernel
modules. Today, the same kernel module for different kernel versions has the
same name for both kernels and differ only in its package version. This
makes package managers remove the old kernel module when the newer one is
installed. As a result a safe fall back to the older kernel, whith its (old) 
modules is very difficult.

I have a solution that works for me by changing the kernel-module-split.bbclass 
to add the KERNEL_VERSION string to the kernel module package name. I assume 
this could be useful also in the general case. I am not a package management 
expert though, so let me know what you think.

/Ola

Ola Redell (1):
  Append KERNEL_VERSION string to kernel module package names

 meta/classes/kernel-module-split.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core