Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-14 Thread Gionatan Danti

Il 2022-01-14 15:30 Johnny Hughes ha scritto:

No .. none of the CentOS Kernels were EVER binary compatible with any
RHEL kernel.

CentOS Linux has always been (now also including CentOS Stream 8 and
9) a completely separate 'closed' build system.

We use the SAME source code to build things, modified to remove
branding.  But CentOS has NEVER been (nor is any other rebuild
distribution now) Binary Compatible.

Want to see how .. just extract two rpms with the same name from two
different distributions into separate directories and run a sha256sum
on all the files in the different directories with find command.  Some
files may be identical (most text files that are copied), others will
not be.

It is virtually impossible for all produced packages to be 'binary
compatible' UNLESS they are built with exact the same files (not files
BUILT fromt he same sources .. the exact same files) in the build root
AND with exactly the same software doing the building.  Any group that
claims 'binary compatibility' is either lying or they do not
understand compiling and linking.

CentOS never had that.  Neither does any rebuild.

This is why the CentOS Project 'CHANGED' our term from binary
compatible to 'Functionally Compatible' a long time ago.  (Using same
source code, we produce DIFFERENT software .. that works the same way
but has different SHASUM values.  Don't be fooled by key words like
'binary compatible' .. check it out for yourself.

If you build kpatches to kernels, to make them work you need to build
the kpatch for the specific kernel (CentOS would need to build against
CentOS kernels, etc).  Also, there are the certificate signing issues
and keys that you would need to take into account.  You need to have
the CA Trust to be able to create signatures that the system will
allow.
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Thank you so much for the detailed explanation, very appreciated.
Regards.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-14 Thread Johnny Hughes

On 1/14/22 08:30, Johnny Hughes wrote:

On 1/14/22 07:57, Gionatan Danti wrote:

Il 2022-01-14 13:17 Josh Boyer ha scritto:

RHEL's kernel live patching uses upstream open source kpatch.  The
sources to the kpatches are delivered in customer facing CDN repos at
the same time as the kpatch itself.  We do not use proprietary code to
produce or apply the kpatches.

I can only speculate on whether RHEL kpatches would work on a CentOS
kernel, but my assumption is that they would not due to how they are
signed.


Is (well, was) the CentOS kernel identical at binary level to the RHEL 
one?
If so, the same kpatch should be applicable to both RHEL and CentOS 
(the old one).


But I seem to understand that the two kernels are *not* bytewise 
identical, so a binary kpatch can not be applied the CentOS. Is this 
true?


Anyway, RH kpatches are surely not compatible with CentOS stream. So I 
asked if some project was started to provide live kernel patching to 
the new CentOS project. If I don't miss something, this is not the case.


Regards.



No .. none of the CentOS Kernels were EVER binary compatible with any 
RHEL kernel.


CentOS Linux has always been (now also including CentOS Stream 8 and 9) 
a completely separate 'closed' build system.


We use the SAME source code to build things, modified to remove 
branding.  But CentOS has NEVER been (nor is any other rebuild 
distribution now) Binary Compatible.


Want to see how .. just extract two rpms with the same name from two 
different distributions into separate directories and run a sha256sum on 
all the files in the different directories with find command.  Some 
files may be identical (most text files that are copied), others will 
not be.


It is virtually impossible for all produced packages to be 'binary 
compatible' UNLESS they are built with exact the same files (not files 
BUILT fromt he same sources .. the exact same files) in the build root 
AND with exactly the same software doing the building.  Any group that 
claims 'binary compatibility' is either lying or they do not understand 
compiling and linking.


CentOS never had that.  Neither does any rebuild.

This is why the CentOS Project 'CHANGED' our term from binary compatible 
to 'Functionally Compatible' a long time ago.  (Using same source code, 
we produce DIFFERENT software .. that works the same way but has 
different SHASUM values.  Don't be fooled by key words like 'binary 
compatible' .. check it out for yourself.


If you build kpatches to kernels, to make them work you need to build 
the kpatch for the specific kernel (CentOS would need to build against 
CentOS kernels, etc).  Also, there are the certificate signing issues 
and keys that you would need to take into account.  You need to have the 
CA Trust to be able to create signatures that the system will allow.

___



As a bit of a helper .. we used to use this script to find differences:

https://vault.centos.org/4.9/build/distro/tmverifyrpms

Way back in the CentOS Linux 4 days.

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-14 Thread Johnny Hughes

On 1/14/22 07:57, Gionatan Danti wrote:

Il 2022-01-14 13:17 Josh Boyer ha scritto:

RHEL's kernel live patching uses upstream open source kpatch.  The
sources to the kpatches are delivered in customer facing CDN repos at
the same time as the kpatch itself.  We do not use proprietary code to
produce or apply the kpatches.

I can only speculate on whether RHEL kpatches would work on a CentOS
kernel, but my assumption is that they would not due to how they are
signed.


Is (well, was) the CentOS kernel identical at binary level to the RHEL one?
If so, the same kpatch should be applicable to both RHEL and CentOS (the 
old one).


But I seem to understand that the two kernels are *not* bytewise 
identical, so a binary kpatch can not be applied the CentOS. Is this true?


Anyway, RH kpatches are surely not compatible with CentOS stream. So I 
asked if some project was started to provide live kernel patching to the 
new CentOS project. If I don't miss something, this is not the case.


Regards.



No .. none of the CentOS Kernels were EVER binary compatible with any 
RHEL kernel.


CentOS Linux has always been (now also including CentOS Stream 8 and 9) 
a completely separate 'closed' build system.


We use the SAME source code to build things, modified to remove 
branding.  But CentOS has NEVER been (nor is any other rebuild 
distribution now) Binary Compatible.


Want to see how .. just extract two rpms with the same name from two 
different distributions into separate directories and run a sha256sum on 
all the files in the different directories with find command.  Some 
files may be identical (most text files that are copied), others will 
not be.


It is virtually impossible for all produced packages to be 'binary 
compatible' UNLESS they are built with exact the same files (not files 
BUILT fromt he same sources .. the exact same files) in the build root 
AND with exactly the same software doing the building.  Any group that 
claims 'binary compatibility' is either lying or they do not understand 
compiling and linking.


CentOS never had that.  Neither does any rebuild.

This is why the CentOS Project 'CHANGED' our term from binary compatible 
to 'Functionally Compatible' a long time ago.  (Using same source code, 
we produce DIFFERENT software .. that works the same way but has 
different SHASUM values.  Don't be fooled by key words like 'binary 
compatible' .. check it out for yourself.


If you build kpatches to kernels, to make them work you need to build 
the kpatch for the specific kernel (CentOS would need to build against 
CentOS kernels, etc).  Also, there are the certificate signing issues 
and keys that you would need to take into account.  You need to have the 
CA Trust to be able to create signatures that the system will allow.

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-14 Thread Gionatan Danti

Il 2022-01-14 13:17 Josh Boyer ha scritto:

RHEL's kernel live patching uses upstream open source kpatch.  The
sources to the kpatches are delivered in customer facing CDN repos at
the same time as the kpatch itself.  We do not use proprietary code to
produce or apply the kpatches.

I can only speculate on whether RHEL kpatches would work on a CentOS
kernel, but my assumption is that they would not due to how they are
signed.


Is (well, was) the CentOS kernel identical at binary level to the RHEL 
one?
If so, the same kpatch should be applicable to both RHEL and CentOS (the 
old one).


But I seem to understand that the two kernels are *not* bytewise 
identical, so a binary kpatch can not be applied the CentOS. Is this 
true?


Anyway, RH kpatches are surely not compatible with CentOS stream. So I 
asked if some project was started to provide live kernel patching to the 
new CentOS project. If I don't miss something, this is not the case.


Regards.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-14 Thread Josh Boyer
On Thu, Jan 13, 2022 at 2:13 PM Valeri Galtsev
 wrote:
>
>
>
> On 1/13/22 1:01 PM, Gordon Messmer wrote:
> > On 1/13/22 09:32, Valeri Galtsev wrote:
> >> In layman's language summary: RedHat Enterprise features (including
> >> "live" kernel patching) are to be expected _only_ in RedHat Enterprise
> >> "binary replica" distributions, which CentOS Stream is not.
> >
> >
> > I don't think that's true, exactly.  As far as I know, rebuild
> > distributions never had the "Enterprise" features*.  Critically, I think
> > that a lot of people mistakenly believed that CentOS *did* have
> > Enterprise features, because it was rebuilt from RHEL code, and that
> > misunderstanding underlies a great deal of the negative response toward
> > CentOS Stream.
> >
>
> Thanks for correcting my layman's representation. It should have better
> said that "binary replica" is "binary compatible" in a sense whatever
> software distributed as binary for RHEL will work the same on "binary
> replica". I guess my views and wordings got skewed by latest changes of
> CentOS paradigms.
>
> >
> > *: "Enterprise" features include but are not limited to:
> >
> > 1. Minor releases with independent life cycles / Extended Update Support
> > 2. Classification for updates (security, bugfix, enhancement)
> > 3. Live patching for kernel security vulnerabilities
>
> We never had it in CentOS in the past, but I'm just curious: is live
> patching proprietary piece of RHEL? I know there are several solutions,
> way back there was paid one called splice, my Boss's son was one of the
> developers of that. Just curious, as, if it is paid, it is stripped off
> as part of CentOS composition, but if it is not paid, open source, then
> it would "just work", or not?

RHEL's kernel live patching uses upstream open source kpatch.  The
sources to the kpatches are delivered in customer facing CDN repos at
the same time as the kpatch itself.  We do not use proprietary code to
produce or apply the kpatches.

I can only speculate on whether RHEL kpatches would work on a CentOS
kernel, but my assumption is that they would not due to how they are
signed.

josh

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-13 Thread Kenneth Porter
--On Thursday, January 13, 2022 2:10 PM -0500 Valeri Galtsev 
 wrote:



We never had it in CentOS in the past, but I'm just curious: is live
patching proprietary piece of RHEL? I know there are several solutions,
way back there was paid one called splice, my Boss's son was one of the
developers of that. Just curious, as, if it is paid, it is stripped off
as part of CentOS composition, but if it is not paid, open source, then
it would "just work", or not?


Indeed, we're talking the software versus the organization. I never 
expected CentOS the organization to provide anything more than repackaging 
(rebuilding and mirroring).


For kernel patching, there's the matter of rebuilding and distributing the 
patches, and then whether the software can do anything with that. If it's 
proprietary, the issue is moot.


But maybe it's like the update classification and differentiation, which 
was never implemented for CentOS, because of the extra effort the 
organization would have to provide.




___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-13 Thread Valeri Galtsev



On 1/13/22 1:01 PM, Gordon Messmer wrote:

On 1/13/22 09:32, Valeri Galtsev wrote:
In layman's language summary: RedHat Enterprise features (including 
"live" kernel patching) are to be expected _only_ in RedHat Enterprise 
"binary replica" distributions, which CentOS Stream is not. 



I don't think that's true, exactly.  As far as I know, rebuild 
distributions never had the "Enterprise" features*.  Critically, I think 
that a lot of people mistakenly believed that CentOS *did* have 
Enterprise features, because it was rebuilt from RHEL code, and that 
misunderstanding underlies a great deal of the negative response toward 
CentOS Stream.




Thanks for correcting my layman's representation. It should have better 
said that "binary replica" is "binary compatible" in a sense whatever 
software distributed as binary for RHEL will work the same on "binary 
replica". I guess my views and wordings got skewed by latest changes of 
CentOS paradigms.




*: "Enterprise" features include but are not limited to:

1. Minor releases with independent life cycles / Extended Update Support
2. Classification for updates (security, bugfix, enhancement)
3. Live patching for kernel security vulnerabilities


We never had it in CentOS in the past, but I'm just curious: is live 
patching proprietary piece of RHEL? I know there are several solutions, 
way back there was paid one called splice, my Boss's son was one of the 
developers of that. Just curious, as, if it is paid, it is stripped off 
as part of CentOS composition, but if it is not paid, open source, then 
it would "just work", or not?



4. Support


Oops, as features I meant functionality of CentOS, nothing beyond that.

Valeri



___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-13 Thread Gordon Messmer

On 1/13/22 09:32, Valeri Galtsev wrote:
In layman's language summary: RedHat Enterprise features (including 
"live" kernel patching) are to be expected _only_ in RedHat Enterprise 
"binary replica" distributions, which CentOS Stream is not. 



I don't think that's true, exactly.  As far as I know, rebuild 
distributions never had the "Enterprise" features*.  Critically, I think 
that a lot of people mistakenly believed that CentOS *did* have 
Enterprise features, because it was rebuilt from RHEL code, and that 
misunderstanding underlies a great deal of the negative response toward 
CentOS Stream.



*: "Enterprise" features include but are not limited to:

1. Minor releases with independent life cycles / Extended Update Support
2. Classification for updates (security, bugfix, enhancement)
3. Live patching for kernel security vulnerabilities
4. Support

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-13 Thread Valeri Galtsev



On 1/13/22 12:28 PM, Johnny Hughes wrote:

On 1/7/22 12:18, Gordon Messmer wrote:

On 1/7/22 09:39, Gionatan Danti wrote:
is kernel live patching working for CentOS Stream 9? 



https://access.redhat.com/solutions/2206511

My understanding of live kernel patching is that the feature allows 
systems to update specific individual kernel functions, and is 
primarily useful for addressing security vulnerabilities (and not, for 
example, for updating from one kernel version to another).  I don't 
know for a fact, but my expectation is that CentOS Stream systems 
aren't going to get "live" patches because there's no ongoing support 
for individual kernels.




Indeed .. you will get those things rolled into the next kernel, but not 
as live patches.


In layman's language summary: RedHat Enterprise features (including 
"live" kernel patching) are to be expected _only_ in RedHat Enterprise 
"binary replica" distributions, which CentOS Stream is not.


Valeri
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-13 Thread Johnny Hughes

On 1/7/22 12:18, Gordon Messmer wrote:

On 1/7/22 09:39, Gionatan Danti wrote:
is kernel live patching working for CentOS Stream 9? 



https://access.redhat.com/solutions/2206511

My understanding of live kernel patching is that the feature allows 
systems to update specific individual kernel functions, and is primarily 
useful for addressing security vulnerabilities (and not, for example, 
for updating from one kernel version to another).  I don't know for a 
fact, but my expectation is that CentOS Stream systems aren't going to 
get "live" patches because there's no ongoing support for individual 
kernels.




Indeed .. you will get those things rolled into the next kernel, but not 
as live patches.

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-07 Thread Gionatan Danti

Il 2022-01-07 19:07 Brian Stinson ha scritto:

There are 2 things to note here about kernel live patching:
- We do not provide patch files in CentOS Stream (or previously in
CentOS Linux, for that matter). We've always recommended RHEL as a
better fit for folks that have hard requirements on this sort of
workflow.
- RHEL 9 is not yet released, my understanding is that patch files for
RHEL proper will begin showing up after GA


Yup, this matches my expectations.
Thank you all for sharing.
Regards.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-07 Thread Gordon Messmer

On 1/7/22 10:36, Kenneth Porter wrote:
If Stream is to be the next RHEL, wouldn't you want to test this kind 
of thing so the RHEL subscribers don't have to? 



Red Hat does not rely on end-users to test their software.  (And that's 
definitely not what Stream is for.)


___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-07 Thread Josh Boyer
On Fri, Jan 7, 2022 at 1:36 PM Kenneth Porter  wrote:
>
> On 1/7/2022 10:07 AM, Brian Stinson wrote:
> > - We do not provide patch files in CentOS Stream (or previously in
> > CentOS Linux, for that matter). We've always recommended RHEL as a
> > better fit for folks that have hard requirements on this sort of
> > workflow.
>
> If Stream is to be the next RHEL, wouldn't you want to test this kind of
> thing so the RHEL subscribers don't have to?

RHEL tests live patches for released, supported kernels before they
are delivered to customers.

josh

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-07 Thread Kenneth Porter

On 1/7/2022 10:07 AM, Brian Stinson wrote:

- We do not provide patch files in CentOS Stream (or previously in
CentOS Linux, for that matter). We've always recommended RHEL as a
better fit for folks that have hard requirements on this sort of
workflow.


If Stream is to be the next RHEL, wouldn't you want to test this kind of 
thing so the RHEL subscribers don't have to?


___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-07 Thread Gordon Messmer

On 1/7/22 09:39, Gionatan Danti wrote:
is kernel live patching working for CentOS Stream 9? 



https://access.redhat.com/solutions/2206511

My understanding of live kernel patching is that the feature allows 
systems to update specific individual kernel functions, and is primarily 
useful for addressing security vulnerabilities (and not, for example, 
for updating from one kernel version to another).  I don't know for a 
fact, but my expectation is that CentOS Stream systems aren't going to 
get "live" patches because there's no ongoing support for individual 
kernels.


___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] Kernel live patching on CentOS Stream 9

2022-01-07 Thread Brian Stinson
On Fri, Jan 7, 2022 at 11:39 AM Gionatan Danti  wrote:
>
> Dear all,
> is kernel live patching working for CentOS Stream 9?
>
> I know that I can enable it via cockpit or the command line, but are
> kernel patch files really available or do they require an active RHEL
> subscription?
>
> Thanks.
>
> --
> Danti Gionatan
> Supporto Tecnico
> Assyoma S.r.l. - www.assyoma.it
> email: g.da...@assyoma.it - i...@assyoma.it
> GPG public key ID: FF5F32A8
> ___
> CentOS mailing list
> CentOS@centos.org
> https://lists.centos.org/mailman/listinfo/centos
>

There are 2 things to note here about kernel live patching:
- We do not provide patch files in CentOS Stream (or previously in
CentOS Linux, for that matter). We've always recommended RHEL as a
better fit for folks that have hard requirements on this sort of
workflow.
- RHEL 9 is not yet released, my understanding is that patch files for
RHEL proper will begin showing up after GA

--Brian

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos