On 04/26/2019 09:01 AM, Mario Smarduch wrote:
>
>
> On 04/26/2019 06:32 AM, Joe Lawrence wrote:
>> On 4/26/19 3:51 AM, Mario Smarduch wrote:
>>> Hello,
>>> I noticed that with livepatch the old and new functions can
>>> run in parallel, but with kpatch the kpatch module load fails on
>>> "activeness safety check".
>>
>> Hi Mario,
>>
>> I think you are referring to kpatch's use of stop_machine() to quiesce
>> the system before patching vs. livepatch's consistency model[1] and its
>> convergence on a patched state?
>>
>>> Am I missing something with livepatch? Is there another way to
>>> build and run it?
>>
>> AFAIK there is no way around the consistency model with livepatch. What
>> you might be able to do is hack the kpatch-build to utilize the kpatch
>> support module instead of the livepatch API. (Look for the
>> $KPATCH_MODULE code in the kpatch-build script.) I don't think this use
>> case is very well tested and in fact, the kpatch.ko will be deprecated
>> eventually.
>>
>> Do you have a particular case in mind which the livepatch consistency
>> model does not support?
>
> I'm also wondering if this still holds today
>
> https://groups.google.com/forum/#!topic/fa.linux.kernel/FhGaKm0Vhm0
Sorry about that link is bad.
Hi,
Since Jiri posted the kGraft patches [1], I wanted to share an
alternative live patching solution called kpatch, which is something
we've been working on at Red Hat for quite a while.
The kernel piece of it ("kpatch core module") is completely
self-contained in a GPL module. It compiles and works without needing
to change any kernel code, and in fact we already have it working fine
with Fedora 20 [2] without any distro kernel patches needed. We'd
definitely like to see it (or some combination of it and kGraft) merged
into Linux.
This patch set is for the core module, which provides the kernel
infrastructure for kpatch. It has a kpatch_register() interface which
allows kernel modules ("patch modules") to replace old functions with
new functions which are loaded with the modules.
There are also some user space tools [2] which aren't included in this
patch set, which magically generate binary patch modules from source
diffs, and manage the loading and unloading of these modules. I didn't
include them here because I think we should agree on what the kernel
parts should look like before trying to discuss the user space tools
(and whether they should be in-tree).
kpatch vs kGraft
----------------
I think the biggest difference between kpatch and kGraft is how they
ensure that the patch is applied atomically and safely.
kpatch checks the backtraces of all tasks in stop_machine() to ensure
that no instances of the old function are running when the new function
is applied. I think the biggest downside of this approach is that
stop_machine() has to idle all other CPUs during the patching process,
so it inserts a small amount of latency (a few ms on an idle system).
Instead, kGraft uses per-task consistency: each task either sees the old
version or the new version of the function. This gives a consistent
view with respect to functions, but _not_ data, because the old and new
functions are allowed to run simultaneously and share data. This could
be dangerous if a patch changes how a function uses a data structure.
The new function could make a data change that the old function wasn't
expecting.
With kpatch, that's not an issue because all the functions are patched
at the same time. So kpatch is safer with respect to data interactions.
Other advantages of the kpatch stop_machine() approach:
- IMO, the kpatch code is much simpler than kGraft. The implementation
is very straightforward and is completely self-contained. It requires
zero changes to the kernel.
(However a new TAINT_KPATCH flag would be a good idea, and we do
anticipate some minor changes to kprobes and ftrace for better
compatibility.)
- The use of stop_machine() will enable an important not-yet-implemented
feature to call a user-supplied callback function at loading time
which can be used to atomically update data structures when applying a
patch. I don't see how such a feature would be possible with the
kGraft approach.
- kpatch applies patches immediately without having to send signals to
sleeping processes, and without having to hope that those processes
handle the signal appropriately.
- kpatch's patching behavior is more deterministic because
stop_machine() ensures that all tasks are sleeping and interrupts are
disabled when the patching occurs.
- kpatch already supports other cool features like:
- removing patches and rolling back to the original functions
- atomically replacing existing patches
- incremental patching
- loading multiple patch modules
>>
>> [1]
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/livepatch/livepatch.txt?h=v5.1-rc6#n63
>>
>>
>> Regards,
>>
>> -- Joe
>>
>> _______________________________________________
>> kpatch mailing list
>> [email protected]
>> https://www.redhat.com/mailman/listinfo/kpatch
_______________________________________________
kpatch mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/kpatch
