On Wed, Mar 04, 2026 at 02:33:03PM -0500, Joe Lawrence wrote:
> On Mon, Mar 02, 2026 at 09:38:17AM +0100, Miroslav Benes wrote:
> > Him
> >
> > > > We store test modules in tools/testing/selftests/livepatch/test_modules/
> > > > now. Could you move klp_test_module.c there, please? You might also
> > > > reuse
> > > > existing ones for the purpose perhaps.
> > >
> > > IIUC, tools/testing/selftests/livepatch/test_modules/ is more like an out
> > > of tree module. In the case of testing klp-build, we prefer to have it to
> > > work the same as in-tree modules. This is important because klp-build
> > > is a toolchain, and any changes of in-tree Makefiles may cause issues
> > > with klp-build. Current version can catch these issues easily. If we build
> > > the test module as an OOT module, we may miss some of these issues.
> > > In the longer term, we should consider adding klp-build support to build
> > > livepatch for OOT modules. But for now, good test coverage for in-tree
> > > modules are more important.
> >
> > Ok. I thought it would not matter but it is a fair point.
> >
> > > > What about vmlinux? I understand that it provides a lot more flexibility
> > > > to have separate functions for testing but would it be somehow
> > > > sufficient
> > > > to use the existing (real) kernel functions? Like cmdline_proc_show()
> > > > and
> > > > such which we use everywhere else? Or would it be to limited? I am fine
> > > > if
> > > > you find it necessary in the end. I just think that reusing as much as
> > > > possible is generally a good approach.
> > >
> > > I think using existing functions would be too limited, and Joe seems to
> > > agree with this based on his experience. To be able to test corner cases
> > > of the compiler/linker, such as LTO, we need special code patterns.
> > > OTOH, if we want to use an existing kernel function for testing, it needs
> > > to be relatively stable, i.e., not being changed very often. It is not
> > > always
> > > easy to find some known to be stable code that follows specific patterns.
> > > If we add dedicated code as test targets, things will be much easier
> > > down the road.
> >
> > Fair enough.
> >
>
> I've been tinkering with ideas in this space, though I took it in a very
> different direction.
>
> (First a disclaimer, this effort is largely the result of vibe coding
> with Claude to prototype testing concepts, so I don't believe any of it
> is reliable or upstream-worthy at this point.)
>
> From a top-down perspective, I might start with the generated test
> reports:
Thanks Song and Joe, these are some great ideas. Testing will be
extremely important for the success of klp-build.
Below are some back of napkin thoughts based on my experiences with
kpatch testing. In my opinion the unit/integration test split for
kpatch was successful and we should try to continue that tradition with
klp-build.
- unit testing (.o diff)
- great for preventing past regressions
- nice and fast
- tests the most fragile bits
- catches 99% of bugs (almost everything except for new kernel/compiler
features)
- try to enforce a rule: one test for every regression
- if we're storing binaries we may need to host the test suite elsewhere
- extracting .o's could be more of a challenge now that vmlinux.o diff is the
norm
- the making of regression tests needs to be as easy as possible
- or maybe they can be built on demand with .c and/or .S?
- that *might* be more difficult
- we'd need meta-tests to ensure the tests continue to test what they're
supposed to
- integration testing:
- good for testing "this complex end-to-end operation still works"
- good for finding issues with new toolchain versions and new kernel features
- can be used for testing certain runtime-critical features like static
branches/calls
- not necessarily good for testing regressions, the underlying code is free
to change at any time
- I like Song's idea of adding a fake (yet stable) test component in vmlinux
for which we can create .patch files to test
- I also like Joe's idea of scripted patch generation (e.g., automatically
patching meminfo_proc_show() somehow)
And in general, we may want to look to porting as many of the kpatch
unit/integration tests as possible so we can take advantage of all those
accumulated testing experiences.
--
Josh
