Implementation of out-of-line static calls for PowerPC 64-bit ELF V2 ABI. Static calls patch an indirect branch into a direct branch at runtime. Out-of-line specifically has a caller directly call a trampoline, and the trampoline gets patched to directly call the target.
Previous version here: https://lore.kernel.org/all/20220916062330.430468-1-bg...@linux.ibm.com/ I couldn't see a dedicated ftrace benchmark in the kernel, but my own benchmarking showed no significant impact to ftrace activation. The __patch_memory function is meant to be accessed through the size checking patch_memory wrapper. I don't think there's a way to expose the macro without also exposing __patch_memory though. I considered making the type an explicit macro param, but using the value type seemed more ergonomic. V2: Mostly accounting for feedback from Christophe: * Code patching rewritten - Rename to *_memory - Use __always_inline to get the compiler to realise it can collapse all the sub-functions - Pass data directly instead of through a pointer, elliding a redundant load - Flush the last byte of data too (technically redundant if an instrucion, but saves a conditional branch + the isync will be the bottleneck). - Handle a non-cohenrent icache, assume a coherent dcache - Handle when we don't assume a 64 byte icache on 64-bits - Flatten the poke address init and teardown - Check the data size in patch_memory at build time (inline function was suggested, but a macro makes checking based on the data type easier). - It builds now on 32 bit and without strict RWX * Static call enabling is no longer configurable * Refactored arch_static_call_transform to minimise casting * Made the KUnit tests more robust (previously they changed non-volatile registers in the init hook, but that's incorrect because it returns to the KUnit framework before the test case is called). * Some other minor refactoring in other patches Benjamin Gray (6): powerpc/code-patching: Implement generic text patching function powerpc/module: Handle caller-saved TOC in module linker powerpc/module: Optimise nearby branches in ELF V2 ABI stub static_call: Move static call selftest to static_call_selftest.c powerpc/64: Add support for out-of-line static calls powerpc/64: Add tests for out-of-line static calls arch/powerpc/Kconfig | 12 +- arch/powerpc/include/asm/code-patching.h | 8 + arch/powerpc/include/asm/static_call.h | 80 +++++++- arch/powerpc/kernel/Makefile | 4 +- arch/powerpc/kernel/module_64.c | 27 ++- arch/powerpc/kernel/static_call.c | 151 +++++++++++++- arch/powerpc/kernel/static_call_test.c | 251 +++++++++++++++++++++++ arch/powerpc/kernel/static_call_test.h | 56 +++++ arch/powerpc/lib/code-patching.c | 90 +++++--- kernel/Makefile | 1 + kernel/static_call_inline.c | 43 ---- kernel/static_call_selftest.c | 41 ++++ 12 files changed, 682 insertions(+), 82 deletions(-) create mode 100644 arch/powerpc/kernel/static_call_test.c create mode 100644 arch/powerpc/kernel/static_call_test.h create mode 100644 kernel/static_call_selftest.c base-commit: 3d7a198cfdb47405cfb4a3ea523876569fe341e6 -- 2.37.3