__riscv_copy_words_unaligned() and __riscv_copy_bytes_unaligned() are
called indirectly through function pointers from measure_cycles() (via
compare_unaligned_access()) during the boot-time unaligned-access
probe. Under kCFI, an indirect call checks the type id stored in the
word immediately preceding the callee against the type id of the
function-pointer type at the call site. These two routines are defined
with SYM_FUNC_START(), which emits no __cfi_ prefix, so the check reads
the alignment padding ahead of the function (zero) instead of a type id
and traps:
CFI failure at measure_cycles.constprop.0+0x34
(target: __riscv_copy_words_unaligned+0x0; expected type: 0x00000000)
Kernel panic - not syncing: Fatal exception in interrupt
The call site is correct: it loads the actual type id from [target-4]
and compares against the expected 0xf1200a56, which matches the
compiler-generated __kcfi_typeid___riscv_copy_words_unaligned. Only the
callee is missing its prefix word.
Switch both routines to SYM_TYPED_FUNC_START() so the assembler emits
the __cfi_ type-id prefix, matching the existing treatment of other
indirectly-called riscv assembly routines (e.g. ftrace_stub in
mcount.S and __cpu_resume_enter in suspend_entry.S), and add the
<linux/cfi_types.h> include that provides the macro.
Build and boot tested ARCH=riscv defconfig+CONFIG_CFI=y with GCC
17.0.0 20260615 (experimental kCFI tree) under qemu; the boot-time
unaligned-access probe no longer traps.
Fixes: 584ea6564bca ("RISC-V: Probe for unaligned access speed")
Assisted-by: Claude:claude-opus-4-8[1m]
Signed-off-by: Kees Cook <[email protected]>
---
arch/riscv/kernel/copy-unaligned.S | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/riscv/kernel/copy-unaligned.S
b/arch/riscv/kernel/copy-unaligned.S
index 2b3d9398c113..c649ad3d19e0 100644
--- a/arch/riscv/kernel/copy-unaligned.S
+++ b/arch/riscv/kernel/copy-unaligned.S
@@ -2,6 +2,7 @@
/* Copyright (C) 2023 Rivos Inc. */
#include <linux/linkage.h>
+#include <linux/cfi_types.h>
#include <asm/asm.h>
.text
@@ -9,7 +10,7 @@
/* void __riscv_copy_words_unaligned(void *, const void *, size_t) */
/* Performs a memcpy without aligning buffers, using word loads and stores. */
/* Note: The size is truncated to a multiple of 8 * SZREG */
-SYM_FUNC_START(__riscv_copy_words_unaligned)
+SYM_TYPED_FUNC_START(__riscv_copy_words_unaligned)
andi a4, a2, ~((8*SZREG)-1)
beqz a4, 2f
add a3, a1, a4
@@ -41,7 +42,7 @@ SYM_FUNC_END(__riscv_copy_words_unaligned)
/* void __riscv_copy_bytes_unaligned(void *, const void *, size_t) */
/* Performs a memcpy without aligning buffers, using only byte accesses. */
/* Note: The size is truncated to a multiple of 8 */
-SYM_FUNC_START(__riscv_copy_bytes_unaligned)
+SYM_TYPED_FUNC_START(__riscv_copy_bytes_unaligned)
andi a4, a2, ~(8-1)
beqz a4, 2f
add a3, a1, a4
--
2.34.1
