RISC-V enables interrupts in handle_page_fault() before checking whether the fault occurred with fault handling disabled. A nofault access from an atomic context can therefore run tracepoints and open an interrupt window before reaching the exception-table fixup.
Handle an exception-table entry before entering the generic fault path when fault handling is disabled. Also keep interrupts disabled until such a fault has been resolved. This makes RISC-V consistent with the expectation that an in-atomic nofault access does not enter the normal fault-handling path. It also removes one source of re-entry when perf sampling is delivered through an SBI Supervisor Software Event (SSE). Signed-off-by: Zhanpeng Zhang <[email protected]> --- arch/riscv/mm/fault.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 04ed6f8acae4..520495420462 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -294,6 +294,13 @@ void handle_page_fault(struct pt_regs *regs) if (kprobe_page_fault(regs, cause)) return; + /* + * Nofault accesses must be resolved through the exception table before + * entering the generic fault path or enabling interrupts. + */ + if (unlikely(faulthandler_disabled()) && fixup_exception(regs)) + return; + if (user_mode(regs)) trace_page_fault_user(addr, regs, cause); else @@ -314,8 +321,8 @@ void handle_page_fault(struct pt_regs *regs) return; } - /* Enable interrupts if they were enabled in the parent context. */ - if (!regs_irqs_disabled(regs)) + /* Do not open an interrupt window before a nofault fixup completes. */ + if (!regs_irqs_disabled(regs) && !faulthandler_disabled()) local_irq_enable(); /* -- 2.50.1 (Apple Git-155)
