Tracing still needs a transition recursion allowance because a callback can
run after architectural hardirq entry but before generic entry updates
preempt_count(). Steven asked whether HARDIRQ_OFFSET could be installed
early enough for those callbacks to observe hardirq context. This RFC
implements that contract for arm64.
arm64 must first distinguish ordinary interrupts from pseudo-NMIs. For an
ordinary interrupt, stage the raw preempt-count addition and establish
lockdep state immediately after that decision and before instrumentable
entry work. Keep time, tick, RCU and IRQ accounting at the existing generic
boundary. With DEBUG_PREEMPT and without separate reschedule bits, an
already-negative count preserves the legacy DEBUG_LOCKS_WARN_ON() oops short
circuit, attempts deferred reporting at the instrumentable finish, and skips
post-add bookkeeping and the matching subtraction when rejected.
The exit side has a second gap. Keep any applied hardirq count and lockdep
classification through traceable generic exit work, then remove them before
a possible schedule. Carry ordinary-hardirq provenance through that schedule
in a compact three-state byte, keeping irqentry_state_t two bytes wide, and
restore a classification-only tail around architecture tracing, deferred
hrtimer work, IRQ-on preparation and context-tracking preparation. The tail
does not repeat RCU, tick or IRQ accounting.
This is a design RFC, not a merge-ready proposal. The questions are:
- Should generic entry expose separate raw-update and bookkeeping phases?
- Are preempt-count and lockdep state sufficient for the return tail?
- Is arm64 a suitable first user of this contract?
The series is based on d135ab453a74 ("Merge tag 'backlight-next-7.4' of
git://git.kernel.org/pub/scm/linux/kernel/git/lee/backlight"). Patches 1
and 2 include two arm64 noinstr inlining prerequisites so the series
applies directly to mainline:
- "arm64: Always inline raw preempt count updates"
provenance:
https://lore.kernel.org/all/c3a90368dd6c096af9d918e036a27728696fb8a4.1786603168.git.hongyan....@transsion.com/
- "arm64: percpu: Annotate __kern_my_cpu_offset() as '__always_inline'"
provenance: https://lore.kernel.org/r/[email protected]
The first is the relevant hunk extracted from Hongyan Xia's larger posted
RFC. The second is a current-mainline rebase of Will Deacon's posted patch.
They preserve the authorship and links of the original submissions.
Repository: https://github.com/kmehltretter82/linux
prerequisites: steve-early-hardirq-arm64-prereqs-v2-20261027
8bdce63fa17d
generic core: steve-early-hardirq-core-v2-folded-20261027
8651af55ecbc
arm64 RFC: steve-early-hardirq-arm64-rfc-v2-folded-20261027
56016d197e1b
The transition recursion slots remain unchanged. A separate
recursion-lifetime series keeps the task recursion bit inside one
preemption-disabled interval. A development-only patch removed the slots
for a matched experiment. Neither that removal nor the lifetime series is
part of this RFC.
All three kernels in the experiment contained this early-hardirq series.
On a native 16-vCPU Google Axion system, matched arm64 debug kernels
completed 100 deterministic ftrace callback and IPI overlaps per cell:
done litmus mixed collisions
Slots kept, lifetime fix 100/100 0 0 0
Slots removed, no fix 100/100 33 60 13
Slots removed, lifetime 100/100 0 0 0
The litmus and mixed columns count ftrace recursion records. The collisions
column counts transition-sensitive scheduler and return-tail records.
Complete dmesg captures were clean in all three cells.
The second cell is a counterexample to removing the slots without the
lifetime fix in the tested path. The third supports the combined design on
this machine. It does not establish every architecture-specific early-entry
interval, so this RFC retains the slots and keeps the lifetime work
separate.
Before this rebase, the patch-identical tree was validated across a
nine-cell cross-architecture W=1 full-link matrix: RISC-V 64, i386,
powerpc64le, arm64, x86-64 and ARM32 with GCC 13.3 and Clang/LLD 18.1.3. It
also passed 600-second arm64 SMP8 and RISC-V SMP14 stress gates, a RISC-V
AIA/IMSIC SMP1/SMP2/SMP8 gate under QEMU 11.1.50, and a focused four-row
DEBUG_PREEMPT staged-add oops oracle with 403 of 403 locking selftests.
An ARM64 PREEMPT_RT SMP4 build passed targeted return-tail tasklet and
forced-thread timer closure tests, three SMP2 priority-masked pseudo-NMI
return runs, and an SMP1 NO_HZ_IDLE and RCU_EQS_DEBUG return gate with 386
Startirq-to-Endirq intervals and 1,229 IRQ-disable/enable pairs. All
patches remain unchanged according to git range-diff after the rebase.
Changes since v1:
- rebase onto d135ab453a74 and include the two arm64 noinstr inlining
prerequisites as patches 1 and 2 so the series applies directly to
mainline
- keep the recursion-lifetime change in a separate series
- carry a scoped hardirq-entry cookie across staged count updates so
!HAS_SEPARATE_PREEMPT_RESCHED_BITS && DEBUG_PREEMPT configurations
preserve the legacy DEBUG_LOCKS_WARN_ON() oops decision and skip
post-add work and teardown when rejected
- encode ordinary-hardirq provenance and staged-count application in a
three-state byte so irqentry_state_t remains two bytes wide
- fully initialize the arm64 EL1 debug entry state before provenance is
added
- retain irqentry_exit_pcpu_adjust() ordering across the return tail
- accept and test nonzero nested lockdep hardirq depth in the tail
- add a matched native test of the separate recursion-lifetime work and
- tighten the contract and documentation wording
v1: https://lore.kernel.org/all/[email protected]/
Assisted-by: LLM
Hongyan Xia (1):
arm64: Always inline raw preempt count updates
Karl Mehltretter (11):
sched: Split preempt-count bookkeeping from raw updates
lockdep: Use raw access for pinned hardirq state
entry: Add the pre-counted ordinary-hardirq entry/exit contract
entry: Skip inactive IRQ-return reschedule calls
context_tracking: Split IRQ-exit preparation from the EQS transition
lockdep: Prepare IRQ-on state from a hardirq return tail
entry: Carry ordinary-hardirq provenance through kernel return
entry: Classify ordinary hardirq return-tail work as hardirq context
arm64: Use the pre-counted hardirq lifecycle
Documentation: entry: Describe the pre-counted hardirq lifecycle
locking/selftest: Exercise hardirq return-tail preparation
Will Deacon (1):
arm64: percpu: Annotate __kern_my_cpu_offset() as '__always_inline'
Documentation/core-api/entry.rst | 89 +++++++++++--
arch/arm64/include/asm/percpu.h | 2 +-
arch/arm64/include/asm/preempt.h | 6 +-
arch/arm64/kernel/entry-common.c | 34 +++--
include/linux/context_tracking_irq.h | 32 ++++-
include/linux/hardirq.h | 33 +++++
include/linux/interrupt.h | 2 +-
include/linux/irq-entry-common.h | 130 +++++++++++++++++--
include/linux/irqentry_hardirq.h | 81 ++++++++++++
include/linux/irqflags.h | 14 ++-
kernel/context_tracking.c | 78 +++++++++---
kernel/entry/common.c | 45 ++++++-
kernel/locking/lockdep.c | 49 +++++---
kernel/sched/core.c | 101 +++++++++++++--
kernel/softirq.c | 181 +++++++++++++++++++++++----
lib/locking-selftest.c | 53 ++++++++
16 files changed, 828 insertions(+), 102 deletions(-)
create mode 100644 include/linux/irqentry_hardirq.h
base-commit: d135ab453a7400a34930bdda1ef4bdc8acfac80e
--
2.39.5 (Apple Git-154)
