On Wed, 2026-10-07 at 13:59 -0700, Paul E. McKenney wrote: > Some dedicated srcu_struct users have read-side critical sections > which are short, never sleep, and never block on anything which may > itself depend on memory allocation — because they were, until > recently, spinlock or rwlock critical sections. For such a domain the > update side can safely wait for readers by spinning, from contexts > where sleeping is undesirable or the grace-period machinery's > latency (workqueue scheduling, jiffy-paced retries) dominates the > actual reader drain time. > > But that is only safe if every reader keeps the promise. Make the > promise explicit and machine-checkable: > > - srcu_read_lock_atomic() / srcu_read_unlock_atomic() enter the > usual (smp_mb-based) read-side critical section with preemption > disabled and (except in hardirq, where it is redundant) > non_block_start() armed, recording SRCU_READ_FLAVOR_ATOMIC in the > per-CPU reader flavor. Disabling preemption enforces the > no-sleeping promise on every configuration and bounds the section, > so it is always running on some CPU; non_block_start() extends the > enforcement to even potentially-sleeping calls on paths which > happen not to block. The existing reader-flavor consistency checks > complain about any mixing with other flavors. > > - synchronize_srcu_atomic() waits for all pre-existing readers by > repeating the try_synchronize_srcu() both-epoch counter proof with > cpu_relax() until it succeeds: no sleeping, no index flip, no > grace-period sequence update, and therefore no interaction with > concurrent call_srcu(), synchronize_srcu() or srcu_barrier(). It > always provides the full grace-period guarantee: if the domain > turns out to have had readers of any other flavor — a caller bug, > since such a reader may be asleep and spinning on it would be > unbounded — it complains and falls back to a real (sleeping) grace > period internally, that being the only correct wait for a > possibly-sleeping reader. The flavor mask is rechecked on every > iteration so a first non-atomic reader appearing mid-spin takes > the same path. > > The immediate motivation is the proposed conversion of KVM's > gfn_to_pfn_cache to SRCU¹, whose mmu_notifier invalidation path drains > readers before the primary MMU zaps a page. With the readers declared > atomic, that drain becomes spin-only: no sleeping at all in the > notifier, bounded by the longest reader section, satisfying even the > strictest reading of the OOM-reaper non-blocking requirement without > needing to touch the non_block_start() annotation². > > This commit also adds atomic-SRCU-specific initializers: > init_srcu_struct_atomic(), DEFINE_SRCU_ATOMIC(), and > DEFINE_STATIC_SRCU_ATOMIC(). > > This commit implements only Tree SRCU. Tiny SRCU will follow. > > ¹ https://lore.kernel.org/all/[email protected]/ > ² https://lore.kernel.org/all/[email protected]/ > > [ paulmck: Apply Kunwu Chan feedback. ] > > Co-developed-by: David Woodhouse <[email protected]> > Signed-off-by: David Woodhouse <[email protected]> > Assisted-by: Claude:claude-mythos-5 > Signed-off-by: Paul E. McKenney <[email protected]>
Thank you for seeing this concept through to completion. If it isn't already too late (i.e. if it hasn't already made it to a tree where changing commit-ids is painful), please could you change my attribution to use the [email protected] email address for both Co- developed-by: and Signed-off-by: tags? I'm fairly sure you made up that Signed-off-by: with the infradead address, which is naughty of you :) I tell people you should only ever cut-and-paste SoB lines, because that way the magic is intact. So here are the two you need for patches 4 and 5 of this series: Signed-off-by: David Woodhouse <[email protected]> Signed-off-by: David Woodhouse <[email protected]> Thanks.
smime.p7s
Description: S/MIME cryptographic signature
