On Sat, Sep 12, 2026 at 8:41 AM Zqiang <[email protected]> wrote:
>
> >
> > On Thu, Sep 10, 2026 at 8:16 PM Zqiang <[email protected]> wrote:
> >
> > >
> > > From: Kunwu Chan <[email protected]>
> > >
> > > This series rounds out atomic SRCU support with a reader-free fastpath,
> > > some atomic-context fixes, and additional test coverage.
> > >
> > > The core change is a reader-free fastpath for synchronize_srcu_atomic().
> > > Atomic SRCU's read-side critical sections disable preemption, so when
> > > the summed lock and unlock counts match for both ranks,
> > > synchronize_srcu_atomic() can end the grace period immediately, without
> > > the index flip or the srcu_advance_state() scans. Correctness depends
> > > on the grace-period anchor written by srcu_gp_start() preceding the
> > > lock-count scan; two LKMM litmus tests (patches 1-2) verify that
> > > ordering.
> > >
> > > For synchronize_srcu_atomic(), the following items maybe also
> > > need to be added:
> > >
> > Hi Zqiang
> > Sent as:
> > https://lore.kernel.org/rcu/[email protected]/
> >
> > Did a small late_initcall that enters an atomic read-side critical section
> > and
> > calls synchronize_srcu_atomic(). srcu_lock_sync() catches it:
> > WARNING: possible recursive locking detected
> > swapper/0/1 is trying to acquire lock: (&ss) at:
> > synchronize_srcu_atomic+0x28/0x380
> > but task is already holding lock: (&ss) at:
> > srcu_lockdep_test_init+0xd0/0x170
> > *** DEADLOCK ***
> > Rcutorture SRCU-N/P also pass.
>
> Hi, kunwu
>
> You can add srcu_atomic lockdep test to rcutorture to verify
> your change, and I also saw that Paul suggested the same thing :)
>
Hi Zqiang,
Thanks for the suggestion. This is very helpful.
I saw Paul's suggestion too. I'll use your example as a starting
point, verify the atomic SRCU lockdep cases, and check the
--do-srcu-lockdep handling in torture.sh, including the normal RCU
read-side case Paul mentioned.
I'll add a Suggested-by tag for your contribution when I post the
changes.
Thanks,
Kunwu
> For example, as shown below (I am not test or verify it):
>
> diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
> index d795e07a12ad..d4f7399acbd5 100644
> --- a/kernel/rcu/rcutorture.c
> +++ b/kernel/rcu/rcutorture.c
> @@ -4662,6 +4662,17 @@ static DECLARE_RWSEM(rwsem7);
> static DECLARE_RWSEM(rwsem8);
> static DECLARE_RWSEM(rwsem9);
>
> +static DEFINE_RAW_SPINLOCK(spin0);
> +static DEFINE_RAW_SPINLOCK(spin1);
> +static DEFINE_RAW_SPINLOCK(spin2);
> +static DEFINE_RAW_SPINLOCK(spin3);
> +static DEFINE_RAW_SPINLOCK(spin4);
> +static DEFINE_RAW_SPINLOCK(spin5);
> +static DEFINE_RAW_SPINLOCK(spin6);
> +static DEFINE_RAW_SPINLOCK(spin7);
> +static DEFINE_RAW_SPINLOCK(spin8);
> +static DEFINE_RAW_SPINLOCK(spin9);
> +
> DEFINE_STATIC_SRCU(srcu0);
> DEFINE_STATIC_SRCU(srcu1);
> DEFINE_STATIC_SRCU(srcu2);
> @@ -4673,6 +4684,17 @@ DEFINE_STATIC_SRCU(srcu7);
> DEFINE_STATIC_SRCU(srcu8);
> DEFINE_STATIC_SRCU(srcu9);
>
> +DEFINE_STATIC_SRCU_ATOMIC(srcu0_atomic);
> +DEFINE_STATIC_SRCU_ATOMIC(srcu1_atomic);
> +DEFINE_STATIC_SRCU_ATOMIC(srcu2_atomic);
> +DEFINE_STATIC_SRCU_ATOMIC(srcu3_atomic);
> +DEFINE_STATIC_SRCU_ATOMIC(srcu4_atomic);
> +DEFINE_STATIC_SRCU_ATOMIC(srcu5_atomic);
> +DEFINE_STATIC_SRCU_ATOMIC(srcu6_atomic);
> +DEFINE_STATIC_SRCU_ATOMIC(srcu7_atomic);
> +DEFINE_STATIC_SRCU_ATOMIC(srcu8_atomic);
> +DEFINE_STATIC_SRCU_ATOMIC(srcu9_atomic);
> +
> static int srcu_lockdep_next(const char *f, const char *fl, const char *fs,
> const char *fu, int i,
> int cyclelen, int deadlock)
> {
> @@ -4702,6 +4724,13 @@ static void rcu_torture_init_srcu_lockdep(void)
> &rwsem5, &rwsem6, &rwsem7, &rwsem8,
> &rwsem9 };
> struct srcu_struct *srcus[] = { &srcu0, &srcu1, &srcu2, &srcu3,
> &srcu4,
> &srcu5, &srcu6, &srcu7, &srcu8,
> &srcu9 };
> +
> + raw_spinlock_t *spins[] = { &spin0, &spin1, &spin2, &spin3, &spin4,
> &spin5,
> + &spin6, &spin7, &spin8, &spin9};
> + struct srcu_struct *srcus_atomic[] = {&srcu0_atomic, &srcu1_atomic,
> &srcu2_atomic,
> + &srcu3_atomic, &srcu4_atomic,
> &srcu5_atomic,
> + &srcu6_atomic, &srcu7_atomic,
> &srcu8_atomic,
> + &srcu9_atomic};
> int testtype;
>
> if (!test_srcu_lockdep)
> @@ -4812,11 +4841,48 @@ static void rcu_torture_init_srcu_lockdep(void)
> }
> #endif // #ifdef CONFIG_TASKS_TRACE_RCU
>
> + if (testtype == 4) {
> + pr_info("%s: test_srcu_lockdep = %05d: SRCU_ATOMIC %d-way
> %sdeadlock.\n",
> + __func__, test_srcu_lockdep, cyclelen, deadlock ? ""
> : "non-");
> + if (deadlock && cyclelen == 1)
> + pr_info("%s: Expect hang.\n", __func__);
> + for (i = 0; i < cyclelen; i++) {
> + j = srcu_lockdep_next(__func__,
> "srcu_read_lock_atomic", "synchronize_srcu_atomic",
> + "srcu_read_unlock_atomic", i,
> cyclelen, deadlock);
> + idx = srcu_read_lock_atomic(srcus_atomic[i]);
> + if (j >= 0)
> + synchronize_srcu_atomic(srcus_atomic[j]);
> + srcu_read_unlock_atomic(srcus_atomic[i], idx);
> + }
> + return;
> + }
> +
> + if (testtype == 1) {
> + pr_info("%s: test_srcu_lockdep = %05d:
> SRCU_ATOMIC/raw_spinlock %d-way %sdeadlock.\n",
> + __func__, test_srcu_lockdep, cyclelen, deadlock ? ""
> : "non-");
> + for (i = 0; i < cyclelen; i++) {
> + pr_info("%s: srcu_read_lock_atomic(%d),
> raw_spin_lock(%d), raw_spin_unlock(%d), srcu_read_unlock_atomic(%d)\n",
> + __func__, i, i, i, i);
> + idx = srcu_read_lock_atomic(srcus_atomic[i]);
> + raw_spin_lock(spins[i]);
> + raw_spin_unlock(spins[i]);
> + srcu_read_unlock_atomic(srcus_atomic[i], idx);
> +
> + j = srcu_lockdep_next(__func__, "raw_spin_lock",
> "synchronize_srcu_atomic",
> + "raw_spin_unlock", i,
> cyclelen, deadlock);
> + raw_spin_lock(spins[i]);
> + if (j >= 0)
> + synchronize_srcu_atomic(srcus_atomic[j]);
> + raw_spin_unlock(spins[i]);
> + }
> + return;
> + }
> +
> err_out:
> pr_info("%s: test_srcu_lockdep = %05d does nothing.\n", __func__,
> test_srcu_lockdep);
> pr_info("%s: test_srcu_lockdep = DNNL.\n", __func__);
> pr_info("%s: D: Deadlock if nonzero.\n", __func__);
> - pr_info("%s: NN: Test number, 0=SRCU, 1=SRCU/mutex, 2=SRCU/rwsem,
> 3=SRCU/Tasks Trace RCU.\n", __func__);
> + pr_info("%s: NN: Test number, 0=SRCU, 1=SRCU/mutex, 2=SRCU/rwsem,
> 3=SRCU/Tasks Trace RCU, 4=SRCU_ATOMIC, 5=SRCU_ATOMIC/raw_spinlock.\n",
> __func__);
> pr_info("%s: L: Cycle length.\n", __func__);
> if (!IS_ENABLED(CONFIG_TASKS_TRACE_RCU))
> pr_info("%s: NN=3 disallowed because kernel is built with
> CONFIG_TASKS_TRACE_RCU=n\n", __func__);
>
>
> Thanks
> Zqiang
>
>
>
>
>
> >
> > Thanks, Kunwu
> >
> > >
> > > srcu_lock_sync(&ssp->dep_map);
> > >
> > > RCU_LOCKDEP_WARN(lockdep_is_held(ssp),
> > > "Illegal synchronize_srcu() in same-type SRCU read-side critical
> > > section");
> > >
> > > if (rcu_scheduler_active == RCU_SCHEDULER_INACTIVE)
> > > return;
> > >
> > > Thanks
> > > Zqiang
> > >
> > > The series is:
> > >
> > > 1-2 LKMM litmus tests for the reader-free fastpath ordering.
> > > 3 srcutree: add the reader-free fastpath.
> > > 4 rcutorture: add --do-atomic-srcu to torture.sh.
> > > 5-7 srcutree: honor and propagate is_atomic so atomic SRCU never
> > > transitions to big.
> > > 8 srcutree: forbid srcu_expedite_current() on atomic SRCU.
> > > 9 rcutorture: disable srcu_expedite_current() for atomic SRCU.
> > > 10 srcutree: skip callback scheduling for atomic SRCU grace periods.
> > > 11 srcutree: drop the srcu_barrier() sleep for atomic SRCU.
> > > 12 srcutree: remove leftover debug pr_alert()s.
> > > 13 srcu: restrict the non_block annotation to task context, fixing a
> > > KCSAN data race.
> > >
> > > The KCSAN race in patch 13 is a data race on the interrupted task's
> > > ->non_block_count when a timer callback runs atomic-SRCU readers in
> > > inline softirq context.
> > >
> > > Tested with herd7 7.58 (litmus tests), torture.sh --do-atomic-srcu, and
> > > torture.sh --do-atomic-srcu --do-kcsan (KCSAN+PROVE_LOCKING).
> > >
> > > Kunwu Chan (13):
> > > litmus: Add SRCU fastpath anchor-before-scan test
> > > litmus: Add SRCU fastpath scan-before-anchor test
> > > srcutree: Add reader-free fastpath to synchronize_srcu_atomic()
> > > rcutorture: Add atomic-SRCU support to torture.sh
> > > srcutree: Honor is_atomic in check_init_srcu_struct()
> > > srcutree: Make init_srcu_struct_atomic() prevent transition to big
> > > srcutree: Don't transition atomic SRCU to big in srcu_gp_end()
> > > srcutree: Forbid srcu_expedite_current() on atomic SRCU
> > > rcutorture: Disable srcu_expedite_current() for atomic SRCU
> > > srcutree: Skip callback scheduling for atomic SRCU grace periods
> > > srcutree: Remove srcu_barrier() sleep for atomic SRCU
> > > srcutree: Remove debug pr_alert()s
> > > srcu: Restrict atomic-SRCU non_block annotation to task context
> > >
> > > include/linux/srcu.h | 10 +-
> > > kernel/rcu/rcutorture.c | 1 +
> > > kernel/rcu/srcutree.c | 95 ++++++++++++++-----
> > > tools/memory-model/litmus-tests/README | 16 ++++
> > > .../SRCU-fastpath-anchor-before-scan.litmus | 56 +++++++++++
> > > .../SRCU-fastpath-scan-before-anchor.litmus | 53 +++++++++++
> > > .../selftests/rcutorture/bin/torture.sh | 24 +++++
> > > 7 files changed, 229 insertions(+), 26 deletions(-)
> > > create mode 100644
> > > tools/memory-model/litmus-tests/SRCU-fastpath-anchor-before-scan.litmus
> > > create mode 100644
> > > tools/memory-model/litmus-tests/SRCU-fastpath-scan-before-anchor.litmus
> > >
> > > base-commit: ef15070f96c5f07de3a9b593c36f6e83bc9e3d58
> > > --
> > > 2.43.0
> > >
> >
