Currently raw_cpu_write() is used to clear the ->rcu_need_heavy_qs field of the per-CPU rcu_data structure. However, on x86 this is a normal assignment, which does not play well with concurrent accesses. This commit therefore upgrades the uses of raw_cpu_write() to its concurrency-safe counterpart WRITE_ONCE() of an rdp pointer obtained from this_cpu_ptr(&rcu_data).
KCSAN located this issue. Signed-off-by: Paul E. McKenney <[email protected]> --- kernel/rcu/tree.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 03a43d3d261607..91d5b4dd08bf5c 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -357,9 +357,10 @@ bool rcu_watching_zero_in_eqs(int cpu, int *vp) */ notrace void rcu_momentary_eqs(void) { + struct rcu_data *rdp = this_cpu_ptr(&rcu_data); int seq; - raw_cpu_write(rcu_data.rcu_need_heavy_qs, false); + WRITE_ONCE(rdp->rcu_need_heavy_qs, false); seq = ct_state_inc(2 * CT_RCU_WATCHING); /* It is illegal to call this from idle state. */ WARN_ON_ONCE(!(seq & CT_RCU_WATCHING)); -- 2.40.1
