If a CONFIG_RCU_NOCB_CPUS kernel invokes call_rcu() with interrupts disabled, wakeups must be deferred in order to avoid self-deadlock in the cases where the disabled interrupts are due to scheduler locks being held. In this case, a flag is set and is checked on entry to extended quiescent states (usermode, idle), on exit from the RCU_SOFTIRQ handler, when the CPU in question goes offline, on a subsequent invocation of call_rcu(), and from rcu_all_qs(). However, a given CPU could avoid all of those states for a considerable length of time.
This commit therefore allows an invocation of rcu_note_context_switch() to do the wakeup. It also makes the wakeup function clear the deferred-wakeup flag. Signed-off-by: Paul E. McKenney <[email protected]> --- kernel/rcu/tree.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index eaa2b6d7abe2..7fa46967021f 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -495,12 +495,16 @@ static void rcu_momentary_dyntick_idle(void) */ void rcu_note_context_switch(void) { + struct rcu_state *rsp; + barrier(); /* Avoid RCU read-side critical sections leaking down. */ trace_rcu_utilization(TPS("Start context switch")); rcu_sched_qs(); rcu_preempt_note_context_switch(); if (unlikely(raw_cpu_read(rcu_sched_qs_mask))) rcu_momentary_dyntick_idle(); + for_each_rcu_flavor(rsp) + do_nocb_deferred_wakeup(this_cpu_ptr(rsp->rda)); trace_rcu_utilization(TPS("End context switch")); barrier(); /* Avoid RCU read-side critical sections leaking up. */ } -- 2.5.2
