->qsmask of an RCU leaf node is usually more sparse than the corresponding cpu_possible_mask. So replace the for_each_leaf_node_possible_cpu() in force_qs_rnp() with for_each_leaf_node_cpu() to save several checks.
[Note we need to use "1UL << bit" instead of "1 << bit" to generate the corresponding mask for a bit because @mask is unsigned long, this was spotted by Colin Ian King <[email protected]> and CoverityScan in a previous version of this patch.] Signed-off-by: Boqun Feng <[email protected]> --- kernel/rcu/tree.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 4e5b81c843de..1ef13e63bc95 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3046,13 +3046,11 @@ static void force_qs_rnp(struct rcu_state *rsp, continue; } } - for_each_leaf_node_possible_cpu(rnp, cpu) { - unsigned long bit = leaf_node_cpu_bit(rnp, cpu); - if ((rnp->qsmask & bit) != 0) { - if (f(per_cpu_ptr(rsp->rda, cpu), isidle, maxj)) - mask |= bit; - } - } + + for_each_leaf_node_cpu(rnp, rnp->qsmask, cpu) + if (f(per_cpu_ptr(rsp->rda, cpu), isidle, maxj)) + mask |= leaf_node_cpu_bit(rnp, cpu); + if (mask != 0) { /* Idle/offline CPUs, report (releases rnp->lock. */ rcu_report_qs_rnp(mask, rsp, rnp, rnp->gpnum, flags); -- 2.10.2
