On Thu, Apr 13, 2017 at 08:23:09PM +0200, Peter Zijlstra wrote: > On Thu, Apr 13, 2017 at 11:19:26AM -0700, Paul E. McKenney wrote: > > > First get me some system-level data showing that the current layout is > > causing a real problem. RCU's fastpath code doesn't come anywhere near > > the rcu_node tree, so in the absence of such data, I of course remain > > quite doubtful that there is a real need. And painfully aware of the > > required increase in complexity. > > > > But if there is a real need demonstrated by real system-level data, > > I will of course make the needed changes, as I have done many times in > > the past in response to other requests. > > I read what you wrote here: > > > > > Increasing it reduces the number of rcu_node structures, and thus the > > > > number of cache misses during grace-period initialization and cleanup. > > > > This has proven necessary in the past on large machines having long > > > > memory latencies. And there are starting to be some pretty big machines > > > > running in production, and even for typical commerical workloads. > > to mean you had exactly that pain. Or am I now totally not understanding > you?
I believe that you are missing the fact that RCU grace-period initialization and cleanup walks through the rcu_node tree breadth first, using rcu_for_each_node_breadth_first(). This macro (shown below) implements this breadth-first walk using a simple sequential traversal of the ->node[] array that provides the structures making up the rcu_node tree. As you can see, this scan is completely independent of how CPU numbers might be mapped to rcu_data slots in the leaf rcu_node structures. Thanx, Paul /* * Do a full breadth-first scan of the rcu_node structures for the * specified rcu_state structure. */ #define rcu_for_each_node_breadth_first(rsp, rnp) \ for ((rnp) = &(rsp)->node[0]; \ (rnp) < &(rsp)->node[rcu_num_nodes]; (rnp)++)