On Thu, Jul 30, 2015 at 06:22:02PM +0200, Peter Zijlstra wrote:
> On Thu, Jul 30, 2015 at 08:54:54AM -0700, Paul E. McKenney wrote:
>
> > Good point, and it already does, and I clearly was confused, apologies.
> >
> > So the real way to make this happen is (for example) to build
> > with CONFIG_RCU_FANOUT=2 and CONFIG_RCU_FANOUT_LEAF=16 (the
> > default), which could accommodate up to 128 CPUs. Then boot with
> > rcutree.rcu_fanout_leaf=2 on a system with more than 16 CPUs, with
> > rcutree.rcu_fanout_leaf=3 on a system with more than 24 CPUs, and so on.
>
> Ah, runtime overrides and operator error, but then we can WARN(), reset
> the arguments and try again, right? No need to panic the machine and
> fail to boot.
Good point, like the patch below? Which also legitimizes my
example after the fact, as it previously simply prohibited having
rcutree.rcu_fanout_leaf less than CONFIG_RCU_FANOUT_LEAF. :-/
> > Of course, the truly macho way to get this error message is to build
> > with CONFIG_RCU_FANOUT=64 and CONFIG_RCU_FANOUT_LEAF=64, then boot with
> > rcutree.rcu_fanout_leaf=63 on a system with more than 16,515,072 CPUs.
> > Of course, you get serious style points if the system manages to stay
> > up for more than 24 hours without a hardware failure. ;-)
>
> Yes, I'll go power up the nuclear reactor in the basement first :-)
Only one? ;-)
Thanx, Paul
------------------------------------------------------------------------
rcu: Eliminate panic when silly boot-time fanout specified
This commit loosens rcutree.rcu_fanout_leaf range checks
and replaces a panic() with a fallback to compile-time values.
This fallback is accompanied by a WARN_ON(), and both occur when the
rcutree.rcu_fanout_leaf value is too small to accommodate the number of
CPUs. For example, given the current four-level limit for the rcu_node
tree, a system with more than 16 CPUs built with CONFIG_FANOUT=2 must
have rcutree.rcu_fanout_leaf larger than 2.
Reported-by: Peter Zijlstra <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
diff --git a/Documentation/kernel-parameters.txt
b/Documentation/kernel-parameters.txt
index 01b5b68a237a..2a5d4696bdb9 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -3059,9 +3059,12 @@ bytes respectively. Such letter suffixes can also be
entirely omitted.
cache-to-cache transfer latencies.
rcutree.rcu_fanout_leaf= [KNL]
- Increase the number of CPUs assigned to each
- leaf rcu_node structure. Useful for very large
- systems.
+ Change the number of CPUs assigned to each
+ leaf rcu_node structure. Useful for very
+ large systems, which will choose the value 64,
+ and for NUMA systems with large remote-access
+ latencies, which will choose a value aligned
+ with the appropriate hardware boundaries.
rcutree.jiffies_till_sched_qs= [KNL]
Set required age in jiffies for a
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index ce43fac5ff91..9f8040396d3e 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -4216,13 +4216,12 @@ static void __init rcu_init_geometry(void)
rcu_fanout_leaf, nr_cpu_ids);
/*
- * The boot-time rcu_fanout_leaf parameter is only permitted
- * to increase the leaf-level fanout, not decrease it. Of course,
- * the leaf-level fanout cannot exceed the number of bits in
- * the rcu_node masks. Complain and fall back to the compile-
- * time values if these limits are exceeded.
+ * The boot-time rcu_fanout_leaf parameter must be at least two
+ * and cannot exceed the number of bits in the rcu_node masks.
+ * Complain and fall back to the compile-time values if this
+ * limit is exceeded.
*/
- if (rcu_fanout_leaf < RCU_FANOUT_LEAF ||
+ if (rcu_fanout_leaf < 2 ||
rcu_fanout_leaf > sizeof(unsigned long) * 8) {
rcu_fanout_leaf = RCU_FANOUT_LEAF;
WARN_ON(1);
@@ -4239,10 +4238,13 @@ static void __init rcu_init_geometry(void)
/*
* The tree must be able to accommodate the configured number of CPUs.
- * If this limit is exceeded than we have a serious problem elsewhere.
+ * If this limit is exceeded, fall back to the compile-time values.
*/
- if (nr_cpu_ids > rcu_capacity[RCU_NUM_LVLS - 1])
- panic("rcu_init_geometry: rcu_capacity[] is too small");
+ if (nr_cpu_ids > rcu_capacity[RCU_NUM_LVLS - 1]) {
+ rcu_fanout_leaf = RCU_FANOUT_LEAF;
+ WARN_ON(1);
+ return;
+ }
/* Calculate the number of levels in the tree. */
for (i = 0; nr_cpu_ids > rcu_capacity[i]; i++) {
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
