Re: [PATCH] powerpc/smp: Use GFP_ATOMIC while allocating tmp mask
Srikar Dronamraju writes:
> Qian Cai reported a regression where CPU Hotplug fails with the latest
> powerpc/next
>
> BUG: sleeping function called from invalid context at mm/slab.h:494
> in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 0, name: swapper/88
> no locks held by swapper/88/0.
> irq event stamp: 18074448
> hardirqs last enabled at (18074447): []
> tick_nohz_idle_enter+0x9c/0x110
> hardirqs last disabled at (18074448): [] do_idle+0x138/0x3b0
> do_idle at kernel/sched/idle.c:253 (discriminator 1)
> softirqs last enabled at (18074440): []
> irq_enter_rcu+0x94/0xa0
> softirqs last disabled at (18074439): []
> irq_enter_rcu+0x70/0xa0
> CPU: 88 PID: 0 Comm: swapper/88 Tainted: GW
> 5.9.0-rc8-next-20201007 #1
> Call Trace:
> [c0002a4bfcf0] [c0649e98] dump_stack+0xec/0x144 (unreliable)
> [c0002a4bfd30] [c00f6c34] ___might_sleep+0x2f4/0x310
> [c0002a4bfdb0] [c0354f94]
> slab_pre_alloc_hook.constprop.82+0x124/0x190
> [c0002a4bfe00] [c035e9e8] __kmalloc_node+0x88/0x3a0
> slab_alloc_node at mm/slub.c:2817
> (inlined by) __kmalloc_node at mm/slub.c:4013
> [c0002a4bfe80] [c06494d8] alloc_cpumask_var_node+0x38/0x80
> kmalloc_node at include/linux/slab.h:577
> (inlined by) alloc_cpumask_var_node at lib/cpumask.c:116
> [c0002a4bfef0] [c003eedc] start_secondary+0x27c/0x800
> update_mask_by_l2 at arch/powerpc/kernel/smp.c:1267
> (inlined by) add_cpu_to_masks at arch/powerpc/kernel/smp.c:1387
> (inlined by) start_secondary at arch/powerpc/kernel/smp.c:1420
> [c0002a4bff90] [c000c468] start_secondary_resume+0x10/0x14
>
> Allocating a temporary mask while performing a CPU Hotplug operation
> with CONFIG_CPUMASK_OFFSTACK enabled, leads to calling a sleepable
> function from a atomic context. Fix this by allocating the temporary
> mask with GFP_ATOMIC flag.
>
> If there is a failure to allocate a mask, scheduler is going to observe
> that this CPU's topology is broken. Instead of having to speculate why
> the topology is broken, add a WARN_ON_ONCE.
>
> Fixes: 70a94089d7f7 ("powerpc/smp: Optimize update_coregroup_mask")
> Fixes: 3ab33d6dc3e9 ("powerpc/smp: Optimize update_mask_by_l2")
> Reported-by: Qian Cai
> Suggested-by: Qian Cai
> Signed-off-by: Srikar Dronamraju
> Cc: linuxppc-dev
> Cc: LKML
> Cc: Michael Ellerman
> Cc: Nathan Lynch
> Cc: Gautham R Shenoy
> Cc: Ingo Molnar
> Cc: Peter Zijlstra
> Cc: Valentin Schneider
> Cc: Qian Cai
> ---
> arch/powerpc/kernel/smp.c | 6 --
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
> index 0dc1b85..1268558 100644
> --- a/arch/powerpc/kernel/smp.c
> +++ b/arch/powerpc/kernel/smp.c
> @@ -1264,7 +1264,8 @@ static bool update_mask_by_l2(int cpu)
> return false;
> }
>
> - alloc_cpumask_var_node(, GFP_KERNEL, cpu_to_node(cpu));
> + /* In CPU-hotplug path, hence use GFP_ATOMIC */
> + WARN_ON_ONCE(!alloc_cpumask_var_node(, GFP_ATOMIC,
> cpu_to_node(cpu)));
A failed memory allocation is not something that should trigger a WARN,
a pr_warn() maybe.
But ...
> cpumask_and(mask, cpu_online_mask, cpu_cpu_mask(cpu));
If the allocation failed this will oops (mask will be NULL).
cheers
[PATCH] powerpc/smp: Use GFP_ATOMIC while allocating tmp mask
Qian Cai reported a regression where CPU Hotplug fails with the latest
powerpc/next
BUG: sleeping function called from invalid context at mm/slab.h:494
in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 0, name: swapper/88
no locks held by swapper/88/0.
irq event stamp: 18074448
hardirqs last enabled at (18074447): []
tick_nohz_idle_enter+0x9c/0x110
hardirqs last disabled at (18074448): [] do_idle+0x138/0x3b0
do_idle at kernel/sched/idle.c:253 (discriminator 1)
softirqs last enabled at (18074440): []
irq_enter_rcu+0x94/0xa0
softirqs last disabled at (18074439): []
irq_enter_rcu+0x70/0xa0
CPU: 88 PID: 0 Comm: swapper/88 Tainted: GW
5.9.0-rc8-next-20201007 #1
Call Trace:
[c0002a4bfcf0] [c0649e98] dump_stack+0xec/0x144 (unreliable)
[c0002a4bfd30] [c00f6c34] ___might_sleep+0x2f4/0x310
[c0002a4bfdb0] [c0354f94]
slab_pre_alloc_hook.constprop.82+0x124/0x190
[c0002a4bfe00] [c035e9e8] __kmalloc_node+0x88/0x3a0
slab_alloc_node at mm/slub.c:2817
(inlined by) __kmalloc_node at mm/slub.c:4013
[c0002a4bfe80] [c06494d8] alloc_cpumask_var_node+0x38/0x80
kmalloc_node at include/linux/slab.h:577
(inlined by) alloc_cpumask_var_node at lib/cpumask.c:116
[c0002a4bfef0] [c003eedc] start_secondary+0x27c/0x800
update_mask_by_l2 at arch/powerpc/kernel/smp.c:1267
(inlined by) add_cpu_to_masks at arch/powerpc/kernel/smp.c:1387
(inlined by) start_secondary at arch/powerpc/kernel/smp.c:1420
[c0002a4bff90] [c000c468] start_secondary_resume+0x10/0x14
Allocating a temporary mask while performing a CPU Hotplug operation
with CONFIG_CPUMASK_OFFSTACK enabled, leads to calling a sleepable
function from a atomic context. Fix this by allocating the temporary
mask with GFP_ATOMIC flag.
If there is a failure to allocate a mask, scheduler is going to observe
that this CPU's topology is broken. Instead of having to speculate why
the topology is broken, add a WARN_ON_ONCE.
Fixes: 70a94089d7f7 ("powerpc/smp: Optimize update_coregroup_mask")
Fixes: 3ab33d6dc3e9 ("powerpc/smp: Optimize update_mask_by_l2")
Reported-by: Qian Cai
Suggested-by: Qian Cai
Signed-off-by: Srikar Dronamraju
Cc: linuxppc-dev
Cc: LKML
Cc: Michael Ellerman
Cc: Nathan Lynch
Cc: Gautham R Shenoy
Cc: Ingo Molnar
Cc: Peter Zijlstra
Cc: Valentin Schneider
Cc: Qian Cai
---
arch/powerpc/kernel/smp.c | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 0dc1b85..1268558 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1264,7 +1264,8 @@ static bool update_mask_by_l2(int cpu)
return false;
}
- alloc_cpumask_var_node(, GFP_KERNEL, cpu_to_node(cpu));
+ /* In CPU-hotplug path, hence use GFP_ATOMIC */
+ WARN_ON_ONCE(!alloc_cpumask_var_node(, GFP_ATOMIC,
cpu_to_node(cpu)));
cpumask_and(mask, cpu_online_mask, cpu_cpu_mask(cpu));
if (has_big_cores)
@@ -1344,7 +1345,8 @@ static void update_coregroup_mask(int cpu)
int coregroup_id = cpu_to_coregroup_id(cpu);
int i;
- alloc_cpumask_var_node(, GFP_KERNEL, cpu_to_node(cpu));
+ /* In CPU-hotplug path, hence use GFP_ATOMIC */
+ WARN_ON_ONCE(!alloc_cpumask_var_node(, GFP_ATOMIC,
cpu_to_node(cpu)));
cpumask_and(mask, cpu_online_mask, cpu_cpu_mask(cpu));
if (shared_caches)
--
1.8.3.1
