Re: [PATCH] powerpc/cpu: nr_cpu_ids should be aligned on threads_per_core
Hi Pingfan, Thank you for the patch! Yet something to improve: [auto build test ERROR on powerpc/next] [also build test ERROR on v4.17-rc5 next-20180511] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Pingfan-Liu/powerpc-cpu-nr_cpu_ids-should-be-aligned-on-threads_per_core/20180514-141629 base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next config: powerpc-allnoconfig (attached as .config) compiler: powerpc-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0 reproduce: wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # save the attached .config to linux build tree make.cross ARCH=powerpc All errors (new ones prefixed by >>): arch/powerpc/kernel/prom.c: In function 'early_init_dt_scan_cpus': >> arch/powerpc/kernel/prom.c:370:14: error: lvalue required as left operand of >> assignment nr_cpu_ids = nr_cpus_aligned; ^ vim +370 arch/powerpc/kernel/prom.c 306 307 static int __init early_init_dt_scan_cpus(unsigned long node, 308const char *uname, int depth, 309void *data) 310 { 311 const char *type = of_get_flat_dt_prop(node, "device_type", NULL); 312 const __be32 *prop; 313 const __be32 *intserv; 314 int i, nthreads; 315 int len; 316 int found = -1; 317 int found_thread = 0; 318 unsigned int nr_cpus_aligned; 319 320 /* We are scanning "cpu" nodes only */ 321 if (type == NULL || strcmp(type, "cpu") != 0) 322 return 0; 323 324 /* Get physical cpuid */ 325 intserv = of_get_flat_dt_prop(node, "ibm,ppc-interrupt-server#s", &len); 326 if (!intserv) 327 intserv = of_get_flat_dt_prop(node, "reg", &len); 328 329 nthreads = len / sizeof(int); 330 331 /* 332 * Now see if any of these threads match our boot cpu. 333 * NOTE: This must match the parsing done in smp_setup_cpu_maps. 334 */ 335 for (i = 0; i < nthreads; i++) { 336 /* 337 * version 2 of the kexec param format adds the phys cpuid of 338 * booted proc. 339 */ 340 if (fdt_version(initial_boot_params) >= 2) { 341 if (be32_to_cpu(intserv[i]) == 342 fdt_boot_cpuid_phys(initial_boot_params)) { 343 found = boot_cpu_count; 344 found_thread = i; 345 } 346 } else { 347 /* 348 * Check if it's the boot-cpu, set it's hw index now, 349 * unfortunately this format did not support booting 350 * off secondary threads. 351 */ 352 if (of_get_flat_dt_prop(node, 353 "linux,boot-cpu", NULL) != NULL) 354 found = boot_cpu_count; 355 } 356 #ifdef CONFIG_SMP 357 /* logical cpu id is always 0 on UP kernels */ 358 boot_cpu_count++; 359 #endif 360 } 361 362 /* Not the boot CPU */ 363 if (found < 0) 364 return 0; 365 /* to work around p8 subcore mode */ 366 nr_cpus_aligned = _ALIGN_UP(nr_cpu_ids, nthreads); 367 if (nr_cpus_aligned != nr_cpu_ids) { 368 pr_info("nr_cpus is forced to align up on threads_per_core," 369 "from %u to %u\n", nr_cpu_ids, nr_cpus_aligned); > 370 nr_cpu_ids = nr_cpus_aligned; 371 } 372 DBG("boot cpu: logical %d physical %d\n", found, 373 be32_to_cpu(intserv[found_thread])); 374 boot_cpuid = found; 375 376 /* 377 * PAPR defines "logical" PVR values for cpus that 378 * meet various levels of the architecture: 379 * 0x0f01 Architecture version 2.04 380 * 0x0f02 Architecture version 2.05 381 * If the cpu-version property in the cpu node contains 382 * such a value, we call identify_cpu again with the 383 * logical PVR value in order to use the cpu feature 384 * bits appropriate for the architecture level. 385 * 386 * A POWER6 partition in "POWER6 architected" mode
[PATCH] powerpc/cpu: nr_cpu_ids should be aligned on threads_per_core
With p8 subcore mode, if nr_cpus is not aligned on threads_per_core, then we will hit the following panic: [0.310006] Unpacking initramfs... [1.105015] Freeing initrd memory: 35968K [1.105466] rtas_flash: no firmware flash support [1.105688] Unable to handle kernel paging request for data at address 0x1035 [1.105768] Faulting instruction address: 0xc0099bb8 [1.105837] Oops: Kernel access of bad area, sig: 11 [#1] [1.105890] LE SMP NR_CPUS=2048 NUMA PowerNV [1.105947] Modules linked in: [1.105992] CPU: 8 PID: 11 Comm: migration/8 Not tainted 4.17.0-rc4+ #26 [1.106059] NIP: c0099bb8 LR: c00ac554 CTR: c0099b70 [1.106139] REGS: c00ff649f860 TRAP: 0380 Not tainted (4.17.0-rc4+) [1.106205] MSR: 90001033 CR: 28004022 XER: [1.106293] CFAR: c00ac550 SOFTE: 3 [1.106293] GPR00: c00ac554 c00ff649fae0 c12e6d00 0060 [1.106293] GPR04: 0001 c00ac914 90001033 c03fff80 [1.106293] GPR08: 0001 90001003 [1.106293] GPR12: c0099b70 c170 c0141c28 c00ffa201180 [1.106293] GPR16: [1.106293] GPR20: 0001 c00fffd13840 [1.106293] GPR24: c00fffd13838 0001 0010 [1.106293] GPR28: 0008 0001 c1370548 000c [1.106984] NIP [c0099bb8] icp_native_cause_ipi+0x48/0x70 [1.107053] LR [c00ac554] pnv_cause_ipi+0x54/0x80 [1.107106] Call Trace: [1.107137] [c00ff649fae0] [c00478b4] doorbell_try_core_ipi+0x74/0x90 (unreliable) [1.107232] [c00ff649fb10] [c00ac554] pnv_cause_ipi+0x54/0x80 [1.107301] [c00ff649fb40] [c004ccfc] smp_muxed_ipi_message_pass+0x6c/0x90 [1.107384] [c00ff649fb80] [c00acbe8] cpu_do_split+0x2b8/0x2f0 [1.107453] [c00ff649fbf0] [c00acdb8] cpu_update_split_mode+0x68/0x204 [1.107536] [c00ff649fc40] [c0204ab8] multi_cpu_stop+0x1c8/0x200 [1.107617] [c00ff649fca0] [c0204eb4] cpu_stopper_thread+0x114/0x1f0 [1.107699] [c00ff649fd60] [c0148650] smpboot_thread_fn+0x290/0x2a0 [1.107780] [c00ff649fdc0] [c0141d84] kthread+0x164/0x1b0 [1.107850] [c00ff649fe30] [c000b628] ret_from_kernel_thread+0x5c/0xb4 [1.107929] Instruction dump: [1.107971] 7c7f1b78 6000 6000 3d420004 394ab070 7be31f24 3921 e90a [1.108058] 3d42001b 394a2650 7d08182a 7d4a182a <99281035> 3904 7c0004ac 990a000c [1.108150] ---[ end trace 83dad19c162dc306 ]--- [1.110137] [2.110227] Kernel panic - not syncing: Fatal exception This is caused by that during subcore_init, it IPIs all cpu inside the core, but due to the limitation imposed by nr_cpus, there are no mem allocated for percpu cpu_sibling_map. To avoid a complicated fix, this patch just forces the nr_cpu_ids to be aligned on thread_per_code at the cost of a bit more memory. Signed-off-by: Pingfan Liu --- arch/powerpc/kernel/prom.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c index 9dbed48..358adb4 100644 --- a/arch/powerpc/kernel/prom.c +++ b/arch/powerpc/kernel/prom.c @@ -315,6 +315,7 @@ static int __init early_init_dt_scan_cpus(unsigned long node, int len; int found = -1; int found_thread = 0; + unsigned int nr_cpus_aligned; /* We are scanning "cpu" nodes only */ if (type == NULL || strcmp(type, "cpu") != 0) @@ -361,7 +362,13 @@ static int __init early_init_dt_scan_cpus(unsigned long node, /* Not the boot CPU */ if (found < 0) return 0; - + /* to work around p8 subcore mode */ + nr_cpus_aligned = _ALIGN_UP(nr_cpu_ids, nthreads); + if (nr_cpus_aligned != nr_cpu_ids) { + pr_info("nr_cpus is forced to align up on threads_per_core," + "from %u to %u\n", nr_cpu_ids, nr_cpus_aligned); + nr_cpu_ids = nr_cpus_aligned; + } DBG("boot cpu: logical %d physical %d\n", found, be32_to_cpu(intserv[found_thread])); boot_cpuid = found; -- 2.7.4