Here are some optimizations and fixes to make CPU online/offline
faster and hence result in faster bootup.
Its based on top of my v5 coregroup support patchset.
https://lore.kernel.org/linuxppc-dev/20200810071834.92514-1-sri...@linux.vnet.ibm.com/t/#u
Anton reported that his 4096 cpu (1024 cores in a socket) was taking too
long to boot. He also analyzed that most of the time was being spent on
updating cpu_core_mask.
The first two patches should solve Anton's immediate problem.
On the unofficial patches, Anton reported that the boot time came from 30
mins to 6 seconds. (Basically a high core count in a single socket
configuration). Satheesh also reported similar numbers.
The rest are cleanups/optimizations.
Since cpu_core_mask is an exported symbol for a long duration, lets retain
as a snapshot of cpumask_of_node.
$ lscpu
Architecture:ppc64le
Byte Order: Little Endian
CPU(s): 1024
On-line CPU(s) list: 0-1023
Thread(s) per core: 8
Core(s) per socket: 8
Socket(s): 16
NUMA node(s):16
Model: 2.0 (pvr 004d 0200)
Model name: POWER8 (architected), altivec supported
Hypervisor vendor: pHyp
Virtualization type: para
L1d cache: 64K
L1i cache: 32K
L2 cache:512K
L3 cache:8192K
NUMA node0 CPU(s): 0-63
NUMA node1 CPU(s): 64-127
NUMA node2 CPU(s): 128-191
NUMA node3 CPU(s): 192-255
NUMA node4 CPU(s): 256-319
NUMA node5 CPU(s): 320-383
NUMA node6 CPU(s): 384-447
NUMA node7 CPU(s): 448-511
NUMA node8 CPU(s): 512-575
NUMA node9 CPU(s): 576-639
NUMA node10 CPU(s): 640-703
NUMA node11 CPU(s): 704-767
NUMA node12 CPU(s): 768-831
NUMA node13 CPU(s): 832-895
NUMA node14 CPU(s): 896-959
NUMA node15 CPU(s): 960-1023
$ dmesg -k | grep -i -e Bringing -e Brought -e sysrq -e bug
With powerp/next
[0.00] printk: debug: ignoring loglevel setting.
[0.354971] smp: Bringing up secondary CPUs ...
[ 233.354676] smp: Brought up 16 nodes, 1024 CPUs
[ 330.023073] sysrq: Changing Loglevel
[ 330.023101] sysrq: Loglevel set to 9
With +patchset
[0.00] printk: debug: ignoring loglevel setting.
[0.351703] smp: Bringing up secondary CPUs ...
[4.059859] smp: Brought up 16 nodes, 1024 CPUs
[ 98.309015] sysrq: Changing Loglevel
[ 98.309044] sysrq: Loglevel set to 9
Observations:
CPU bringup time reduced to 4 seconds from 233 seconds on this 1024 CPU
system. This resulted in System boot up time reducing to 98 seconds from
330 seconds. The actual improvement would depend on your system topology.
Topology verification post patchset on a 2 node Power9 PowerVM LPAR
powerpc/next+patchset
-
$ lscpu
Architecture:ppc64le
Architecture:ppc64le
Byte Order: Little Endian Byte Order:
Little Endian
CPU(s): 128CPU(s):
128
On-line CPU(s) list: 0-127 On-line
CPU(s) list: 0-127
Thread(s) per core: 8 Thread(s)
per core: 8
Core(s) per socket: 8 Core(s) per
socket: 8
Socket(s): 2 Socket(s):
2
NUMA node(s):2 NUMA
node(s):2
Model: 2.2 (pvr 004e 0202)Model:
2.2 (pvr 004e 0202)
Model name: POWER9 (architected), altivec supportedModel name:
POWER9 (architected), altivec supported
Hypervisor vendor: pHyp Hypervisor
vendor: pHyp
Virtualization type: para
Virtualization type: para
L1d cache: 32KL1d cache:
32K
L1i cache: 32KL1i cache:
32K
L2 cache:512K L2 cache:
512K
L3 cache:10240K L3 cache:
10240K
NUMA node0 CPU(s): 0-63 NUMA node0
CPU(s): 0-63
NUMA node1 CPU(s): 64-127 NUMA node1
CPU(s): 64-127
$ tail -f /proc/cpuinfo
processor : 127 processor
: 127
cpu : POWER9 (architected), altivec supported cpu
: POWER9 (architected), altivec supported
clock : 3000.00MHzclock
: 3000.00MHz
revision: 2.2 (pvr 004e 0202) revision
:
