bug#42044: nproc says "1" on mobile processors
On Fri, Jun 26, 2020 at 07:26:16PM +0100, Pádraig Brady wrote: > On 25/06/2020 18:37, Adam Borowski wrote: > > Hi! > > I'm afraid that `nproc` shows only the number of _currently_ online CPUs, > > which on mobile processors tends to be 1 when starting a job. As there's > > a need to conserve power, holding cores online when they have nothing to > > do would be a waste, thus they constantly get onlined and offlined. > > Ie, could you please make nproc include all available CPUs rather than > > only online ones? > > Does `nproc --all` suffice for your use case? It works on the mobile box, but doesn't obey affinity mask anymore: [~]$ numactl -N 0 nproc --all 64 [~]$ numactl -N 0 nproc 16 I'd wish for fitting all uses, big and small. Ie, the answer "on this machine, with currently imposed limits, what's the fastest level of parallelism?". Meow. -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good. ⢿⡄⠘⠷⠚⠋⠀ -- on #linux-sunxi ⠈⠳⣄
bug#42044: nproc says "1" on mobile processors
On 25/06/2020 18:37, Adam Borowski wrote: Hi! I'm afraid that `nproc` shows only the number of _currently_ online CPUs, which on mobile processors tends to be 1 when starting a job. As there's a need to conserve power, holding cores online when they have nothing to do would be a waste, thus they constantly get onlined and offlined. For example, on a 10-core box where the processor consists of three clusters, 4+4+2 cores, topline graph shows: (▁▁▁ oo) (▁ ▁ oo) (▃o) (oo) (▂o) (▁▁▁ oo) (▁ ▁ oo) (▁▁ oo) (▁ ▁oo) (▁▁ oo) (▁ oo) (1 line = 1 second, "o" means offline core, otherwise it's utilization level.) Because of noisy GUI (that eg. draws this very graph), the machine keeps flipping between onlining just CPU 0, or a cluster of CPU4..7. Thus, `nproc` will randomly says either "1" or "4", while the user expects to run her compile with all 10 cores. I found out that numa_num_task_cpus() from libnuma gets the right answer, handling both affinity mask and CPUs present-but-currently-offline. But alas, it doesn't provide a command-line tool, and thousands of scripts already use nproc, thus switching a tool would be a waste of effort. Ie, could you please make nproc include all available CPUs rather than only online ones? Does `nproc --all` suffice for your use case?
bug#42044: nproc says "1" on mobile processors
Hi! I'm afraid that `nproc` shows only the number of _currently_ online CPUs, which on mobile processors tends to be 1 when starting a job. As there's a need to conserve power, holding cores online when they have nothing to do would be a waste, thus they constantly get onlined and offlined. For example, on a 10-core box where the processor consists of three clusters, 4+4+2 cores, topline graph shows: (▁▁▁ oo) (▁ ▁ oo) (▃o) (oo) (▂o) (▁▁▁ oo) (▁ ▁ oo) (▁▁ oo) (▁ ▁oo) (▁▁ oo) (▁ oo) (1 line = 1 second, "o" means offline core, otherwise it's utilization level.) Because of noisy GUI (that eg. draws this very graph), the machine keeps flipping between onlining just CPU 0, or a cluster of CPU4..7. Thus, `nproc` will randomly says either "1" or "4", while the user expects to run her compile with all 10 cores. I found out that numa_num_task_cpus() from libnuma gets the right answer, handling both affinity mask and CPUs present-but-currently-offline. But alas, it doesn't provide a command-line tool, and thousands of scripts already use nproc, thus switching a tool would be a waste of effort. Ie, could you please make nproc include all available CPUs rather than only online ones? Meow! -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good. ⢿⡄⠘⠷⠚⠋⠀ -- on #linux-sunxi ⠈⠳⣄
