Re: No Linux logs when doing `ppc64_cpu --smt=off/8`
On Thu, 17 Feb 2022 at 01:07, Michael Ellerman wrote: > > Michal Suchánek writes: > > On Mon, Feb 14, 2022 at 01:33:24PM +0100, Paul Menzel wrote: > >> Yes, simple `nproc` suffice, but I was more thinking about, that the Linux > >> log is often used for debugging and the changes of amount of processing > >> units might be good to have. `ppc64_cpu --smt=off` or `=8` seems to block > >> for quite some time, and each thread/processing unit seems to powered > >> down/on sequentially, so it takes quite some time and it blocks. So 140 > >> messages would indeed be quite noise. No idea how `ppc64_cpu` works, and if > >> it could log a message at the beginning and end. > > > > Yes, it enables/disables threads one by one. AFAICT the kernel cannot know > > that > > ppc64_cpu will enable/disable more threads later, it can either log each > > or none. Rate limiting would not show the whole picture so it's not > > great solution either. > > Right, ppc64_cpu just uses the sysfs online files, so it's doing them one at a > time. The kernel has no knowledge that ppc64_cpu is turning all > secondaries on/off so there's no easy way for the kernel to do a summary > message. > > An easy solution would be for ppc64_cpu to log something via syslog(3). Another option would be to implement this sysfs knob for powerpc: /sys/devices/system/cpu/smt/control https://github.com/torvalds/linux/commit/05736e4ac13c08a4a9b1ef2de26dd31a32cbee57 It currently returns "notimplemented" on power, but it's designed (for x86) to turn SMT on and off. With this implemented we could offline all CPUs at once before returning to userspace. It "just" needs someone with some time to work on it. ...which I spent the afternoon attempting to do: https://lore.kernel.org/linuxppc-dev/20220217070419.351683-1-j...@jms.id.au/ It does what it says on the tin, but there's a few rough edges that need working out. It won't solve the problem you had of wanting a message in the kernel logs (I notice that the x86 cpu hotplug code does print something; I wonder how that goes on a large box). If you or anyone else wants to run with the idea then free free to use my patch as a starting point. Cheers, Joel
Re: No Linux logs when doing `ppc64_cpu --smt=off/8`
Michal Suchánek writes: > On Mon, Feb 14, 2022 at 01:33:24PM +0100, Paul Menzel wrote: >> Am 14.02.22 um 10:43 schrieb Michal Suchánek: >> > On Mon, Feb 14, 2022 at 07:08:07AM +0100, Paul Menzel wrote: >> > > On the POWER8 server IBM S822LC running `ppc64_cpu --smt=off` or >> > > `ppc64_cpu >> > > --smt=8`, Linux 5.17-rc4 does not log anything. I would have expected a >> > > message about the change in number of processing units. >> > >> > IIRC it was considered too noisy for systems with many CPUs and the >> > message was dropped. You can always check the resulting state with >> > ppc64_cpu or examining sysfs. One of the messages was removed because it was potentially buggy: ed8029d7b472 ("powerpc/pseries: Stop calling printk in rtas_stop_self()") We may have removed some other messages, but my grepping skills can't find anything relevant at the moment. But in general yes, it used to be far too verbose on large systems. >> Yes, simple `nproc` suffice, but I was more thinking about, that the Linux >> log is often used for debugging and the changes of amount of processing >> units might be good to have. `ppc64_cpu --smt=off` or `=8` seems to block >> for quite some time, and each thread/processing unit seems to powered >> down/on sequentially, so it takes quite some time and it blocks. So 140 >> messages would indeed be quite noise. No idea how `ppc64_cpu` works, and if >> it could log a message at the beginning and end. > > Yes, it enables/disables threads one by one. AFAICT the kernel cannot know > that > ppc64_cpu will enable/disable more threads later, it can either log each > or none. Rate limiting would not show the whole picture so it's not > great solution either. Right, ppc64_cpu just uses the sysfs online files, so it's doing them one at a time. The kernel has no knowledge that ppc64_cpu is turning all secondaries on/off so there's no easy way for the kernel to do a summary message. An easy solution would be for ppc64_cpu to log something via syslog(3). cheers
Re: No Linux logs when doing `ppc64_cpu --smt=off/8`
On Mon, Feb 14, 2022 at 01:33:24PM +0100, Paul Menzel wrote: > Dear Michal, > > > Thank you for your reply. > > Am 14.02.22 um 10:43 schrieb Michal Suchánek: > > > On Mon, Feb 14, 2022 at 07:08:07AM +0100, Paul Menzel wrote: > > > Dear PPC folks, > > > > > > > > > On the POWER8 server IBM S822LC running `ppc64_cpu --smt=off` or > > > `ppc64_cpu > > > --smt=8`, Linux 5.17-rc4 does not log anything. I would have expected a > > > message about the change in number of processing units. > > > > IIRC it was considered too noisy for systems with many CPUs and the > > message was dropped. You can always check the resulting state with > > ppc64_cpu or examining sysfs. > > Yes, simple `nproc` suffice, but I was more thinking about, that the Linux > log is often used for debugging and the changes of amount of processing > units might be good to have. `ppc64_cpu --smt=off` or `=8` seems to block > for quite some time, and each thread/processing unit seems to powered > down/on sequentially, so it takes quite some time and it blocks. So 140 > messages would indeed be quite noise. No idea how `ppc64_cpu` works, and if > it could log a message at the beginning and end. Yes, it enables/disables threads one by one. AFAICT the kernel cannot know that ppc64_cpu will enable/disable more threads later, it can either log each or none. Rate limiting would not show the whole picture so it's not great solution either. Thanks Michal
Re: No Linux logs when doing `ppc64_cpu --smt=off/8`
Dear Michal, Thank you for your reply. Am 14.02.22 um 10:43 schrieb Michal Suchánek: On Mon, Feb 14, 2022 at 07:08:07AM +0100, Paul Menzel wrote: Dear PPC folks, On the POWER8 server IBM S822LC running `ppc64_cpu --smt=off` or `ppc64_cpu --smt=8`, Linux 5.17-rc4 does not log anything. I would have expected a message about the change in number of processing units. IIRC it was considered too noisy for systems with many CPUs and the message was dropped. You can always check the resulting state with ppc64_cpu or examining sysfs. Yes, simple `nproc` suffice, but I was more thinking about, that the Linux log is often used for debugging and the changes of amount of processing units might be good to have. `ppc64_cpu --smt=off` or `=8` seems to block for quite some time, and each thread/processing unit seems to powered down/on sequentially, so it takes quite some time and it blocks. So 140 messages would indeed be quite noise. No idea how `ppc64_cpu` works, and if it could log a message at the beginning and end. Kind regards, Paul
Re: No Linux logs when doing `ppc64_cpu --smt=off/8`
Hello, On Mon, Feb 14, 2022 at 07:08:07AM +0100, Paul Menzel wrote: > Dear PPC folks, > > > On the POWER8 server IBM S822LC running `ppc64_cpu --smt=off` or `ppc64_cpu > --smt=8`, Linux 5.17-rc4 does not log anything. I would have expected a > message about the change in number of processing units. IIRC it was considered too noisy for systems with many CPUs and the message was dropped. You can always check the resulting state with ppc64_cpu or examining sysfs. Thanks Michal
No Linux logs when doing `ppc64_cpu --smt=off/8`
Dear PPC folks, On the POWER8 server IBM S822LC running `ppc64_cpu --smt=off` or `ppc64_cpu --smt=8`, Linux 5.17-rc4 does not log anything. I would have expected a message about the change in number of processing units. Kind regards, Paul