[Kernel-packages] [Bug 2051733] Re: Specifying nohz_full breaks CPU frequency reporting
I'm not familiar with Linux kernel internals, or how scaling_cur_freq is implemented internally, but that doesn't look like a valid logic to me. If it takes an IPI (or two, as the commit message suggests) to read the core frequency, then make those IPIs. It doesn't matter how expensive it is - if the user wants to read the current frequency then he is willing to pay for that information. This likely won't be a frequent operation anyway. Providing an interface to read this information and then feeding bogus data through it is not acceptable, IMO. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-signed-lowlatency-hwe-6.5 in Ubuntu. https://bugs.launchpad.net/bugs/2051733 Title: Specifying nohz_full breaks CPU frequency reporting Status in linux-signed-lowlatency-hwe-6.5 package in Ubuntu: Confirmed Bug description: With the lowlatency kernel, if I specify "nohz_full=1-15" boot parameter then CPU frequency reporting doesn't work for the logical cores 1-15. That is, only logical core 0 shows varying CPU frequency in its /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq file, all other cores constantly show 80 in their scaling_cur_freq files (which is the lowest supported frequency) regardless of the CPU load. Steps to reproduce: 1. Add "nohz_full=1-15" (specify the core numbers to include all logical cores except 0) to kernel boot options in /etc/default/grub. 2. Run `sudo update-grub` and reboot. 3. Upon booting, run a multithreaded workload. For example, run `openssl speed -multi $(nproc --all)`. 4. In another console, monitor CPU frequencies by running `watch cat /sys/devices/system/cpu/cpu[0-9]*/cpufreq/scaling_cur_freq`. Actual results: All cores specified in "nohz_full" parameter always report their lowest frequency. Despite that, the actual performance seems to be as if frequency scaling actually works (i.e. according to benchmarks, the performance seems to be similar with and without the "nohz_full" parameter). Expected results: All cores must report their actual frequency depending on the load. ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: linux-image-6.5.0-15-lowlatency 6.5.0-15.15.1.1~22.04.1 ProcVersionSignature: Ubuntu 6.5.0-15.15.1.1~22.04.1-lowlatency 6.5.3 Uname: Linux 6.5.0-15-lowlatency x86_64 NonfreeKernelModules: nvidia_modeset nvidia ApportVersion: 2.20.11-0ubuntu82.5 Architecture: amd64 CasperMD5CheckResult: unknown CurrentDesktop: KDE Date: Tue Jan 30 23:39:51 2024 InstallationDate: Installed on 2015-05-01 (3196 days ago) InstallationMedia: Kubuntu 15.04 "Vivid Vervet" - Release amd64 (20150422) SourcePackage: linux-signed-lowlatency-hwe-6.5 UpgradeStatus: Upgraded to jammy on 2022-05-14 (626 days ago) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-signed-lowlatency-hwe-6.5/+bug/2051733/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2051733] Re: Specifying nohz_full breaks CPU frequency reporting
The way it is currently done, I don't think valid CPU frequency listing via "scaling_cur_freq", or /proc/cpuinfo, is expected to work. Why not? Because the required code is never executed, on purpose. Here is an excerpt from a commit (see the bit about NOHZ full) commit f3eca381bd49d708073ba1a9af4fa6ea5d5810a6 Author: Thomas Gleixner Date: Fri Apr 15 21:20:04 2022 +0200 x86/aperfmperf: Replace arch_freq_get_on_cpu() Reading the current CPU frequency from /sys//scaling_cur_freq involves in the worst case two IPIs due to the ad hoc sampling. The frequency invariance infrastructure provides the APERF/MPERF samples already. Utilize them and consolidate this with the /proc/cpuinfo readout. The sample is considered valid for 20ms. So for idle or isolated NOHZ full CPUs the function returns 0, which is matching the previous behaviour. There was couple of later commits and now it prints out the minimum CPU frequency when it thinks the number are stale. With NOHz full it always thinks the numbers are stale. The intel_cpufreq driver seems to display CPU frequencies okay, but only the pstate that was requested, not the actual frequency granted. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-signed-lowlatency-hwe-6.5 in Ubuntu. https://bugs.launchpad.net/bugs/2051733 Title: Specifying nohz_full breaks CPU frequency reporting Status in linux-signed-lowlatency-hwe-6.5 package in Ubuntu: Confirmed Bug description: With the lowlatency kernel, if I specify "nohz_full=1-15" boot parameter then CPU frequency reporting doesn't work for the logical cores 1-15. That is, only logical core 0 shows varying CPU frequency in its /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq file, all other cores constantly show 80 in their scaling_cur_freq files (which is the lowest supported frequency) regardless of the CPU load. Steps to reproduce: 1. Add "nohz_full=1-15" (specify the core numbers to include all logical cores except 0) to kernel boot options in /etc/default/grub. 2. Run `sudo update-grub` and reboot. 3. Upon booting, run a multithreaded workload. For example, run `openssl speed -multi $(nproc --all)`. 4. In another console, monitor CPU frequencies by running `watch cat /sys/devices/system/cpu/cpu[0-9]*/cpufreq/scaling_cur_freq`. Actual results: All cores specified in "nohz_full" parameter always report their lowest frequency. Despite that, the actual performance seems to be as if frequency scaling actually works (i.e. according to benchmarks, the performance seems to be similar with and without the "nohz_full" parameter). Expected results: All cores must report their actual frequency depending on the load. ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: linux-image-6.5.0-15-lowlatency 6.5.0-15.15.1.1~22.04.1 ProcVersionSignature: Ubuntu 6.5.0-15.15.1.1~22.04.1-lowlatency 6.5.3 Uname: Linux 6.5.0-15-lowlatency x86_64 NonfreeKernelModules: nvidia_modeset nvidia ApportVersion: 2.20.11-0ubuntu82.5 Architecture: amd64 CasperMD5CheckResult: unknown CurrentDesktop: KDE Date: Tue Jan 30 23:39:51 2024 InstallationDate: Installed on 2015-05-01 (3196 days ago) InstallationMedia: Kubuntu 15.04 "Vivid Vervet" - Release amd64 (20150422) SourcePackage: linux-signed-lowlatency-hwe-6.5 UpgradeStatus: Upgraded to jammy on 2022-05-14 (626 days ago) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-signed-lowlatency-hwe-6.5/+bug/2051733/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2051733] Re: Specifying nohz_full breaks CPU frequency reporting
CPU frequency scaling driver: intel_pstate CPU frequency scaling governor: powersave HWP: disabled. Purpose to verify that the driver is working correctly, regardless of CPU frequencies reported. A single threaded load was applied to CPU 5 at 347 hertz sleep/work frequency. The load was increased then deceased. The intel_pstate_tracer.py utility was run during the test capturing the attached. All pstates were used and appropriate per the load. ** Attachment added: "all_cpu_frequencies.png" https://bugs.launchpad.net/ubuntu/+source/linux-signed-lowlatency-hwe-6.5/+bug/2051733/+attachment/5744261/+files/all_cpu_frequencies.png -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-signed-lowlatency-hwe-6.5 in Ubuntu. https://bugs.launchpad.net/bugs/2051733 Title: Specifying nohz_full breaks CPU frequency reporting Status in linux-signed-lowlatency-hwe-6.5 package in Ubuntu: Confirmed Bug description: With the lowlatency kernel, if I specify "nohz_full=1-15" boot parameter then CPU frequency reporting doesn't work for the logical cores 1-15. That is, only logical core 0 shows varying CPU frequency in its /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq file, all other cores constantly show 80 in their scaling_cur_freq files (which is the lowest supported frequency) regardless of the CPU load. Steps to reproduce: 1. Add "nohz_full=1-15" (specify the core numbers to include all logical cores except 0) to kernel boot options in /etc/default/grub. 2. Run `sudo update-grub` and reboot. 3. Upon booting, run a multithreaded workload. For example, run `openssl speed -multi $(nproc --all)`. 4. In another console, monitor CPU frequencies by running `watch cat /sys/devices/system/cpu/cpu[0-9]*/cpufreq/scaling_cur_freq`. Actual results: All cores specified in "nohz_full" parameter always report their lowest frequency. Despite that, the actual performance seems to be as if frequency scaling actually works (i.e. according to benchmarks, the performance seems to be similar with and without the "nohz_full" parameter). Expected results: All cores must report their actual frequency depending on the load. ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: linux-image-6.5.0-15-lowlatency 6.5.0-15.15.1.1~22.04.1 ProcVersionSignature: Ubuntu 6.5.0-15.15.1.1~22.04.1-lowlatency 6.5.3 Uname: Linux 6.5.0-15-lowlatency x86_64 NonfreeKernelModules: nvidia_modeset nvidia ApportVersion: 2.20.11-0ubuntu82.5 Architecture: amd64 CasperMD5CheckResult: unknown CurrentDesktop: KDE Date: Tue Jan 30 23:39:51 2024 InstallationDate: Installed on 2015-05-01 (3196 days ago) InstallationMedia: Kubuntu 15.04 "Vivid Vervet" - Release amd64 (20150422) SourcePackage: linux-signed-lowlatency-hwe-6.5 UpgradeStatus: Upgraded to jammy on 2022-05-14 (626 days ago) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-signed-lowlatency-hwe-6.5/+bug/2051733/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2051733] Re: Specifying nohz_full breaks CPU frequency reporting
There is a high probability that the root issue here is related to some work done in August September. There was already an outstanding issue with intel_cpufreq driver / schedutil governor, hwp enabled. References: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d51847acb018d83186e4af67bc93f9a00a8644f7 https://bugzilla.kernel.org/show_bug.cgi?id=217597 ** Bug watch added: Linux Kernel Bug Tracker #217597 https://bugzilla.kernel.org/show_bug.cgi?id=217597 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-signed-lowlatency-hwe-6.5 in Ubuntu. https://bugs.launchpad.net/bugs/2051733 Title: Specifying nohz_full breaks CPU frequency reporting Status in linux-signed-lowlatency-hwe-6.5 package in Ubuntu: Confirmed Bug description: With the lowlatency kernel, if I specify "nohz_full=1-15" boot parameter then CPU frequency reporting doesn't work for the logical cores 1-15. That is, only logical core 0 shows varying CPU frequency in its /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq file, all other cores constantly show 80 in their scaling_cur_freq files (which is the lowest supported frequency) regardless of the CPU load. Steps to reproduce: 1. Add "nohz_full=1-15" (specify the core numbers to include all logical cores except 0) to kernel boot options in /etc/default/grub. 2. Run `sudo update-grub` and reboot. 3. Upon booting, run a multithreaded workload. For example, run `openssl speed -multi $(nproc --all)`. 4. In another console, monitor CPU frequencies by running `watch cat /sys/devices/system/cpu/cpu[0-9]*/cpufreq/scaling_cur_freq`. Actual results: All cores specified in "nohz_full" parameter always report their lowest frequency. Despite that, the actual performance seems to be as if frequency scaling actually works (i.e. according to benchmarks, the performance seems to be similar with and without the "nohz_full" parameter). Expected results: All cores must report their actual frequency depending on the load. ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: linux-image-6.5.0-15-lowlatency 6.5.0-15.15.1.1~22.04.1 ProcVersionSignature: Ubuntu 6.5.0-15.15.1.1~22.04.1-lowlatency 6.5.3 Uname: Linux 6.5.0-15-lowlatency x86_64 NonfreeKernelModules: nvidia_modeset nvidia ApportVersion: 2.20.11-0ubuntu82.5 Architecture: amd64 CasperMD5CheckResult: unknown CurrentDesktop: KDE Date: Tue Jan 30 23:39:51 2024 InstallationDate: Installed on 2015-05-01 (3196 days ago) InstallationMedia: Kubuntu 15.04 "Vivid Vervet" - Release amd64 (20150422) SourcePackage: linux-signed-lowlatency-hwe-6.5 UpgradeStatus: Upgraded to jammy on 2022-05-14 (626 days ago) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-signed-lowlatency-hwe-6.5/+bug/2051733/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
