[Kernel-packages] [Bug 1713576] Comment bridged from LTC Bugzilla
--- Comment From jhop...@us.ibm.com 2017-09-12 10:08 EDT--- (In reply to comment #14) > Thanks for the update. Can this issue now be closed? Should we wait until it is in an official build? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1713576 Title: More migrations with constant load Status in The Ubuntu-power-systems project: Triaged Status in linux package in Ubuntu: Triaged Bug description: == Comment: #0 - PUVICHAKRAVARTHY RAMACHANDRAN - 2017-08-06 13:44:45 == ---Problem Description--- Significantly higher number of task migrations when the load is fixed but not balanced across cores. ---uname output--- Linux isvbos3 4.10.0-29-generic #33~16.04.1-Ubuntu SMP Tue Jul 25 18:17:06 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux ---Additional Hardware Info--- Power9 dd2.0 Machine Type = Power9 ---Steps to Reproduce--- Benchmark : Multithreaded - cpu intensive. The system had 2 socket/ 32 cores/ SMT4 mode. When 64 threads was run - the migrations were less over 10s interval. when 80 threads were run - the migrations were very high. Ideally, it should have been very minimal, as the over all load was constant == Comment: #3 - SRIKAR DRONAMRAJU - 2017-08-11 06:56:47 == As suspected (commit : 05b40e0577 : "sched/fair: Prefer sibiling only if local group is under-utilized") https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=05b40e0577 should fix the problem Ran ' perf stat -a -r 5 -e sched:sched_migrate_task /home/srikar/work/ebizzy-0.3/ebizzy -t 35 -S 100' to detect the problem and verify the fix Here is perf stat without fix. Performance counter stats for 'system wide' (5 runs): 7,758 sched:sched_migrate_task ( +- 1.28% ) 100.015658079 seconds time elapsed ( +- 0.00% ) perf stat with fix. Performance counter stats for 'system wide' (5 runs): 415 sched:sched_migrate_task ( +- 11.74% ) 100.016021787 seconds time elapsed ( +- 0.00% ) git describe on upstream kernel says v4.11-rc2 # git describe 05b40e0577 v4.11-rc2-227-g05b40e0 == Comment: #4 - SRIKAR DRONAMRAJU - 2017-08-11 07:05:37 == Attaching the patch that needs to be applied to fix this bug. Verified that patch fixes the problem. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1713576/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1713576] Comment bridged from LTC Bugzilla
--- Comment From jhop...@us.ibm.com 2017-09-11 15:49 EDT--- Thanks, I was able to make perf work by uninstalling the "linux-tools-common" test .deb and installing the stock version. This test build resolves the problem by significantly reducing sched migrations in the under utilized case which improves performance by 30-40% for this ebizzy testcase: stock kernel (4.10.0-33-generic) : 6,010 sched:sched_migrate_task test kernel (4.10.0-33-generic #37~lp1713576) : 676 sched:sched_migrate_task Testcase was: perf stat -a -r 5 -e sched:sched_migrate_task ./ebizzy -t 35 -S 100 where ebizzy was using 35 threads on a system w/ 128 logical cpus total -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1713576 Title: More migrations with constant load Status in The Ubuntu-power-systems project: Triaged Status in linux package in Ubuntu: Triaged Bug description: == Comment: #0 - PUVICHAKRAVARTHY RAMACHANDRAN - 2017-08-06 13:44:45 == ---Problem Description--- Significantly higher number of task migrations when the load is fixed but not balanced across cores. ---uname output--- Linux isvbos3 4.10.0-29-generic #33~16.04.1-Ubuntu SMP Tue Jul 25 18:17:06 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux ---Additional Hardware Info--- Power9 dd2.0 Machine Type = Power9 ---Steps to Reproduce--- Benchmark : Multithreaded - cpu intensive. The system had 2 socket/ 32 cores/ SMT4 mode. When 64 threads was run - the migrations were less over 10s interval. when 80 threads were run - the migrations were very high. Ideally, it should have been very minimal, as the over all load was constant == Comment: #3 - SRIKAR DRONAMRAJU - 2017-08-11 06:56:47 == As suspected (commit : 05b40e0577 : "sched/fair: Prefer sibiling only if local group is under-utilized") https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=05b40e0577 should fix the problem Ran ' perf stat -a -r 5 -e sched:sched_migrate_task /home/srikar/work/ebizzy-0.3/ebizzy -t 35 -S 100' to detect the problem and verify the fix Here is perf stat without fix. Performance counter stats for 'system wide' (5 runs): 7,758 sched:sched_migrate_task ( +- 1.28% ) 100.015658079 seconds time elapsed ( +- 0.00% ) perf stat with fix. Performance counter stats for 'system wide' (5 runs): 415 sched:sched_migrate_task ( +- 11.74% ) 100.016021787 seconds time elapsed ( +- 0.00% ) git describe on upstream kernel says v4.11-rc2 # git describe 05b40e0577 v4.11-rc2-227-g05b40e0 == Comment: #4 - SRIKAR DRONAMRAJU - 2017-08-11 07:05:37 == Attaching the patch that needs to be applied to fix this bug. Verified that patch fixes the problem. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1713576/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1713576] Comment bridged from LTC Bugzilla
--- Comment From jhop...@us.ibm.com 2017-09-08 15:59 EDT--- Yes I installed all 9 .debs, I just re-dowloaded, installed, booted but still no luck. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1713576 Title: More migrations with constant load Status in The Ubuntu-power-systems project: New Status in linux package in Ubuntu: Triaged Bug description: == Comment: #0 - PUVICHAKRAVARTHY RAMACHANDRAN - 2017-08-06 13:44:45 == ---Problem Description--- Significantly higher number of task migrations when the load is fixed but not balanced across cores. ---uname output--- Linux isvbos3 4.10.0-29-generic #33~16.04.1-Ubuntu SMP Tue Jul 25 18:17:06 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux ---Additional Hardware Info--- Power9 dd2.0 Machine Type = Power9 ---Steps to Reproduce--- Benchmark : Multithreaded - cpu intensive. The system had 2 socket/ 32 cores/ SMT4 mode. When 64 threads was run - the migrations were less over 10s interval. when 80 threads were run - the migrations were very high. Ideally, it should have been very minimal, as the over all load was constant == Comment: #3 - SRIKAR DRONAMRAJU - 2017-08-11 06:56:47 == As suspected (commit : 05b40e0577 : "sched/fair: Prefer sibiling only if local group is under-utilized") https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=05b40e0577 should fix the problem Ran ' perf stat -a -r 5 -e sched:sched_migrate_task /home/srikar/work/ebizzy-0.3/ebizzy -t 35 -S 100' to detect the problem and verify the fix Here is perf stat without fix. Performance counter stats for 'system wide' (5 runs): 7,758 sched:sched_migrate_task ( +- 1.28% ) 100.015658079 seconds time elapsed ( +- 0.00% ) perf stat with fix. Performance counter stats for 'system wide' (5 runs): 415 sched:sched_migrate_task ( +- 11.74% ) 100.016021787 seconds time elapsed ( +- 0.00% ) git describe on upstream kernel says v4.11-rc2 # git describe 05b40e0577 v4.11-rc2-227-g05b40e0 == Comment: #4 - SRIKAR DRONAMRAJU - 2017-08-11 07:05:37 == Attaching the patch that needs to be applied to fix this bug. Verified that patch fixes the problem. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1713576/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1713576] Comment bridged from LTC Bugzilla
--- Comment From jhop...@us.ibm.com 2017-09-07 17:48 EDT--- (In reply to comment #8) > I built a Zesty test kernel with commit 05b40e0577. The test kernel can be > downloaded from: > > http://kernel.ubuntu.com/~jsalisbury/lp1713576 > > Can you test this kernel and see if it resolves this bug? Thanks, but I am having trouble with the perf packages.. any idea what is wrong? # uname -a Linux ltc-boston114 4.10.0-33-generic #37~lp1713576 SMP Wed Sep 6 15:50:28 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux # dpkg -i /home/migration_test_build/linux-tools-common_4.10.0-33.37~lp1713576_all.deb (Reading database ... 99780 files and directories currently installed.) Preparing to unpack .../linux-tools-common_4.10.0-33.37~lp1713576_all.deb ... Unpacking linux-tools-common (4.10.0-33.37~lp1713576) over (4.10.0-33.37~lp1713576) ... Setting up linux-tools-common (4.10.0-33.37~lp1713576) ... # dpkg -i /home/migration_test_build/linux-cloud-tools-common_4.10.0-33.37~lp1713576_all.deb (Reading database ... 99780 files and directories currently installed.) Preparing to unpack .../linux-cloud-tools-common_4.10.0-33.37~lp1713576_all.deb ... Unpacking linux-cloud-tools-common (4.10.0-33.37~lp1713576) over (4.10.0-33.37~lp1713576) ... Setting up linux-cloud-tools-common (4.10.0-33.37~lp1713576) ... # perf list The program 'perf' is currently not installed. You can install it by typing: apt install linux-tools-common # apt install linux-tools-common Reading package lists... Done Building dependency tree Reading state information... Done linux-tools-common is already the newest version (4.10.0-33.37~lp1713576). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. --- Ebizzy does show a 42% improvement on average so that is probably enough, although I was hoping to validate the sched migration counts are lower. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1713576 Title: More migrations with constant load Status in The Ubuntu-power-systems project: New Status in linux package in Ubuntu: Triaged Bug description: == Comment: #0 - PUVICHAKRAVARTHY RAMACHANDRAN - 2017-08-06 13:44:45 == ---Problem Description--- Significantly higher number of task migrations when the load is fixed but not balanced across cores. ---uname output--- Linux isvbos3 4.10.0-29-generic #33~16.04.1-Ubuntu SMP Tue Jul 25 18:17:06 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux ---Additional Hardware Info--- Power9 dd2.0 Machine Type = Power9 ---Steps to Reproduce--- Benchmark : Multithreaded - cpu intensive. The system had 2 socket/ 32 cores/ SMT4 mode. When 64 threads was run - the migrations were less over 10s interval. when 80 threads were run - the migrations were very high. Ideally, it should have been very minimal, as the over all load was constant == Comment: #3 - SRIKAR DRONAMRAJU - 2017-08-11 06:56:47 == As suspected (commit : 05b40e0577 : "sched/fair: Prefer sibiling only if local group is under-utilized") https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=05b40e0577 should fix the problem Ran ' perf stat -a -r 5 -e sched:sched_migrate_task /home/srikar/work/ebizzy-0.3/ebizzy -t 35 -S 100' to detect the problem and verify the fix Here is perf stat without fix. Performance counter stats for 'system wide' (5 runs): 7,758 sched:sched_migrate_task ( +- 1.28% ) 100.015658079 seconds time elapsed ( +- 0.00% ) perf stat with fix. Performance counter stats for 'system wide' (5 runs): 415 sched:sched_migrate_task ( +- 11.74% ) 100.016021787 seconds time elapsed ( +- 0.00% ) git describe on upstream kernel says v4.11-rc2 # git describe 05b40e0577 v4.11-rc2-227-g05b40e0 == Comment: #4 - SRIKAR DRONAMRAJU - 2017-08-11 07:05:37 == Attaching the patch that needs to be applied to fix this bug. Verified that patch fixes the problem. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1713576/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
