Re: [LKP] [rcu] 00f5159aeb6: +55.3% will-it-scale.time.involuntary_context_switches
On Wed, Jan 21, 2015 at 09:06:47AM +0100, Mike Galbraith wrote:
> On Tue, 2015-01-20 at 23:52 -0800, Paul E. McKenney wrote:
> > On Wed, Jan 21, 2015 at 03:32:44PM +0800, Huang Ying wrote:
> > > FYI, we noticed the below changes on
> > >
> > > git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
> > > stall.2015.01.06a
> > > commit 00f5159aeb68f08ce29597be86be87d6db0c1ba1 ("rcu: Run grace-period
> > > kthreads at real-time priority")
> >
> > Glad this helps!
>
> The way I read it, it's hurting by increasing context switches... but if
> you'll need a boot param to turn it on in the future, this is moot :)
Heh! I guess I should read it more carefully. Anyway, the commit does
help in some heavy-load situations, but something more deft is clearly
required if it is to be enabled more generally.
Thanx, Paul
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [LKP] [rcu] 00f5159aeb6: +55.3% will-it-scale.time.involuntary_context_switches
On Tue, 2015-01-20 at 23:52 -0800, Paul E. McKenney wrote:
> On Wed, Jan 21, 2015 at 03:32:44PM +0800, Huang Ying wrote:
> > FYI, we noticed the below changes on
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
> > stall.2015.01.06a
> > commit 00f5159aeb68f08ce29597be86be87d6db0c1ba1 ("rcu: Run grace-period
> > kthreads at real-time priority")
>
> Glad this helps!
The way I read it, it's hurting by increasing context switches... but if
you'll need a boot param to turn it on in the future, this is moot :)
-Mike
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [LKP] [rcu] 00f5159aeb6: +55.3% will-it-scale.time.involuntary_context_switches
On Tue, 2015-01-20 at 23:52 -0800, Paul E. McKenney wrote: On Wed, Jan 21, 2015 at 03:32:44PM +0800, Huang Ying wrote: FYI, we noticed the below changes on git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git stall.2015.01.06a commit 00f5159aeb68f08ce29597be86be87d6db0c1ba1 (rcu: Run grace-period kthreads at real-time priority) Glad this helps! The way I read it, it's hurting by increasing context switches... but if you'll need a boot param to turn it on in the future, this is moot :) -Mike -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [LKP] [rcu] 00f5159aeb6: +55.3% will-it-scale.time.involuntary_context_switches
On Wed, Jan 21, 2015 at 09:06:47AM +0100, Mike Galbraith wrote: On Tue, 2015-01-20 at 23:52 -0800, Paul E. McKenney wrote: On Wed, Jan 21, 2015 at 03:32:44PM +0800, Huang Ying wrote: FYI, we noticed the below changes on git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git stall.2015.01.06a commit 00f5159aeb68f08ce29597be86be87d6db0c1ba1 (rcu: Run grace-period kthreads at real-time priority) Glad this helps! The way I read it, it's hurting by increasing context switches... but if you'll need a boot param to turn it on in the future, this is moot :) Heh! I guess I should read it more carefully. Anyway, the commit does help in some heavy-load situations, but something more deft is clearly required if it is to be enabled more generally. Thanx, Paul -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [LKP] [rcu] 00f5159aeb6: +55.3% will-it-scale.time.involuntary_context_switches
On Wed, Jan 21, 2015 at 03:32:44PM +0800, Huang Ying wrote:
> FYI, we noticed the below changes on
>
> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
> stall.2015.01.06a
> commit 00f5159aeb68f08ce29597be86be87d6db0c1ba1 ("rcu: Run grace-period
> kthreads at real-time priority")
Glad this helps!
However, the version that I am pushing during the upcoming merge window
requires a boot parameter rcutree.kthread_prio= set to a value in the
range 1..99 to have effect. The reason is that we need to focus any
risks from this change on the people needing it.
Thanx, Paul
> testbox/testcase/testparams: lkp-a03/will-it-scale/performance-lock1
>
> e3663b1024d1f946 00f5159aeb68f08ce29597be86
> --
> %stddev %change %stddev
> \ |\
> 11321 ± 5% +55.3% 17582 ± 14%
> will-it-scale.time.involuntary_context_switches
> 28999 ± 44%+109.3% 60700 ± 25% sched_debug.cpu#1.ttwu_count
> 58384 ± 44%+103.6% 118851 ± 24% sched_debug.cpu#1.nr_switches
> 26144 ± 48% +90.3% 49762 ± 26% sched_debug.cpu#1.sched_goidle
>827 ± 34% -57.3%353 ± 21% cpuidle.C2-ATM.usage
> 58976 ± 43%+103.0% 119712 ± 24% sched_debug.cpu#1.sched_count
>503 ± 29% -56.5%219 ± 27% cpuidle.C4-ATM.usage
> 11321 ± 5% +55.3% 17582 ± 14%
> time.involuntary_context_switches
>156 ± 14% -29.6%110 ± 19% cpuidle.POLL.usage
> 50583 ± 2% -21.7% 39630 ± 4% cpuidle.C6-ATM.usage
> 51597156 ± 2% +23.7% 63848564 ± 3% cpuidle.C1E-ATM.time
> 39349 ± 2% -12.0% 34622 ± 2% softirqs.SCHED
> 758790 ± 8% -15.3% 642336 ± 12% sched_debug.cpu#2.sched_count
> 378981 ± 8% -15.6% 319776 ± 12% sched_debug.cpu#2.ttwu_count
> 374440 ± 8% -15.5% 316513 ± 12% sched_debug.cpu#2.ttwu_local
> 5307 ± 0% -1.0% 5255 ± 0% vmstat.system.in
>
> testbox/testcase/testparams: wsm/will-it-scale/performance-open2
>
> e3663b1024d1f946 00f5159aeb68f08ce29597be86
> --
> 22128 ± 3%+120.2% 48719 ± 8%
> will-it-scale.time.involuntary_context_switches
> 5577 ± 45% +1492.2% 88802 ± 33% sched_debug.cpu#6.sched_count
> 5457 ± 46% +1526.2% 88751 ± 33% sched_debug.cpu#6.nr_switches
> 0.45 ± 43% -73.3% 0.12 ± 15% turbostat.%c3
> 10 ± 24% -55.8% 4 ± 40%
> sched_debug.cpu#9.nr_uninterruptible
>147 ± 11% -58.5% 61 ± 32% sched_debug.cpu#11.load
>147 ± 11% -58.5% 61 ± 32% sched_debug.cfs_rq[11]:/.load
> 10821106 ± 39% -59.8%4347430 ± 20% cpuidle.C3-NHM.time
>356 ± 41% +76.6%629 ± 3%
> sched_debug.cfs_rq[6]:/.blocked_load_avg
> 22128 ± 3%+120.2% 48719 ± 8%
> time.involuntary_context_switches
>145 ± 36% -35.9% 93 ± 30% sched_debug.cpu#5.load
>469 ± 29% +55.9%732 ± 4%
> sched_debug.cfs_rq[6]:/.tg_load_contrib
>145 ± 36% -34.0% 95 ± 29% sched_debug.cfs_rq[5]:/.load
> 99 ± 8% -27.8% 72 ± 24% sched_debug.cpu#9.load
> 99 ± 8% -27.8% 72 ± 24% sched_debug.cfs_rq[9]:/.load
> 219871 ± 6% -33.3% 146545 ± 19% sched_debug.cpu#3.ttwu_local
> 224610 ± 6% -31.3% 154342 ± 18% sched_debug.cpu#3.sched_goidle
> 451877 ± 6% -31.1% 311378 ± 18% sched_debug.cpu#3.sched_count
> 451737 ± 6% -31.1% 311279 ± 18% sched_debug.cpu#3.nr_switches
> 225208 ± 6% -31.3% 154688 ± 18% sched_debug.cpu#3.ttwu_count
> 4122 ± 7% -23.6% 3148 ± 17% cpuidle.C3-NHM.usage
> 84 ± 11% -27.4% 61 ± 5% sched_debug.cpu#7.cpu_load[0]
> 154565 ± 2% -29.6% 108768 ± 9% cpuidle.C6-NHM.usage
> 124949 ± 25% +41.4% 176670 ± 15% sched_debug.cpu#2.sched_goidle
> 74177 ± 2% -23.6% 56653 ± 4%
> sched_debug.cpu#3.nr_load_updates
> 79 ± 6% -23.1% 60 ± 5% sched_debug.cpu#7.cpu_load[1]
> 50107 ± 6% -20.9% 39621 ± 7%
> sched_debug.cfs_rq[3]:/.exec_clock
> 76 ± 4% -20.6% 60 ± 6% sched_debug.cpu#7.cpu_load[2]
> 385983 ± 9% -20.4% 307347 ± 12%
> sched_debug.cfs_rq[3]:/.min_vruntime
> 74 ± 2% -18.5% 60 ± 6% sched_debug.cpu#7.cpu_load[3]
> 1.50 ± 6% -10.0% 1.35 ± 9%
> perf-profile.cpu-cycles.selinux_file_free_security.security_file_free.__fput.fput.task_work_run
> 46086 ± 1% -25.7% 34260 ± 5% softirqs.SCHED
> 72 ± 1% -15.9% 60 ± 7% sched_debug.cpu#7.cpu_load[4]
> 74 ± 13% -17.7% 61 ± 2%
>
[LKP] [rcu] 00f5159aeb6: +55.3% will-it-scale.time.involuntary_context_switches
FYI, we noticed the below changes on
git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
stall.2015.01.06a
commit 00f5159aeb68f08ce29597be86be87d6db0c1ba1 ("rcu: Run grace-period
kthreads at real-time priority")
testbox/testcase/testparams: lkp-a03/will-it-scale/performance-lock1
e3663b1024d1f946 00f5159aeb68f08ce29597be86
--
%stddev %change %stddev
\ |\
11321 ± 5% +55.3% 17582 ± 14%
will-it-scale.time.involuntary_context_switches
28999 ± 44%+109.3% 60700 ± 25% sched_debug.cpu#1.ttwu_count
58384 ± 44%+103.6% 118851 ± 24% sched_debug.cpu#1.nr_switches
26144 ± 48% +90.3% 49762 ± 26% sched_debug.cpu#1.sched_goidle
827 ± 34% -57.3%353 ± 21% cpuidle.C2-ATM.usage
58976 ± 43%+103.0% 119712 ± 24% sched_debug.cpu#1.sched_count
503 ± 29% -56.5%219 ± 27% cpuidle.C4-ATM.usage
11321 ± 5% +55.3% 17582 ± 14% time.involuntary_context_switches
156 ± 14% -29.6%110 ± 19% cpuidle.POLL.usage
50583 ± 2% -21.7% 39630 ± 4% cpuidle.C6-ATM.usage
51597156 ± 2% +23.7% 63848564 ± 3% cpuidle.C1E-ATM.time
39349 ± 2% -12.0% 34622 ± 2% softirqs.SCHED
758790 ± 8% -15.3% 642336 ± 12% sched_debug.cpu#2.sched_count
378981 ± 8% -15.6% 319776 ± 12% sched_debug.cpu#2.ttwu_count
374440 ± 8% -15.5% 316513 ± 12% sched_debug.cpu#2.ttwu_local
5307 ± 0% -1.0% 5255 ± 0% vmstat.system.in
testbox/testcase/testparams: wsm/will-it-scale/performance-open2
e3663b1024d1f946 00f5159aeb68f08ce29597be86
--
22128 ± 3%+120.2% 48719 ± 8%
will-it-scale.time.involuntary_context_switches
5577 ± 45% +1492.2% 88802 ± 33% sched_debug.cpu#6.sched_count
5457 ± 46% +1526.2% 88751 ± 33% sched_debug.cpu#6.nr_switches
0.45 ± 43% -73.3% 0.12 ± 15% turbostat.%c3
10 ± 24% -55.8% 4 ± 40%
sched_debug.cpu#9.nr_uninterruptible
147 ± 11% -58.5% 61 ± 32% sched_debug.cpu#11.load
147 ± 11% -58.5% 61 ± 32% sched_debug.cfs_rq[11]:/.load
10821106 ± 39% -59.8%4347430 ± 20% cpuidle.C3-NHM.time
356 ± 41% +76.6%629 ± 3%
sched_debug.cfs_rq[6]:/.blocked_load_avg
22128 ± 3%+120.2% 48719 ± 8% time.involuntary_context_switches
145 ± 36% -35.9% 93 ± 30% sched_debug.cpu#5.load
469 ± 29% +55.9%732 ± 4%
sched_debug.cfs_rq[6]:/.tg_load_contrib
145 ± 36% -34.0% 95 ± 29% sched_debug.cfs_rq[5]:/.load
99 ± 8% -27.8% 72 ± 24% sched_debug.cpu#9.load
99 ± 8% -27.8% 72 ± 24% sched_debug.cfs_rq[9]:/.load
219871 ± 6% -33.3% 146545 ± 19% sched_debug.cpu#3.ttwu_local
224610 ± 6% -31.3% 154342 ± 18% sched_debug.cpu#3.sched_goidle
451877 ± 6% -31.1% 311378 ± 18% sched_debug.cpu#3.sched_count
451737 ± 6% -31.1% 311279 ± 18% sched_debug.cpu#3.nr_switches
225208 ± 6% -31.3% 154688 ± 18% sched_debug.cpu#3.ttwu_count
4122 ± 7% -23.6% 3148 ± 17% cpuidle.C3-NHM.usage
84 ± 11% -27.4% 61 ± 5% sched_debug.cpu#7.cpu_load[0]
154565 ± 2% -29.6% 108768 ± 9% cpuidle.C6-NHM.usage
124949 ± 25% +41.4% 176670 ± 15% sched_debug.cpu#2.sched_goidle
74177 ± 2% -23.6% 56653 ± 4% sched_debug.cpu#3.nr_load_updates
79 ± 6% -23.1% 60 ± 5% sched_debug.cpu#7.cpu_load[1]
50107 ± 6% -20.9% 39621 ± 7% sched_debug.cfs_rq[3]:/.exec_clock
76 ± 4% -20.6% 60 ± 6% sched_debug.cpu#7.cpu_load[2]
385983 ± 9% -20.4% 307347 ± 12%
sched_debug.cfs_rq[3]:/.min_vruntime
74 ± 2% -18.5% 60 ± 6% sched_debug.cpu#7.cpu_load[3]
1.50 ± 6% -10.0% 1.35 ± 9%
perf-profile.cpu-cycles.selinux_file_free_security.security_file_free.__fput.fput.task_work_run
46086 ± 1% -25.7% 34260 ± 5% softirqs.SCHED
72 ± 1% -15.9% 60 ± 7% sched_debug.cpu#7.cpu_load[4]
74 ± 13% -17.7% 61 ± 2%
sched_debug.cfs_rq[7]:/.runnable_load_avg
65 ± 2% +16.9% 76 ± 4% sched_debug.cpu#1.cpu_load[3]
1.94 ± 5% -9.9% 1.74 ± 6%
perf-profile.cpu-cycles.security_file_free.__fput.fput.task_work_run.do_notify_resume
58851 ± 1% +20.5% 70942 ± 9% sched_debug.cfs_rq[1]:/.exec_clock
69880 ± 1% -16.9% 58047 ± 11% sched_debug.cpu#7.nr_load_updates
60 ± 5% +9.5% 66 ± 3% sched_debug.cpu#9.cpu_load[4]
12064 ± 7% +12.4% 13561 ± 5% slabinfo.kmalloc-64.num_objs
[LKP] [rcu] 00f5159aeb6: +55.3% will-it-scale.time.involuntary_context_switches
FYI, we noticed the below changes on git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git stall.2015.01.06a commit 00f5159aeb68f08ce29597be86be87d6db0c1ba1 (rcu: Run grace-period kthreads at real-time priority) testbox/testcase/testparams: lkp-a03/will-it-scale/performance-lock1 e3663b1024d1f946 00f5159aeb68f08ce29597be86 -- %stddev %change %stddev \ |\ 11321 ± 5% +55.3% 17582 ± 14% will-it-scale.time.involuntary_context_switches 28999 ± 44%+109.3% 60700 ± 25% sched_debug.cpu#1.ttwu_count 58384 ± 44%+103.6% 118851 ± 24% sched_debug.cpu#1.nr_switches 26144 ± 48% +90.3% 49762 ± 26% sched_debug.cpu#1.sched_goidle 827 ± 34% -57.3%353 ± 21% cpuidle.C2-ATM.usage 58976 ± 43%+103.0% 119712 ± 24% sched_debug.cpu#1.sched_count 503 ± 29% -56.5%219 ± 27% cpuidle.C4-ATM.usage 11321 ± 5% +55.3% 17582 ± 14% time.involuntary_context_switches 156 ± 14% -29.6%110 ± 19% cpuidle.POLL.usage 50583 ± 2% -21.7% 39630 ± 4% cpuidle.C6-ATM.usage 51597156 ± 2% +23.7% 63848564 ± 3% cpuidle.C1E-ATM.time 39349 ± 2% -12.0% 34622 ± 2% softirqs.SCHED 758790 ± 8% -15.3% 642336 ± 12% sched_debug.cpu#2.sched_count 378981 ± 8% -15.6% 319776 ± 12% sched_debug.cpu#2.ttwu_count 374440 ± 8% -15.5% 316513 ± 12% sched_debug.cpu#2.ttwu_local 5307 ± 0% -1.0% 5255 ± 0% vmstat.system.in testbox/testcase/testparams: wsm/will-it-scale/performance-open2 e3663b1024d1f946 00f5159aeb68f08ce29597be86 -- 22128 ± 3%+120.2% 48719 ± 8% will-it-scale.time.involuntary_context_switches 5577 ± 45% +1492.2% 88802 ± 33% sched_debug.cpu#6.sched_count 5457 ± 46% +1526.2% 88751 ± 33% sched_debug.cpu#6.nr_switches 0.45 ± 43% -73.3% 0.12 ± 15% turbostat.%c3 10 ± 24% -55.8% 4 ± 40% sched_debug.cpu#9.nr_uninterruptible 147 ± 11% -58.5% 61 ± 32% sched_debug.cpu#11.load 147 ± 11% -58.5% 61 ± 32% sched_debug.cfs_rq[11]:/.load 10821106 ± 39% -59.8%4347430 ± 20% cpuidle.C3-NHM.time 356 ± 41% +76.6%629 ± 3% sched_debug.cfs_rq[6]:/.blocked_load_avg 22128 ± 3%+120.2% 48719 ± 8% time.involuntary_context_switches 145 ± 36% -35.9% 93 ± 30% sched_debug.cpu#5.load 469 ± 29% +55.9%732 ± 4% sched_debug.cfs_rq[6]:/.tg_load_contrib 145 ± 36% -34.0% 95 ± 29% sched_debug.cfs_rq[5]:/.load 99 ± 8% -27.8% 72 ± 24% sched_debug.cpu#9.load 99 ± 8% -27.8% 72 ± 24% sched_debug.cfs_rq[9]:/.load 219871 ± 6% -33.3% 146545 ± 19% sched_debug.cpu#3.ttwu_local 224610 ± 6% -31.3% 154342 ± 18% sched_debug.cpu#3.sched_goidle 451877 ± 6% -31.1% 311378 ± 18% sched_debug.cpu#3.sched_count 451737 ± 6% -31.1% 311279 ± 18% sched_debug.cpu#3.nr_switches 225208 ± 6% -31.3% 154688 ± 18% sched_debug.cpu#3.ttwu_count 4122 ± 7% -23.6% 3148 ± 17% cpuidle.C3-NHM.usage 84 ± 11% -27.4% 61 ± 5% sched_debug.cpu#7.cpu_load[0] 154565 ± 2% -29.6% 108768 ± 9% cpuidle.C6-NHM.usage 124949 ± 25% +41.4% 176670 ± 15% sched_debug.cpu#2.sched_goidle 74177 ± 2% -23.6% 56653 ± 4% sched_debug.cpu#3.nr_load_updates 79 ± 6% -23.1% 60 ± 5% sched_debug.cpu#7.cpu_load[1] 50107 ± 6% -20.9% 39621 ± 7% sched_debug.cfs_rq[3]:/.exec_clock 76 ± 4% -20.6% 60 ± 6% sched_debug.cpu#7.cpu_load[2] 385983 ± 9% -20.4% 307347 ± 12% sched_debug.cfs_rq[3]:/.min_vruntime 74 ± 2% -18.5% 60 ± 6% sched_debug.cpu#7.cpu_load[3] 1.50 ± 6% -10.0% 1.35 ± 9% perf-profile.cpu-cycles.selinux_file_free_security.security_file_free.__fput.fput.task_work_run 46086 ± 1% -25.7% 34260 ± 5% softirqs.SCHED 72 ± 1% -15.9% 60 ± 7% sched_debug.cpu#7.cpu_load[4] 74 ± 13% -17.7% 61 ± 2% sched_debug.cfs_rq[7]:/.runnable_load_avg 65 ± 2% +16.9% 76 ± 4% sched_debug.cpu#1.cpu_load[3] 1.94 ± 5% -9.9% 1.74 ± 6% perf-profile.cpu-cycles.security_file_free.__fput.fput.task_work_run.do_notify_resume 58851 ± 1% +20.5% 70942 ± 9% sched_debug.cfs_rq[1]:/.exec_clock 69880 ± 1% -16.9% 58047 ± 11% sched_debug.cpu#7.nr_load_updates 60 ± 5% +9.5% 66 ± 3% sched_debug.cpu#9.cpu_load[4] 12064 ± 7% +12.4% 13561 ± 5% slabinfo.kmalloc-64.num_objs
Re: [LKP] [rcu] 00f5159aeb6: +55.3% will-it-scale.time.involuntary_context_switches
On Wed, Jan 21, 2015 at 03:32:44PM +0800, Huang Ying wrote: FYI, we noticed the below changes on git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git stall.2015.01.06a commit 00f5159aeb68f08ce29597be86be87d6db0c1ba1 (rcu: Run grace-period kthreads at real-time priority) Glad this helps! However, the version that I am pushing during the upcoming merge window requires a boot parameter rcutree.kthread_prio= set to a value in the range 1..99 to have effect. The reason is that we need to focus any risks from this change on the people needing it. Thanx, Paul testbox/testcase/testparams: lkp-a03/will-it-scale/performance-lock1 e3663b1024d1f946 00f5159aeb68f08ce29597be86 -- %stddev %change %stddev \ |\ 11321 ± 5% +55.3% 17582 ± 14% will-it-scale.time.involuntary_context_switches 28999 ± 44%+109.3% 60700 ± 25% sched_debug.cpu#1.ttwu_count 58384 ± 44%+103.6% 118851 ± 24% sched_debug.cpu#1.nr_switches 26144 ± 48% +90.3% 49762 ± 26% sched_debug.cpu#1.sched_goidle 827 ± 34% -57.3%353 ± 21% cpuidle.C2-ATM.usage 58976 ± 43%+103.0% 119712 ± 24% sched_debug.cpu#1.sched_count 503 ± 29% -56.5%219 ± 27% cpuidle.C4-ATM.usage 11321 ± 5% +55.3% 17582 ± 14% time.involuntary_context_switches 156 ± 14% -29.6%110 ± 19% cpuidle.POLL.usage 50583 ± 2% -21.7% 39630 ± 4% cpuidle.C6-ATM.usage 51597156 ± 2% +23.7% 63848564 ± 3% cpuidle.C1E-ATM.time 39349 ± 2% -12.0% 34622 ± 2% softirqs.SCHED 758790 ± 8% -15.3% 642336 ± 12% sched_debug.cpu#2.sched_count 378981 ± 8% -15.6% 319776 ± 12% sched_debug.cpu#2.ttwu_count 374440 ± 8% -15.5% 316513 ± 12% sched_debug.cpu#2.ttwu_local 5307 ± 0% -1.0% 5255 ± 0% vmstat.system.in testbox/testcase/testparams: wsm/will-it-scale/performance-open2 e3663b1024d1f946 00f5159aeb68f08ce29597be86 -- 22128 ± 3%+120.2% 48719 ± 8% will-it-scale.time.involuntary_context_switches 5577 ± 45% +1492.2% 88802 ± 33% sched_debug.cpu#6.sched_count 5457 ± 46% +1526.2% 88751 ± 33% sched_debug.cpu#6.nr_switches 0.45 ± 43% -73.3% 0.12 ± 15% turbostat.%c3 10 ± 24% -55.8% 4 ± 40% sched_debug.cpu#9.nr_uninterruptible 147 ± 11% -58.5% 61 ± 32% sched_debug.cpu#11.load 147 ± 11% -58.5% 61 ± 32% sched_debug.cfs_rq[11]:/.load 10821106 ± 39% -59.8%4347430 ± 20% cpuidle.C3-NHM.time 356 ± 41% +76.6%629 ± 3% sched_debug.cfs_rq[6]:/.blocked_load_avg 22128 ± 3%+120.2% 48719 ± 8% time.involuntary_context_switches 145 ± 36% -35.9% 93 ± 30% sched_debug.cpu#5.load 469 ± 29% +55.9%732 ± 4% sched_debug.cfs_rq[6]:/.tg_load_contrib 145 ± 36% -34.0% 95 ± 29% sched_debug.cfs_rq[5]:/.load 99 ± 8% -27.8% 72 ± 24% sched_debug.cpu#9.load 99 ± 8% -27.8% 72 ± 24% sched_debug.cfs_rq[9]:/.load 219871 ± 6% -33.3% 146545 ± 19% sched_debug.cpu#3.ttwu_local 224610 ± 6% -31.3% 154342 ± 18% sched_debug.cpu#3.sched_goidle 451877 ± 6% -31.1% 311378 ± 18% sched_debug.cpu#3.sched_count 451737 ± 6% -31.1% 311279 ± 18% sched_debug.cpu#3.nr_switches 225208 ± 6% -31.3% 154688 ± 18% sched_debug.cpu#3.ttwu_count 4122 ± 7% -23.6% 3148 ± 17% cpuidle.C3-NHM.usage 84 ± 11% -27.4% 61 ± 5% sched_debug.cpu#7.cpu_load[0] 154565 ± 2% -29.6% 108768 ± 9% cpuidle.C6-NHM.usage 124949 ± 25% +41.4% 176670 ± 15% sched_debug.cpu#2.sched_goidle 74177 ± 2% -23.6% 56653 ± 4% sched_debug.cpu#3.nr_load_updates 79 ± 6% -23.1% 60 ± 5% sched_debug.cpu#7.cpu_load[1] 50107 ± 6% -20.9% 39621 ± 7% sched_debug.cfs_rq[3]:/.exec_clock 76 ± 4% -20.6% 60 ± 6% sched_debug.cpu#7.cpu_load[2] 385983 ± 9% -20.4% 307347 ± 12% sched_debug.cfs_rq[3]:/.min_vruntime 74 ± 2% -18.5% 60 ± 6% sched_debug.cpu#7.cpu_load[3] 1.50 ± 6% -10.0% 1.35 ± 9% perf-profile.cpu-cycles.selinux_file_free_security.security_file_free.__fput.fput.task_work_run 46086 ± 1% -25.7% 34260 ± 5% softirqs.SCHED 72 ± 1% -15.9% 60 ± 7% sched_debug.cpu#7.cpu_load[4] 74 ± 13% -17.7% 61 ± 2% sched_debug.cfs_rq[7]:/.runnable_load_avg 65 ± 2% +16.9% 76 ±
