Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-10 Thread Vincent Guittot
On Wed, 10 Mar 2021 at 06:53, Srikar Dronamraju wrote: > > * Vincent Guittot [2021-03-08 14:52:39]: > > > On Fri, 26 Feb 2021 at 17:41, Srikar Dronamraju > > wrote: > > > > > Thanks Vincent for your review comments. > > > > +static int prefer_idler_llc(int this_cpu, int prev_cpu, int sync) > >

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-09 Thread Srikar Dronamraju
* Vincent Guittot [2021-03-08 14:52:39]: > On Fri, 26 Feb 2021 at 17:41, Srikar Dronamraju > wrote: > > Thanks Vincent for your review comments. > > +static int prefer_idler_llc(int this_cpu, int prev_cpu, int sync) > > +{ > > + struct sched_domain_shared *tsds, *psds; > > + int

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-08 Thread Vincent Guittot
On Fri, 26 Feb 2021 at 17:41, Srikar Dronamraju wrote: > > On POWER8 and POWER9, the last level cache (L2) has been at the level of > a group of 8 threads (SMT8 on POWER8, a big-core comprising of a pair of > SMT4 cores on POWER9). However, on POWER10, the LLC domain is at the > level of a group

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-02 Thread Srikar Dronamraju
* Peter Zijlstra [2021-03-02 10:10:32]: > On Tue, Mar 02, 2021 at 01:09:46PM +0530, Srikar Dronamraju wrote: > > > Oh, could be, I didn't grep :/ We could have core code keep track of the > > > smt count I suppose. > > > > Could we use cpumask_weight(cpu_smt_mask(this_cpu)) instead? > >

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-02 Thread Srikar Dronamraju
* Dietmar Eggemann [2021-03-02 10:53:06]: > On 26/02/2021 17:40, Srikar Dronamraju wrote: > > [...] > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > index 8a8bd7b13634..d49bfcdc4a19 100644 > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c > > @@ -5869,6 +5869,36 @@

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-02 Thread Dietmar Eggemann
On 26/02/2021 17:40, Srikar Dronamraju wrote: [...] > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 8a8bd7b13634..d49bfcdc4a19 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -5869,6 +5869,36 @@ wake_affine_weight(struct sched_domain *sd, struct >

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-02 Thread Peter Zijlstra
On Tue, Mar 02, 2021 at 01:09:46PM +0530, Srikar Dronamraju wrote: > > Oh, could be, I didn't grep :/ We could have core code keep track of the > > smt count I suppose. > > Could we use cpumask_weight(cpu_smt_mask(this_cpu)) instead? cpumask_weight() is potentially super expensive. With

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-02 Thread Srikar Dronamraju
* Peter Zijlstra [2021-03-01 18:18:28]: > On Mon, Mar 01, 2021 at 10:36:01PM +0530, Srikar Dronamraju wrote: > > * Peter Zijlstra [2021-03-01 16:44:42]: > > > > > On Sat, Feb 27, 2021 at 02:56:07PM -0500, Rik van Riel wrote: > > > > On Fri, 2021-02-26 at 22:10 +0530, Srikar Dronamraju wrote: >

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-01 Thread Peter Zijlstra
On Mon, Mar 01, 2021 at 10:36:01PM +0530, Srikar Dronamraju wrote: > * Peter Zijlstra [2021-03-01 16:44:42]: > > > On Sat, Feb 27, 2021 at 02:56:07PM -0500, Rik van Riel wrote: > > > On Fri, 2021-02-26 at 22:10 +0530, Srikar Dronamraju wrote: > > > > > > + if (sched_feat(WA_WAKER) &&

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-01 Thread Srikar Dronamraju
* Peter Zijlstra [2021-03-01 16:40:33]: > On Fri, Feb 26, 2021 at 10:10:29PM +0530, Srikar Dronamraju wrote: > > +static int prefer_idler_llc(int this_cpu, int prev_cpu, int sync) > > +{ > > + struct sched_domain_shared *tsds, *psds; > > + int pnr_busy, pllc_size, tnr_busy, tllc_size, diff;

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-01 Thread Srikar Dronamraju
* Peter Zijlstra [2021-03-01 16:44:42]: > On Sat, Feb 27, 2021 at 02:56:07PM -0500, Rik van Riel wrote: > > On Fri, 2021-02-26 at 22:10 +0530, Srikar Dronamraju wrote: > > > > + if (sched_feat(WA_WAKER) && tnr_busy < tllc_size) > > > + return this_cpu; > > > > I wonder if we need to

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-01 Thread Peter Zijlstra
On Sat, Feb 27, 2021 at 02:56:07PM -0500, Rik van Riel wrote: > On Fri, 2021-02-26 at 22:10 +0530, Srikar Dronamraju wrote: > > + if (sched_feat(WA_WAKER) && tnr_busy < tllc_size) > > + return this_cpu; > > I wonder if we need to use a slightly lower threshold on > very large LLCs,

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-01 Thread Peter Zijlstra
On Fri, Feb 26, 2021 at 10:10:29PM +0530, Srikar Dronamraju wrote: > +static int prefer_idler_llc(int this_cpu, int prev_cpu, int sync) > +{ > + struct sched_domain_shared *tsds, *psds; > + int pnr_busy, pllc_size, tnr_busy, tllc_size, diff; > + > + tsds =

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-03-01 Thread Srikar Dronamraju
* Rik van Riel [2021-02-27 14:56:07]: > > In the current situation where waker and previous CPUs are busy, but > > only one of its LLC has an idle CPU, Scheduler may end up picking a > > LLC > > with no idle CPUs. To mitigate this, add a new step between 1 and 2 > > where Scheduler compares idle

Re: [PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-02-27 Thread Rik van Riel
On Fri, 2021-02-26 at 22:10 +0530, Srikar Dronamraju wrote: > Current order of preference to pick a LLC while waking a wake-affine > task: > 1. Between the waker CPU and previous CPU, prefer the LLC of the CPU >that is idle. > > 2. Between the waker CPU and previous CPU, prefer the LLC of

[PATCH] sched/fair: Prefer idle CPU to cache affinity

2021-02-26 Thread Srikar Dronamraju
On POWER8 and POWER9, the last level cache (L2) has been at the level of a group of 8 threads (SMT8 on POWER8, a big-core comprising of a pair of SMT4 cores on POWER9). However, on POWER10, the LLC domain is at the level of a group of SMT4 threads within the SMT8 core. Due to the shrinking in the