Re: [PATCH 5/5] sched/fair: Merge select_idle_core/cpu()

2021-01-20 Thread Mel Gorman
On Wed, Jan 20, 2021 at 10:21:47AM +0100, Vincent Guittot wrote: > On Wed, 20 Jan 2021 at 10:12, Mel Gorman wrote: > > > > On Wed, Jan 20, 2021 at 02:00:18PM +0530, Gautham R Shenoy wrote: > > > > @@ -6157,18 +6169,31 @@ static int select_idle_cpu(struct task

Re: [PATCH 5/5] sched/fair: Merge select_idle_core/cpu()

2021-01-20 Thread Mel Gorman
t; > + set_idle_cores(this, false); > > Shouldn't we set_idle_cores(false) only if this was the last idle > core in the LLC ? > That would involve rechecking the cpumask bits that have not been scanned to see if any of them are an idle core. As the existance of idle cores can change very rapidly, it's not worth the cost. -- Mel Gorman SUSE Labs

Re: [PATCH v3 0/5] Scan for an idle sibling in a single pass

2021-01-19 Thread Mel Gorman
On Tue, Jan 19, 2021 at 12:33:04PM +0100, Vincent Guittot wrote: > On Tue, 19 Jan 2021 at 12:22, Mel Gorman wrote: > > > > Changelog since v2 > > o Remove unnecessary parameters > > o Update nr during scan only when scanning for cpus > > Hi Mel, > > I

[PATCH 5/5] sched/fair: Merge select_idle_core/cpu()

2021-01-19 Thread Mel Gorman
for an idle core. This way we'll only iterate every CPU once. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 101 ++-- 1 file changed, 61 insertions(+), 40 deletions(-) diff --git a/kernel/sched/fair.c b/kernel

[PATCH 4/5] sched/fair: Remove select_idle_smt()

2021-01-19 Thread Mel Gorman
From: Peter Zijlstra (Intel) In order to make the next patch more readable, and to quantify the actual effectiveness of this pass, start by removing it. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 30 -- 1 file changed

[PATCH 3/5] sched/fair: Make select_idle_cpu() proportional to cores

2021-01-19 Thread Mel Gorman
From: Peter Zijlstra (Intel) Instead of calculating how many (logical) CPUs to scan, compute how many cores to scan. This changes behaviour for anything !SMT2. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Mel Gorman --- kernel/sched/core.c | 18 +- kernel/sched

[PATCH 2/5] sched/fair: Move avg_scan_cost calculations under SIS_PROP

2021-01-19 Thread Mel Gorman
As noted by Vincent Guittot, avg_scan_costs are calculated for SIS_PROP even if SIS_PROP is disabled. Move the time calculations under a SIS_PROP check and while we are at it, exclude the cost of initialising the CPU mask from the average scan cost. Signed-off-by: Mel Gorman --- kernel/sched

[PATCH 1/5] sched/fair: Remove SIS_AVG_CPU

2021-01-19 Thread Mel Gorman
dle_cpu(), lets drop SIS_AVG_CPU and focus on SIS_PROP as a throttling mechanism. Signed-off-by: Mel Gorman Reviewed-by: Vincent Guittot --- kernel/sched/fair.c | 20 +--- kernel/sched/features.h | 1 - 2 files changed, 9 insertions(+), 12 deletions(-) diff --git a/kernel/sched/f

[PATCH v3 0/5] Scan for an idle sibling in a single pass

2021-01-19 Thread Mel Gorman
Changelog since v2 o Remove unnecessary parameters o Update nr during scan only when scanning for cpus Changlog since v1 o Move extern declaration to header for coding style o Remove unnecessary parameter from __select_idle_cpu This series of 5 patches reposts three patches from Peter entitled

Re: [PATCH 5/5] sched/fair: Merge select_idle_core/cpu()

2021-01-18 Thread Mel Gorman
return i; > > What if the last idle core is selected here, should we set_idle_cores false > before return? > We'd have to check what bits were still set in the cpus mask and determine if they represent an idle core. I severely doubt it would be worth the cost given that the availability of idle cores can change at any instant. -- Mel Gorman SUSE Labs

Re: [RFC PATCH] Remove redundant sched_numa_balancing check.

2021-01-18 Thread Mel Gorman
still get handled at fault, there is no point tracking the fault information in task_numa_fault and this function can still get called after sched_numa_balancing is disabled. -- Mel Gorman SUSE Labs

Re: [PATCH] mm, compaction: move high_pfn to the for loop scope.

2021-01-18 Thread Mel Gorman
e and freelist match when call move_freelist_head in > fast_isolate_freepages(). > > Fixes: 5a811889de10f1eb ("mm, compaction: use free lists to quickly locate a > migration target") > > Signed-off-by: Rokudo Yan Other than a fixes line, I do not think this changed so my previous Ack still applies Acked-by: Mel Gorman -- Mel Gorman SUSE Labs

Re: [PATCH 3/5] sched/fair: Make select_idle_cpu() proportional to cores

2021-01-18 Thread Mel Gorman
smt_weight if > !CONFIG_SCHED_SMT? > There already is a #define sched_smt_weight for !CONFIG_SCHED_SMT and I do not think an inline wrapper would make it more readable or maintainable. -- Mel Gorman SUSE Labs

[PATCH v2 0/5] Scan for an idle sibling in a single pass

2021-01-15 Thread Mel Gorman
Changlog since v1 o Move extern declaration to header for coding style o Remove unnecessary parameter from __select_idle_cpu This series of 5 patches reposts three patches from Peter entitled "select_idle_sibling() wreckage". It only scans the runqueues in a single pass when searching for an idle

[PATCH 1/5] sched/fair: Remove SIS_AVG_CPU

2021-01-15 Thread Mel Gorman
dle_cpu(), lets drop SIS_AVG_CPU and focus on SIS_PROP as a throttling mechanism. Signed-off-by: Mel Gorman Reviewed-by: Vincent Guittot --- kernel/sched/fair.c | 20 +--- kernel/sched/features.h | 1 - 2 files changed, 9 insertions(+), 12 deletions(-) diff --git a/kernel/sched/f

[PATCH 3/5] sched/fair: Make select_idle_cpu() proportional to cores

2021-01-15 Thread Mel Gorman
From: Peter Zijlstra (Intel) Instead of calculating how many (logical) CPUs to scan, compute how many cores to scan. This changes behaviour for anything !SMT2. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Mel Gorman --- kernel/sched/core.c | 18 +- kernel/sched

[PATCH 5/5] sched/fair: Merge select_idle_core/cpu()

2021-01-15 Thread Mel Gorman
for an idle core. This way we'll only iterate every CPU once. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 97 +++-- 1 file changed, 59 insertions(+), 38 deletions(-) diff --git a/kernel/sched/fair.c b/kernel

[PATCH 2/5] sched/fair: Move avg_scan_cost calculations under SIS_PROP

2021-01-15 Thread Mel Gorman
As noted by Vincent Guittot, avg_scan_costs are calculated for SIS_PROP even if SIS_PROP is disabled. Move the time calculations under a SIS_PROP check and while we are at it, exclude the cost of initialising the CPU mask from the average scan cost. Signed-off-by: Mel Gorman --- kernel/sched

[PATCH 4/5] sched/fair: Remove select_idle_smt()

2021-01-15 Thread Mel Gorman
From: Peter Zijlstra (Intel) In order to make the next patch more readable, and to quantify the actual effectiveness of this pass, start by removing it. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 30 -- 1 file changed

Re: [PATCH 5/5] sched/fair: Merge select_idle_core/cpu()

2021-01-14 Thread Mel Gorman
b/kernel/sched/sched.h index 12ada79d40f3..29aabe98dd1d 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1107,6 +1107,8 @@ static inline void update_idle_core(struct rq *rq) __update_idle_core(rq); } +extern int sched_smt_weight; + #else static inline void update_idle_core(struct rq *rq) { } #endif -- Mel Gorman SUSE Labs

Re: [PATCH 5/5] sched/fair: Merge select_idle_core/cpu()

2021-01-14 Thread Mel Gorman
On Thu, Jan 14, 2021 at 02:25:32PM +0100, Vincent Guittot wrote: > On Thu, 14 Jan 2021 at 10:35, Mel Gorman wrote: > > > > On Wed, Jan 13, 2021 at 06:03:00PM +0100, Vincent Guittot wrote: > > > > @@ -6159,16 +6171,29 @@ static int select_idle_cpu(struct task

Re: [PATCH 5/5] sched/fair: Merge select_idle_core/cpu()

2021-01-14 Thread Mel Gorman
ment but it would be nice to avoid excessive deep scanning when there are a lot of busy CPUs but individual tasks are rapidly idling. However, in the event regressions are found, changing to your suggested behaviour would be an obvious starting point. -- Mel Gorman SUSE Labs

Re: [PATCH 3/5] sched/fair: Make select_idle_cpu() proportional to cores

2021-01-14 Thread Mel Gorman
the need to multiply by > sched_smt_weight one patch 5 is applied > It makes sense in the context of this patch but can be removed again in the last patch and then I think sched_smt_weight only exists in core.c -- Mel Gorman SUSE Labs

Re: [PATCH] mm, compaction: move high_pfn to the for loop scope.

2021-01-12 Thread Mel Gorman
> freepage and freelist match when call move_freelist_head in > fast_isolate_freepages(). > > Link: > http://lkml.kernel.org/r/20190118175136.31341-12-mgor...@techsingularity.net > Fixes: 5a811889de10f1eb ("mm, compaction: use free lists to quickly locate a > migration target") Thanks. Acked-by: Mel Gorman -- Mel Gorman SUSE Labs

Re: [PATCH 3/3 v2] sched/fair: reduce cases for active balance

2021-01-12 Thread Mel Gorman
> condition if ((load >> env->sd->nr_balance_failed) > env->imbalance) in > detach_tasks()) and the waiting task will end up to be selected after a > number of attempts. > > Signed-off-by: Vincent Guittot I didn't see major problems so. Acked-by: Mel Gorman Like

Re: [PATCH] mm, compaction: make sure we isolate a valid freepage when high_pfn is used

2021-01-12 Thread Mel Gorman
et") In that case, please move the high_pfn declaration and initialiser within the for (order = cc->order - 1;...) loop to make it explicit what the scope of high_pfn is. -- Mel Gorman SUSE Labs

Re: [RFC][PATCH 1/5] sched/fair: Fix select_idle_cpu()s cost accounting

2021-01-11 Thread Mel Gorman
a superior selection of a scan > > depth. > > > > Treating a successful scan cost and a failed scan cost as being equal has > > too many corner cases. If we do not want to weight the successful scan > > cost, then the compromise is to keep the old behaviour that accounts for > > I think that keeping the current way to scane_cost id the best option for now > I sent a series that drops this patch for the moment as well as the SIS_PROP for selecting a core. -- Mel Gorman SUSE Labs

[PATCH 4/5] sched/fair: Remove select_idle_smt()

2021-01-11 Thread Mel Gorman
From: Peter Zijlstra (Intel) In order to make the next patch more readable, and to quantify the actual effectiveness of this pass, start by removing it. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 30 -- 1 file changed

[PATCH 3/5] sched/fair: Make select_idle_cpu() proportional to cores

2021-01-11 Thread Mel Gorman
From: Peter Zijlstra (Intel) Instead of calculating how many (logical) CPUs to scan, compute how many cores to scan. This changes behaviour for anything !SMT2. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Mel Gorman --- kernel/sched/core.c | 19 ++- kernel/sched

[PATCH 5/5] sched/fair: Merge select_idle_core/cpu()

2021-01-11 Thread Mel Gorman
only iterate every CPU once. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 97 +++-- 1 file changed, 59 insertions(+), 38 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 12e08da90024

[PATCH 0/5] Scan for an idle sibling in a single pass

2021-01-11 Thread Mel Gorman
This series of 5 patches reposts three patches from Peter entitled "select_idle_sibling() wreckage". It only scans the runqueues in a single pass when searching for an idle sibling. Two patches from Peter were dropped. The first patch altered how scan depth was calculated. Scan depth deletion is

[PATCH 2/5] sched/fair: Move avg_scan_cost calculations under SIS_PROP

2021-01-11 Thread Mel Gorman
As noted by Vincent Guittot, avg_scan_costs are calculated for SIS_PROP even if SIS_PROP is disabled. Move the time calculations under a SIS_PROP check and while we are at it, exclude the cost of initialising the CPU mask from the average scan cost. Signed-off-by: Mel Gorman --- kernel/sched

[PATCH 1/5] sched/fair: Remove SIS_AVG_CPU

2021-01-11 Thread Mel Gorman
dle_cpu(), lets drop SIS_AVG_CPU and focus on SIS_PROP as a throttling mechanism. Signed-off-by: Mel Gorman Reviewed-by: Vincent Guittot --- kernel/sched/fair.c | 20 +--- kernel/sched/features.h | 1 - 2 files changed, 9 insertions(+), 12 deletions(-) diff --git a/kernel/sched/f

Re: [PATCH 2/3 v2] sched/fair: don't set LBF_ALL_PINNED unnecessarily

2021-01-11 Thread Mel Gorman
ion. Otherwise, I didn't spot a real problem so Acked-by: Mel Gorman -- Mel Gorman SUSE Labs

Re: [PATCH 1/3 v2] sched/fair: skip idle cfs_rq

2021-01-11 Thread Mel Gorman
Vincent Guittot > Reviewed-by: Valentin Schneider Acked-by: Mel Gorman -- Mel Gorman SUSE Labs

Re: [RFC][PATCH 1/5] sched/fair: Fix select_idle_cpu()s cost accounting

2021-01-09 Thread Mel Gorman
CPU generation, cache topology, cache characteristics, domain utilisation, architecture, kitchen sink etc. -- Mel Gorman SUSE Labs

Re: [RFC][PATCH 1/5] sched/fair: Fix select_idle_cpu()s cost accounting

2021-01-09 Thread Mel Gorman
On Fri, Jan 08, 2021 at 09:21:48PM +0100, Peter Zijlstra wrote: > On Fri, Jan 08, 2021 at 10:27:38AM +0000, Mel Gorman wrote: > > > 1. avg_scan_cost is now based on the average scan cost of a rq but > >avg_idle is still scaled to the domain size. This is a bit problematic &

Re: [RFC][PATCH 1/5] sched/fair: Fix select_idle_cpu()s cost accounting

2021-01-08 Thread Mel Gorman
ner cases. If we do not want to weight the successful scan cost, then the compromise is to keep the old behaviour that accounts for scan failures only (avoids an sd_llc write at least) but base it on the estimated scan cost for a single rq. The fact that we do not account for scan failures should be explicitly commented so we do not forget it again in the future. -- Mel Gorman SUSE Labs

Re: [RFC][PATCH 1/5] sched/fair: Fix select_idle_cpu()s cost accounting

2021-01-08 Thread Mel Gorman
de-effects of the first patch and the last patch in the series introduced new problems of its own. Hence, I would like to limit the negative impact of the first patch and, if necessary, cut the last patch altogether. -- Mel Gorman SUSE Labs

Re: [RFC][PATCH 1/5] sched/fair: Fix select_idle_cpu()s cost accounting

2021-01-08 Thread Mel Gorman
On Fri, Jan 08, 2021 at 01:01:10PM +, Qais Yousef wrote: > On 01/08/21 10:27, Mel Gorman wrote: > > for_each_cpu_wrap(cpu, cpus, target) { > > - if (available_idle_cpu(cpu) || sched_idle_cpu(cpu)) > > + if (available_idle_cpu(cpu)

Re: [RFC][PATCH 1/5] sched/fair: Fix select_idle_cpu()s cost accounting

2021-01-08 Thread Mel Gorman
2; + break; + } - if (loops >= nr) { + if (++loops >= nr) { cpu = -1; break; } - loops++; } if (sched_feat(SIS_PROP)) { -- Mel Gorman SUSE Labs

Re: [RFC][PATCH 0/5] select_idle_sibling() wreckage

2021-01-04 Thread Mel Gorman
d issue might be responsible for the first issue, not sure. However, it does not suprise me that properly accounting would have an impact on the SMT depth search and likely needs tweaking. -- Mel Gorman SUSE Labs

Re: [RFC][PATCH 1/5] sched/fair: Fix select_idle_cpu()s cost accounting

2020-12-15 Thread Mel Gorman
out or change in a "safe" manner as all sorts of counter-intuitive effects occur. The series is queued and running and takes 1-2 days. I haven't reviewed the patches properly (holiday) but it'll be interesting to get some provisional data at least. -- Mel Gorman SUSE Labs

Re: [RFC PATCH v7] sched/fair: select idle cpu from idle cpumask for task wakeup

2020-12-14 Thread Mel Gorman
On Mon, Dec 14, 2020 at 10:18:16AM +0100, Vincent Guittot wrote: > On Fri, 11 Dec 2020 at 18:45, Peter Zijlstra wrote: > > > > On Thu, Dec 10, 2020 at 12:58:33PM +0000, Mel Gorman wrote: > > > The prequisite patch to make that approach work was rejected though > >

Re: [RFC PATCH v7] sched/fair: select idle cpu from idle cpumask for task wakeup

2020-12-14 Thread Mel Gorman
On Mon, Dec 14, 2020 at 10:31:22AM +0100, Peter Zijlstra wrote: > On Mon, Dec 14, 2020 at 09:11:29AM +0100, Vincent Guittot wrote: > > On Fri, 11 Dec 2020 at 23:50, Mel Gorman > > wrote: > > > > I originally did something like that on purpose too but Vincent called

Re: [RFC PATCH v7] sched/fair: select idle cpu from idle cpumask for task wakeup

2020-12-11 Thread Mel Gorman
On Fri, Dec 11, 2020 at 11:19:05PM +0100, Peter Zijlstra wrote: > On Fri, Dec 11, 2020 at 08:43:37PM +0000, Mel Gorman wrote: > > One bug is in __select_idle_core() though. It's scanning the SMT mask, > > not select_idle_mask so it can return an idle candidate that is not in

Re: [RFC PATCH v7] sched/fair: select idle cpu from idle cpumask for task wakeup

2020-12-11 Thread Mel Gorman
On Fri, Dec 11, 2020 at 06:44:42PM +0100, Peter Zijlstra wrote: > On Thu, Dec 10, 2020 at 12:58:33PM +0000, Mel Gorman wrote: > > The prequisite patch to make that approach work was rejected though > > as on its own, it's not very helpful and Vincent didn't like that the > &

Re: [PATCH 0/4] Reduce scanning of runqueues in select_idle_sibling

2020-12-11 Thread Mel Gorman
On Fri, Dec 11, 2020 at 10:51:17AM +0100, Vincent Guittot wrote: > On Thu, 10 Dec 2020 at 12:04, Mel Gorman wrote: > > > > On Thu, Dec 10, 2020 at 10:38:37AM +0100, Vincent Guittot wrote: > > > > while testing your patchset and Aubrey one on top of tip, I'm facing &

Re: [PATCH 3/4] sched/fair: Do not replace recent_used_cpu with the new target

2020-12-11 Thread Mel Gorman
On Fri, Dec 11, 2020 at 05:34:43PM +0800, Hillf Danton wrote: > On Fri, 11 Dec 2020 09:02:28 +0000 Mel Gorman wrote: > >On Fri, Dec 11, 2020 at 02:25:42PM +0800, Hillf Danton wrote: > >> On Tue, 8 Dec 2020 15:35:00 + Mel Gorman wrote: > >> > @@ -

[tip: sched/core] sched/fair: Clear SMT siblings after determining the core is not idle

2020-12-11 Thread tip-bot2 for Mel Gorman
The following commit has been merged into the sched/core branch of tip: Commit-ID: 13d5a5e9f9b8515da3c04305ae1bb03ab91be7a7 Gitweb: https://git.kernel.org/tip/13d5a5e9f9b8515da3c04305ae1bb03ab91be7a7 Author:Mel Gorman AuthorDate:Mon, 30 Nov 2020 14:40:20 Committer

Re: [PATCH 3/4] sched/fair: Do not replace recent_used_cpu with the new target

2020-12-11 Thread Mel Gorman
On Fri, Dec 11, 2020 at 02:25:42PM +0800, Hillf Danton wrote: > On Tue, 8 Dec 2020 15:35:00 +0000 Mel Gorman wrote: > > @@ -6277,17 +6277,13 @@ static int select_idle_sibling(struct task_struct > > *p, int prev, int target) > > > > /* Check a recently used CPU as

Re: [RFC PATCH v7] sched/fair: select idle cpu from idle cpumask for task wakeup

2020-12-10 Thread Mel Gorman
may break the latency sensitive > workload. > Also the short idle definition need more work. > Indeed, this is why I was advocating an approach that would use the idle CPU mask as a hint to quickly find idle CPUs but still fall through the scan the rest of the domain if necessary. That would cover the "latency sensitive" workload situation because a deeper scan still happens if necessary. While I'm not 100% certain, they were probably thinking about schbench which favours deep searches for idle CPUs and is latency sensitive. The prequisite patch to make that approach work was rejected though as on its own, it's not very helpful and Vincent didn't like that the load_balance_mask was abused to make it effective. -- Mel Gorman SUSE Labs

Re: [RFC PATCH v7] sched/fair: select idle cpu from idle cpumask for task wakeup

2020-12-10 Thread Mel Gorman
pipe where tasks will be entering/exiting rapidly is just 0.004 milliseconds on my desktop. > > > > I suspect the issue is that the mask is only marked busy from the tick > > context which is a very wide window. If select_idle_cpu() picks an idle > > CPU from the mask, it's still marked as idle in the mask. > > > That should be fine because we still check available_idle_cpu() and > sched_idle_cpu for the selected > CPU. And even if that CPU is marked as idle in the mask, that mask should not > be worse(bigger) than > the default sched_domain_span(sd). > I agree that it should never be worse/bigger but if the idle cpu mask is often the same bits as the sched_domain_span then it's not going to be any better either. -- Mel Gorman SUSE Labs

Re: [PATCH 0/4] Reduce scanning of runqueues in select_idle_sibling

2020-12-10 Thread Mel Gorman
n over-saturated. select_idle_sibling is a curse because it's very rare that a change to it is a universal win. On balance, I think it's better to avoid searching the domain at all where possible even if there are cases where searching can have a benefit such as finding an idle core instead of picking an idle CPU with a busy sibling via p->recent_used_cpu. -- Mel Gorman SUSE Labs

Re: [PATCH 2/4] sched/fair: Move avg_scan_cost calculations under SIS_PROP

2020-12-10 Thread Mel Gorman
e nr is throttled, the first > time = cpu_clock(this); > can be optimized. As nr is calculated and we already know which of the > weight of cpumask and nr is greater. > That is also outside the scope of this patch. To do that, cpumask_weight() would have to be calculated but it's likely to be a net loss. Even under light load, nr will be smaller than the domain weight incurring both the cost of cpumask_weight and the clock read in the common case. -- Mel Gorman SUSE Labs

Re: [PATCH 0/4] Reduce scanning of runqueues in select_idle_sibling

2020-12-09 Thread Mel Gorman
On Tue, Dec 08, 2020 at 03:34:57PM +, Mel Gorman wrote: > Changelog since v1 > o Drop single-pass patch > (vincent) > o Scope variables used for SIS_AVG_CPU > (dietmar) > o Remove re

Re: [RFC PATCH v7] sched/fair: select idle cpu from idle cpumask for task wakeup

2020-12-09 Thread Mel Gorman
" and there "Core Miss" is non-zero. I suspect the issue is that the mask is only marked busy from the tick context which is a very wide window. If select_idle_cpu() picks an idle CPU from the mask, it's still marked as idle in the mask. -- Mel Gorman SUSE Labs

Re: [PATCH 2/4] sched/fair: Move avg_scan_cost calculations under SIS_PROP

2020-12-09 Thread Mel Gorman
On Wed, Dec 09, 2020 at 07:07:11PM +0800, Li, Aubrey wrote: > On 2020/12/9 17:05, Mel Gorman wrote: > > On Wed, Dec 09, 2020 at 01:28:11PM +0800, Li, Aubrey wrote: > >>>> nr = div_u64(span_avg, avg_cost); > >>>> els

Re: [PATCH 2/4] sched/fair: Move avg_scan_cost calculations under SIS_PROP

2020-12-09 Thread Mel Gorman
t; nr is the key of this throttling mechanism, need to be placed under > sched_feat(SIS_PROP) as well. > It isn't necessary as nr in initialised to INT_MAX if !SIS_PROP. -- Mel Gorman SUSE Labs

Re: [PATCH 2/4] sched/fair: Move avg_scan_cost calculations under SIS_PROP

2020-12-08 Thread Mel Gorman
On Tue, Dec 08, 2020 at 05:03:21PM +0100, Vincent Guittot wrote: > On Tue, 8 Dec 2020 at 16:35, Mel Gorman wrote: > > > > As noted by Vincent Guittot, avg_scan_costs are calculated for SIS_PROP > > even if SIS_PROP is disabled. Move the time calculations under a SIS_PROP &

[PATCH 4/4] sched/fair: Return an idle cpu if one is found after a failed search for an idle core

2020-12-08 Thread Mel Gorman
for a full scan. This case is handled in the next patch. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 413d895bbbf8..ed6f45832d97 100644 --- a/kernel/sched/fair.c +++ b/kernel

[PATCH 3/4] sched/fair: Do not replace recent_used_cpu with the new target

2020-12-08 Thread Mel Gorman
r time, the graphs show that the sched domain is only scanned at negligible rates until the machine is fully busy. With low utilisation, the "Fast Success Rate" is almost 100% until the machine is fully busy. For 320 clients, the success rate is close to 0% which is unsurprising. Signed-off-by: Mel

[PATCH 1/4] sched/fair: Remove SIS_AVG_CPU

2020-12-08 Thread Mel Gorman
IS_AVG_CPU and focus on SIS_PROP as a throttling mechanism. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 20 +--- kernel/sched/features.h | 1 - 2 files changed, 9 insertions(+), 12 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 98075f

[PATCH 2/4] sched/fair: Move avg_scan_cost calculations under SIS_PROP

2020-12-08 Thread Mel Gorman
As noted by Vincent Guittot, avg_scan_costs are calculated for SIS_PROP even if SIS_PROP is disabled. Move the time calculations under a SIS_PROP check and while we are at it, exclude the cost of initialising the CPU mask from the average scan cost. Signed-off-by: Mel Gorman --- kernel/sched

[PATCH 0/4] Reduce scanning of runqueues in select_idle_sibling

2020-12-08 Thread Mel Gorman
ely uncontroversial Patch 4 returns an idle candidate if one is found while scanning for a free core. -- 2.26.2 Mel Gorman (4): sched/fair: Remove SIS_AVG_CPU sched/fair: Move avg_scan_cost calculations under SIS_PROP sched/fair: Do not replace recent_used_cpu with the new target sched/f

Re: [PATCH 1/4] sched/fair: Remove SIS_AVG_CPU

2020-12-08 Thread Mel Gorman
ROP As noted by Vincent Guittot, avg_scan_costs are calculated for SIS_PROP even if SIS_PROP is disabled. Move the time calculations under a SIS_PROP check and while we are at it, exclude the cost of initialising the CPU mask from the average scan cost. Signed-off-by: Mel Gorman --- kernel/sched/fair.c |

Re: [PATCH 1/4] sched/fair: Remove SIS_AVG_CPU

2020-12-08 Thread Mel Gorman
On Tue, Dec 08, 2020 at 02:43:10PM +0100, Vincent Guittot wrote: > On Tue, 8 Dec 2020 at 14:36, Mel Gorman wrote: > > > > On Tue, Dec 08, 2020 at 02:24:32PM +0100, Vincent Guittot wrote: > > > > > Nitpick: > > > > > > > > > > Since now

Re: [PATCH 1/4] sched/fair: Remove SIS_AVG_CPU

2020-12-08 Thread Mel Gorman
while we are at it, exclude the cost of initialising the CPU mask from the average scan cost. Signed-off-by: Mel Gorman diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 19ca0265f8aa..0fee53b1aae4 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6176,10 +6176,

Re: [PATCH 2/4] sched/fair: Do not replace recent_used_cpu with the new target

2020-12-08 Thread Mel Gorman
On Tue, Dec 08, 2020 at 10:57:29AM +0100, Dietmar Eggemann wrote: > On 07/12/2020 10:15, Mel Gorman wrote: > > After select_idle_sibling, p->recent_used_cpu is set to the > > new target. However on the next wakeup, prev will be the same as > > I'm confused here. Isn't curr

Re: [PATCH 1/4] sched/fair: Remove SIS_AVG_CPU

2020-12-08 Thread Mel Gorman
On Tue, Dec 08, 2020 at 11:07:19AM +0100, Dietmar Eggemann wrote: > On 07/12/2020 10:15, Mel Gorman wrote: > > SIS_AVG_CPU was introduced as a means of avoiding a search when the > > average search cost indicated that the search would likely fail. It > > was a blunt in

Re: [RFC PATCH v5] sched/fair: select idle cpu from idle cpumask for task wakeup

2020-12-07 Thread Mel Gorman
e looking at other runqueues without examining the same runqueue twice. While the patch would not be merged on its own, it may still be relevant in the context of an idle CPU mask depending on how up to date it is. -- Mel Gorman SUSE Labs

Re: [RFC PATCH 0/4] Reduce worst-case scanning of runqueues in select_idle_sibling

2020-12-07 Thread Mel Gorman
On Mon, Dec 07, 2020 at 04:04:41PM +0100, Vincent Guittot wrote: > On Mon, 7 Dec 2020 at 10:15, Mel Gorman wrote: > > > > This is a minimal series to reduce the amount of runqueue scanning in > > select_idle_sibling in the worst case. > > > > Patch 1 remove

[PATCH 4/4] sched/fair: Avoid revisiting CPUs multiple times during select_idle_sibling

2020-12-07 Thread Mel Gorman
() could be limited. Similarly, select_idle_core() often scans when no idle core is available as test_idle_cores is very race-prone and maybe there is a better approach to determining if select_idle_core() is likely to succeed or not. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 40

[PATCH 3/4] sched/fair: Return an idle cpu if one is found after a failed search for an idle core

2020-12-07 Thread Mel Gorman
for a full scan. This case is handled in the next patch. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 01b38fc17bca..00c3b526a5bd 100644 --- a/kernel/sched/fair.c +++ b/kernel

[RFC PATCH 0/4] Reduce worst-case scanning of runqueues in select_idle_sibling

2020-12-07 Thread Mel Gorman
This is a minimal series to reduce the amount of runqueue scanning in select_idle_sibling in the worst case. Patch 1 removes SIS_AVG_CPU because it's unused. Patch 2 improves the hit rate of p->recent_used_cpu to reduce the amount of scanning. It should be relatively uncontroversial

[PATCH 2/4] sched/fair: Do not replace recent_used_cpu with the new target

2020-12-07 Thread Mel Gorman
r time, the graphs show that the sched domain is only scanned at negligible rates until the machine is fully busy. With low utilisation, the "Fast Success Rate" is almost 100% until the machine is fully busy. For 320 clients, the success rate is close to 0% which is unsurprising. Signed-off-by: Mel

[PATCH 1/4] sched/fair: Remove SIS_AVG_CPU

2020-12-07 Thread Mel Gorman
IS_AVG_CPU and focus on SIS_PROP as a throttling mechanism. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 3 --- kernel/sched/features.h | 1 - 2 files changed, 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 98075f9ea9a8..23934dbac635 100644 --- a/kerne

Re: [PATCH 1/1] mm: compaction: avoid fast_isolate_around() to set pageblock_skip on reserved pages

2020-12-06 Thread Mel Gorman
On Sat, Dec 05, 2020 at 09:26:47PM -0500, Andrea Arcangeli wrote: > Hi Mel, > > On Thu, Nov 26, 2020 at 10:47:20AM +0000, Mel Gorman wrote: > > Agreed. This thread has a lot of different directions in it at this > > point so what I'd hope for is first, a patch that initialise

Re: [PATCH 06/10] sched/fair: Clear the target CPU from the cpumask of CPUs searched

2020-12-04 Thread Mel Gorman
On Fri, Dec 04, 2020 at 04:43:05PM +0100, Vincent Guittot wrote: > On Fri, 4 Dec 2020 at 16:40, Mel Gorman wrote: > > > > On Fri, Dec 04, 2020 at 04:23:48PM +0100, Vincent Guittot wrote: > > > On Fri, 4 Dec 2020 at 15:31, Mel Gorman > > > wrote: > > &g

Re: [PATCH 06/10] sched/fair: Clear the target CPU from the cpumask of CPUs searched

2020-12-04 Thread Mel Gorman
On Fri, Dec 04, 2020 at 04:23:48PM +0100, Vincent Guittot wrote: > On Fri, 4 Dec 2020 at 15:31, Mel Gorman wrote: > > > > On Fri, Dec 04, 2020 at 02:47:48PM +0100, Vincent Guittot wrote: > > > > IIUC, select_idle_core and select_idle_cpu share the same >

Re: [PATCH 06/10] sched/fair: Clear the target CPU from the cpumask of CPUs searched

2020-12-04 Thread Mel Gorman
, this would still be compatible with Aubrey's idle cpu mask as long as it's visited and cleared between select_idle_core and select_idle_cpu. It relaxes the contraints on Aubrey to some extent because the idle cpu mask would be a hint so if the information is out of date, an idle cpu may still be found the normal way. -- Mel Gorman SUSE Labs

Re: [PATCH 06/10] sched/fair: Clear the target CPU from the cpumask of CPUs searched

2020-12-04 Thread Mel Gorman
On Fri, Dec 04, 2020 at 02:17:20PM +0100, Vincent Guittot wrote: > On Fri, 4 Dec 2020 at 14:13, Vincent Guittot > wrote: > > > > On Fri, 4 Dec 2020 at 12:30, Mel Gorman wrote: > > > > > > On Fri, Dec 04, 2020 at 11:56:36AM +0100, Vincent Guittot wrote: >

Re: [PATCH 06/10] sched/fair: Clear the target CPU from the cpumask of CPUs searched

2020-12-04 Thread Mel Gorman
linear walk of the cpumask. Similarly, select_idle_cpu() is not guaranteed to scan the sibling first (ordering) or even reach the sibling (throttling). select_idle_smt() is a last-ditch effort. -- Mel Gorman SUSE Labs

Re: [PATCH 2/2] sched: Split the function show_schedstat()

2020-12-04 Thread Mel Gorman
On Fri, Dec 04, 2020 at 09:22:34AM +0800, Yunfeng Ye wrote: > > > On 2020/12/3 17:42, Mel Gorman wrote: > > On Thu, Dec 03, 2020 at 02:47:14PM +0800, Yunfeng Ye wrote: > >> The schedstat include runqueue-specific stats and domain-specific stats, > >> so split i

Re: [PATCH 06/10] sched/fair: Clear the target CPU from the cpumask of CPUs searched

2020-12-03 Thread Mel Gorman
On Thu, Dec 03, 2020 at 05:38:03PM +0100, Vincent Guittot wrote: > On Thu, 3 Dec 2020 at 15:11, Mel Gorman wrote: > > > > The target CPU is definitely not idle in both select_idle_core and > > select_idle_cpu. For select_idle_core(), the SMT is potentially > > checke

Re: [PATCH 04/10] sched/fair: Return an idle cpu if one is found after a failed search for an idle core

2020-12-03 Thread Mel Gorman
if (!available_idle_cpu(cpu)) { idle = false; - if (idle_candidate != -1) - break; + break; } if (idle_candidate == -1 && -- Mel Gorman SUSE Labs

[PATCH 10/10] sched/fair: Avoid revisiting CPUs multiple times during select_idle_sibling

2020-12-03 Thread Mel Gorman
. select_idle_smt() is left alone as the cost of checking one SMT sibling is marginal relative to calling __clear_cpumask_cpu() for evey CPU visited by select_idle_core(). Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 29 - 1 file changed, 16 insertions(+), 13 deletions

[PATCH 09/10] sched/fair: Limit the search for an idle core

2020-12-03 Thread Mel Gorman
by select_idle_core() is duplicated by select_idle_cpu() if an idle candidate is not found. If there is an idle CPU then aborting select_idle_core() can have a negative impact. This is addressed in the next patch. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 16 +--- 1 file

[PATCH 02/10] sched/fair: Track efficiency of task recent_used_cpu

2020-12-03 Thread Mel Gorman
off-by: Mel Gorman --- kernel/sched/debug.c | 2 ++ kernel/sched/fair.c | 23 +-- kernel/sched/sched.h | 2 ++ kernel/sched/stats.c | 7 --- 4 files changed, 21 insertions(+), 13 deletions(-) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 2386cc5e7

[PATCH 09/10] sched/fair: Limit the search for an idle core

2020-12-03 Thread Mel Gorman
by select_idle_core() is duplicated by select_idle_cpu() if an idle candidate is not found. If there is an idle CPU then aborting select_idle_core() can have a negative impact. This is addressed in the next patch. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 16 +--- 1 file

[PATCH 06/10] sched/fair: Clear the target CPU from the cpumask of CPUs searched

2020-12-03 Thread Mel Gorman
The target CPU is definitely not idle in both select_idle_core and select_idle_cpu. For select_idle_core(), the SMT is potentially checked unnecessarily as the core is definitely not idle if the target is busy. For select_idle_cpu(), the first CPU checked is simply a waste. Signed-off-by: Mel

[PATCH 05/10] sched/fair: Do not replace recent_used_cpu with the new target

2020-12-03 Thread Mel Gorman
only scanned at negligible rates until the machine is fully busy. With low utilisation, the "Fast Success Rate" is almost 100% until the machine is fully busy. For 320 clients, the success rate is close to 0% which is unsurprising. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 4 +--

[PATCH 08/10] sched/fair: Reintroduce SIS_AVG_CPU but in the context of SIS_PROP to reduce search depth

2020-12-03 Thread Mel Gorman
Subject says it all but no supporting data at this time. This might help the hackbench case in isolation or throw other workloads under the bus. Final version will have proper data. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 8 1 file changed, 8 insertions(+) diff --git

[PATCH 07/10] sched/fair: Account for the idle cpu/smt search cost

2020-12-03 Thread Mel Gorman
guided by the fact that avg_cost was not always accounted for. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 82 + 1 file changed, 46 insertions(+), 36 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 1d8f5c4b4936

[PATCH 04/10] sched/fair: Return an idle cpu if one is found after a failed search for an idle core

2020-12-03 Thread Mel Gorman
-by: Mel Gorman --- kernel/sched/fair.c | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index fc48cc99b03d..845bc0cd9158 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6066,6 +6066,7 @@ void __update_idle_core

[RFC PATCH 00/10] Reduce time complexity of select_idle_sibling

2020-12-03 Thread Mel Gorman
This is an early prototype that has not been tested heavily. While parts of it may stand on its own, the motivation to release early is Aubrey Li's series on using an idle cpumask to optimise the search and Barry Song's series on representing clusters on die. The series is based on tip/sched/core

[PATCH 03/10] sched/fair: Remove SIS_AVG_CPU

2020-12-03 Thread Mel Gorman
IS_AVG_CPU and focus on SIS_PROP as a throttling mechanism. Signed-off-by: Mel Gorman --- kernel/sched/fair.c | 3 --- kernel/sched/features.h | 1 - 2 files changed, 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d9acd55d309b..fc48cc99b03d 100644 --- a/kerne

[PATCH 01/10] sched/fair: Track efficiency of select_idle_sibling

2020-12-03 Thread Mel Gorman
target, prev or recent CPUs. SIS Success rate: Percentage of scans that found an idle CPU. Signed-off-by: Mel Gorman --- kernel/sched/debug.c | 4 kernel/sched/fair.c | 14 ++ kernel/sched/sched.h | 6 ++ kernel/sched/stats.c | 8 +--- 4 files changed, 29

Re: [PATCH -V6 RESEND 1/3] numa balancing: Migrate on fault among multiple bound nodes

2020-12-03 Thread Mel Gorman
On Thu, Dec 03, 2020 at 11:25:50AM +0100, Peter Zijlstra wrote: > On Wed, Dec 02, 2020 at 11:40:54AM +0000, Mel Gorman wrote: > > On Wed, Dec 02, 2020 at 04:42:32PM +0800, Huang Ying wrote: > > > Now, NUMA balancing can only optimize the page placement among the > > >

Re: [RFC PATCH v2 2/2] scheduler: add scheduler level for clusters

2020-12-03 Thread Mel Gorman
he idle cpumask in a sensible fashion. The main idea I had in mind was special casing EPYC as it has multiple small L3 caches that may benefit from select_idle_sibling looking slightly beyond the LLC as a search domain but it has not even started yet. -- Mel Gorman SUSE Labs

<    1   2   3   4   5   6   7   8   9   10   >