Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-04-01 Thread Preeti U Murthy
Hi Morten, On 04/01/2015 06:33 PM, Morten Rasmussen wrote: >> Alright I see. But it is one additional wake up. And the wake up will be >> within the cluster. We will not wake up any CPU in the neighboring >> cluster unless there are tasks to be pulled. So, we can wake up a core >> out of a deep id

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-04-01 Thread Jason Low
On Wed, 2015-04-01 at 14:03 +0100, Morten Rasmussen wrote: Hi Morten, > > Alright I see. But it is one additional wake up. And the wake up will be > > within the cluster. We will not wake up any CPU in the neighboring > > cluster unless there are tasks to be pulled. So, we can wake up a core > >

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-04-01 Thread Morten Rasmussen
Hi Preeti and Jason, On Wed, Apr 01, 2015 at 07:28:03AM +0100, Preeti U Murthy wrote: > On 03/31/2015 11:00 PM, Jason Low wrote: > > On Tue, 2015-03-31 at 14:28 +0530, Preeti U Murthy wrote: > > > >> Morten, > > > >> I am a bit confused about the problem you are pointing to. > > > >> I am unabl

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-31 Thread Preeti U Murthy
On 03/31/2015 11:00 PM, Jason Low wrote: > On Tue, 2015-03-31 at 14:28 +0530, Preeti U Murthy wrote: > >> Morten, > >> I am a bit confused about the problem you are pointing to. > >> I am unable to see the issue. What is it that I am missing ? > > Hi Preeti, > > Here is one of the potential is

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-31 Thread Jason Low
On Tue, 2015-03-31 at 14:28 +0530, Preeti U Murthy wrote: > Morten, > I am a bit confused about the problem you are pointing to. > I am unable to see the issue. What is it that I am missing ? Hi Preeti, Here is one of the potential issues that have been described from my understanding. In sit

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-31 Thread Preeti U Murthy
On 03/30/2015 05:33 PM, Morten Rasmussen wrote: > On Mon, Mar 30, 2015 at 12:06:32PM +0100, Peter Zijlstra wrote: >> On Fri, Mar 27, 2015 at 05:56:51PM +, Morten Rasmussen wrote: >> >>> I agree that it is hard to predict how many additional cpus you need, >>> but I don't think you necessarily n

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-31 Thread Preeti U Murthy
On 03/30/2015 07:15 PM, Vincent Guittot wrote: > On 26 March 2015 at 14:02, Preeti U Murthy wrote: >> When a CPU is kicked to do nohz idle balancing, it wakes up to do load >> balancing on itself, followed by load balancing on behalf of idle CPUs. >> But it may end up with load after the load bala

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Morten Rasmussen
On Mon, Mar 30, 2015 at 02:29:09PM +0100, Vincent Guittot wrote: > On 30 March 2015 at 14:24, Peter Zijlstra wrote: > > On Mon, Mar 30, 2015 at 01:03:03PM +0100, Morten Rasmussen wrote: > >> On Mon, Mar 30, 2015 at 12:06:32PM +0100, Peter Zijlstra wrote: > >> > On Fri, Mar 27, 2015 at 05:56:51PM +

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Peter Zijlstra
On Mon, Mar 30, 2015 at 03:29:09PM +0200, Vincent Guittot wrote: > On 30 March 2015 at 14:24, Peter Zijlstra wrote: > > @@ -7647,6 +7648,8 @@ static void nohz_idle_balance(struct rq *this_rq, > > enum cpu_idle_type idle) > > break; > > > > rq = cpu_rq(balan

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Vincent Guittot
On 26 March 2015 at 14:02, Preeti U Murthy wrote: > When a CPU is kicked to do nohz idle balancing, it wakes up to do load > balancing on itself, followed by load balancing on behalf of idle CPUs. > But it may end up with load after the load balancing attempt on itself. > This aborts nohz idle bal

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Vincent Guittot
On 30 March 2015 at 14:24, Peter Zijlstra wrote: > On Mon, Mar 30, 2015 at 01:03:03PM +0100, Morten Rasmussen wrote: >> On Mon, Mar 30, 2015 at 12:06:32PM +0100, Peter Zijlstra wrote: >> > On Fri, Mar 27, 2015 at 05:56:51PM +, Morten Rasmussen wrote: >> > >> > > I agree that it is hard to pred

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Peter Zijlstra
On Mon, Mar 30, 2015 at 02:24:49PM +0200, Peter Zijlstra wrote: > @@ -7647,6 +7648,8 @@ static void nohz_idle_balance(struct rq *this_rq, enum > cpu_idle_type idle) > break; > > rq = cpu_rq(balance_cpu); > + if (rq == this_rq) > +

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Peter Zijlstra
On Mon, Mar 30, 2015 at 01:03:03PM +0100, Morten Rasmussen wrote: > On Mon, Mar 30, 2015 at 12:06:32PM +0100, Peter Zijlstra wrote: > > On Fri, Mar 27, 2015 at 05:56:51PM +, Morten Rasmussen wrote: > > > > > I agree that it is hard to predict how many additional cpus you need, > > > but I don'

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Morten Rasmussen
On Mon, Mar 30, 2015 at 12:06:32PM +0100, Peter Zijlstra wrote: > On Fri, Mar 27, 2015 at 05:56:51PM +, Morten Rasmussen wrote: > > > I agree that it is hard to predict how many additional cpus you need, > > but I don't think you necessarily need that information as long as you > > start by fi

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Morten Rasmussen
On Mon, Mar 30, 2015 at 08:26:19AM +0100, Preeti U Murthy wrote: > Hi Morten, > > On 03/27/2015 11:26 PM, Morten Rasmussen wrote: > > > > I agree that the current behaviour is undesirable and should be fixed, > > but IMHO waking up all idle cpus can not be justified. It is only one > > additional

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Peter Zijlstra
On Fri, Mar 27, 2015 at 05:56:51PM +, Morten Rasmussen wrote: > I agree that it is hard to predict how many additional cpus you need, > but I don't think you necessarily need that information as long as you > start by filling up the cpu that was kicked to do the > nohz_idle_balance() first. >

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-30 Thread Preeti U Murthy
Hi Morten, On 03/27/2015 11:26 PM, Morten Rasmussen wrote: > > I agree that the current behaviour is undesirable and should be fixed, > but IMHO waking up all idle cpus can not be justified. It is only one > additional cpu though with your patch so it isn't quite that bad. > > I agree that it is

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-27 Thread Morten Rasmussen
On Fri, Mar 27, 2015 at 04:46:30PM +, Preeti U Murthy wrote: > Hi Morten, > > On 03/27/2015 08:08 PM, Morten Rasmussen wrote: > > Hi Preeti, > > > > On Thu, Mar 26, 2015 at 01:02:44PM +, Preeti U Murthy wrote: > >> Fix this, by checking if a CPU was woken up to do nohz idle load > >> bala

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-27 Thread Preeti U Murthy
Hi Morten, On 03/27/2015 08:08 PM, Morten Rasmussen wrote: > Hi Preeti, > > On Thu, Mar 26, 2015 at 01:02:44PM +, Preeti U Murthy wrote: >> Fix this, by checking if a CPU was woken up to do nohz idle load >> balancing, before it does load balancing upon itself. This way we allow >> idle CPUs

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-27 Thread Preeti U Murthy
Hi Wanpeng, Jason, On 03/27/2015 10:37 AM, Jason Low wrote: > On Fri, 2015-03-27 at 10:12 +0800, Wanpeng Li wrote: >> Hi Preeti, >> On Thu, Mar 26, 2015 at 06:32:44PM +0530, Preeti U Murthy wrote: >>> >>> 1. An ILB CPU was chosen from the first numa domain to trigger nohz idle >>> load balancing [

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-27 Thread Morten Rasmussen
Hi Preeti, On Thu, Mar 26, 2015 at 01:02:44PM +, Preeti U Murthy wrote: > Fix this, by checking if a CPU was woken up to do nohz idle load > balancing, before it does load balancing upon itself. This way we allow > idle CPUs across the system to do load balancing which results in > quicker spr

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-27 Thread Srikar Dronamraju
> When a CPU is kicked to do nohz idle balancing, it wakes up to do load > balancing on itself, followed by load balancing on behalf of idle CPUs. > But it may end up with load after the load balancing attempt on itself. > This aborts nohz idle balancing. As a result several idle CPUs are left > wi

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-27 Thread Wanpeng Li
Hi Srikar, On Fri, Mar 27, 2015 at 11:09:07AM +0530, Srikar Dronamraju wrote: > >Yes, the need_resched() in nohz_idle_balance() would exit the >nohz_idle_balance if it has something to run. However I wonder if we >should move the need_resched check out of the for loop. i.e the >need_resched check s

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-27 Thread Wanpeng Li
On Thu, Mar 26, 2015 at 10:07:21PM -0700, Jason Low wrote: >On Fri, 2015-03-27 at 10:12 +0800, Wanpeng Li wrote: >> Hi Preeti, >> On Thu, Mar 26, 2015 at 06:32:44PM +0530, Preeti U Murthy wrote: >> > >> >1. An ILB CPU was chosen from the first numa domain to trigger nohz idle >> >load balancing [Gi

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-26 Thread Srikar Dronamraju
* Jason Low [2015-03-26 22:07:21]: > On Fri, 2015-03-27 at 10:12 +0800, Wanpeng Li wrote: > > Hi Preeti, > > On Thu, Mar 26, 2015 at 06:32:44PM +0530, Preeti U Murthy wrote: > > > > > >1. An ILB CPU was chosen from the first numa domain to trigger nohz idle > > >load balancing [Given the experime

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-26 Thread Jason Low
On Fri, 2015-03-27 at 10:12 +0800, Wanpeng Li wrote: > Hi Preeti, > On Thu, Mar 26, 2015 at 06:32:44PM +0530, Preeti U Murthy wrote: > > > >1. An ILB CPU was chosen from the first numa domain to trigger nohz idle > >load balancing [Given the experiment, upto 6 CPUs per core could be > >potentially

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-26 Thread Jason Low
On Fri, 2015-03-27 at 10:03 +0530, Preeti U Murthy wrote: > Hi Wanpeng > > On 03/27/2015 07:42 AM, Wanpeng Li wrote: > > Hi Preeti, > > On Thu, Mar 26, 2015 at 06:32:44PM +0530, Preeti U Murthy wrote: > >> > >> 1. An ILB CPU was chosen from the first numa domain to trigger nohz idle > >> load bala

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-26 Thread Wanpeng Li
Hi Preeti, On Fri, Mar 27, 2015 at 10:03:21AM +0530, Preeti U Murthy wrote: >is set to CPU_NOT_IDLE. > >"" >idle = idle_cpu(cpu) ? CPU_IDLE : CPU_NOT_IDLE; > >And, > >When nohz_idle_balance() is called, the state of idle of ILB CPU is >checked before proceeding with load balancing on idle CPUs. > >

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-26 Thread Preeti U Murthy
Hi Wanpeng On 03/27/2015 07:42 AM, Wanpeng Li wrote: > Hi Preeti, > On Thu, Mar 26, 2015 at 06:32:44PM +0530, Preeti U Murthy wrote: >> >> 1. An ILB CPU was chosen from the first numa domain to trigger nohz idle >> load balancing [Given the experiment, upto 6 CPUs per core could be >> potentially

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-26 Thread Wanpeng Li
Hi Preeti, On Thu, Mar 26, 2015 at 06:32:44PM +0530, Preeti U Murthy wrote: > >1. An ILB CPU was chosen from the first numa domain to trigger nohz idle >load balancing [Given the experiment, upto 6 CPUs per core could be >potentially idle in this domain.] > >2. However the ILB CPU would call load_b

Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-26 Thread Jason Low
On Thu, 2015-03-26 at 18:32 +0530, Preeti U Murthy wrote: > kernel/sched/fair.c |8 +--- > 1 file changed, 5 insertions(+), 3 deletions(-) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index bcfe320..8b6d0d5 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @

[PATCH V2] sched: Improve load balancing in the presence of idle CPUs

2015-03-26 Thread Preeti U Murthy
When a CPU is kicked to do nohz idle balancing, it wakes up to do load balancing on itself, followed by load balancing on behalf of idle CPUs. But it may end up with load after the load balancing attempt on itself. This aborts nohz idle balancing. As a result several idle CPUs are left without task