Re: power-efficient scheduling design

2013-06-24 Thread Arjan van de Ven
On 6/24/2013 2:59 PM, Benjamin Herrenschmidt wrote: On Mon, 2013-06-24 at 08:26 -0700, Arjan van de Ven wrote: to bring the system back up if all cores in the whole system are idle and power gated, memory in SR etc... is typically < 250 usec (depends on the exact version of the cpu etc). But t

Re: power-efficient scheduling design

2013-06-24 Thread Benjamin Herrenschmidt
On Mon, 2013-06-24 at 08:26 -0700, Arjan van de Ven wrote: > > to bring the system back up if all cores in the whole system are idle and > power gated, > memory in SR etc... is typically < 250 usec (depends on the exact version > of the cpu etc). But the moment even one core is running, that core

Re: power-efficient scheduling design

2013-06-24 Thread Arjan van de Ven
I guess it depends on the system Sort-of. We have something similar with threads on ppc. IE, the core can only really stop if all threads are. From a Linux persepctive it's a matter of how we define the scope of that 'cluster' Catalin is talking about. I'm sure you do too. Then there is the p

Re: power-efficient scheduling design

2013-06-24 Thread Catalin Marinas
On Mon, Jun 24, 2013 at 12:32:00AM +0100, Benjamin Herrenschmidt wrote: > On Fri, 2013-06-21 at 14:34 -0700, Arjan van de Ven wrote: > > On 6/21/2013 2:23 PM, Catalin Marinas wrote: > > >> > > >> oops sorry I misread your mail (lack of early coffee I suppose) > > >> > > >> I can see your point of h

Re: power-efficient scheduling design

2013-06-23 Thread Benjamin Herrenschmidt
On Fri, 2013-06-21 at 14:34 -0700, Arjan van de Ven wrote: > On 6/21/2013 2:23 PM, Catalin Marinas wrote: > >> > >> oops sorry I misread your mail (lack of early coffee I suppose) > >> > >> I can see your point of having a thing for "did we ask for all the > >> performance > >> we could ask for" p

Re: power-efficient scheduling design

2013-06-23 Thread Ingo Molnar
* Morten Rasmussen wrote: > On Tue, Jun 18, 2013 at 04:20:28PM +0100, Arjan van de Ven wrote: > > On 6/14/2013 9:05 AM, Morten Rasmussen wrote: > > > > > Looking at the discussion it seems that people have slightly different > > > views, but most agree that the goal is an integrated scheduling,

Re: power-efficient scheduling design

2013-06-21 Thread Arjan van de Ven
On 6/21/2013 2:23 PM, Catalin Marinas wrote: oops sorry I misread your mail (lack of early coffee I suppose) I can see your point of having a thing for "did we ask for all the performance we could ask for" prior to doing a load balance (although, for power efficiency, if you have two tasks that

Re: power-efficient scheduling design

2013-06-21 Thread Catalin Marinas
On 21 June 2013 16:38, Arjan van de Ven wrote: > On 6/21/2013 1:50 AM, Morten Rasmussen wrote: >> A hint when a task is moved to a new cpu is too late if the migration >> shouldn't have happened at all. If the scheduler knows that the cpu is >> able to switch to a higher p-state it can decide to w

Re: power-efficient scheduling design

2013-06-21 Thread Arjan van de Ven
On 6/21/2013 1:50 AM, Morten Rasmussen wrote: ypically. A hint when a task is moved to a new cpu is too late if the migration shouldn't have happened at all. If the scheduler knows that the cpu is able to switch to a higher p-state it can decide to wait for the p-state change instead of migratin

Re: power-efficient scheduling design

2013-06-21 Thread Arjan van de Ven
On 6/21/2013 1:50 AM, Morten Rasmussen wrote: in control of the p-state selection and changes it fast enough to match the current load, the scheduler doesn't have to care? By fast enough I mean, faster than the scheduler would notice if a cpu was temporarily overloaded at a low p-state. In that c

Re: power-efficient scheduling design

2013-06-21 Thread Morten Rasmussen
On Tue, Jun 18, 2013 at 04:20:28PM +0100, Arjan van de Ven wrote: > On 6/14/2013 9:05 AM, Morten Rasmussen wrote: > > > Looking at the discussion it seems that people have slightly different > > views, but most agree that the goal is an integrated scheduling, > > frequency, and idle policy like yo

Re: power-efficient scheduling design

2013-06-21 Thread Morten Rasmussen
On Wed, Jun 19, 2013 at 06:08:29PM +0100, Arjan van de Ven wrote: > On 6/19/2013 10:00 AM, Morten Rasmussen wrote: > > On Wed, Jun 19, 2013 at 04:39:39PM +0100, Arjan van de Ven wrote: > >> On 6/18/2013 10:47 AM, David Lang wrote: > >> > >>> > >>> It's bad enough trying to guess the needs of the pr

Re: power-efficient scheduling design

2013-06-20 Thread Ingo Molnar
* Morten Rasmussen wrote: > On Fri, May 31, 2013 at 11:52:04AM +0100, Ingo Molnar wrote: > > > > * Morten Rasmussen wrote: > > > > > Hi, > > > > > > A number of patch sets related to power-efficient scheduling have been > > > posted over the last couple of months. Most of them do not have m

Re: power-efficient scheduling design

2013-06-19 Thread Arjan van de Ven
On 6/19/2013 10:00 AM, Morten Rasmussen wrote: On Wed, Jun 19, 2013 at 04:39:39PM +0100, Arjan van de Ven wrote: On 6/18/2013 10:47 AM, David Lang wrote: It's bad enough trying to guess the needs of the processes, but if you also are reduced to guessing the capabilities of the cores, how can

Re: power-efficient scheduling design

2013-06-19 Thread Morten Rasmussen
On Wed, Jun 19, 2013 at 04:39:39PM +0100, Arjan van de Ven wrote: > On 6/18/2013 10:47 AM, David Lang wrote: > > > > > It's bad enough trying to guess the needs of the processes, but if you also > > are reduced to guessing the capabilities of the cores, how can anything be > > made to work? > >

Re: power-efficient scheduling design

2013-06-19 Thread Arjan van de Ven
On 6/18/2013 10:47 AM, David Lang wrote: It's bad enough trying to guess the needs of the processes, but if you also are reduced to guessing the capabilities of the cores, how can anything be made to work? btw one way to look at this is to assume that (with some minimal hinting) the CPU dri

Re: power-efficient scheduling design

2013-06-19 Thread Morten Rasmussen
On Tue, Jun 18, 2013 at 06:39:27PM +0100, David Lang wrote: > On Tue, 18 Jun 2013, Morten Rasmussen wrote: > > >> I don't think that you are passing nearly enough information around. > >> > >> A fairly simple example > >> > >> take a relatively modern 4-core system with turbo mode where speed cont

Re: power-efficient scheduling design

2013-06-18 Thread Arjan van de Ven
On 6/18/2013 10:47 AM, David Lang wrote: so this sounds to me like the process for changing settings on this Intel hardware is a two phase process something looks up what should be possible and says "switch to mode X" more a case of "I would like to request X" it's not a mandate, it's a pol

Re: power-efficient scheduling design

2013-06-18 Thread Catalin Marinas
On Tue, Jun 18, 2013 at 04:20:28PM +0100, Arjan van de Ven wrote: > On 6/14/2013 9:05 AM, Morten Rasmussen wrote: > > Looking at the discussion it seems that people have slightly different > > views, but most agree that the goal is an integrated scheduling, > > frequency, and idle policy like you p

Re: power-efficient scheduling design

2013-06-18 Thread David Lang
On Tue, 18 Jun 2013, Arjan van de Ven wrote: On 6/14/2013 9:05 AM, Morten Rasmussen wrote: Looking at the discussion it seems that people have slightly different views, but most agree that the goal is an integrated scheduling, frequency, and idle policy like you pointed out from the beginning.

Re: power-efficient scheduling design

2013-06-18 Thread David Lang
On Tue, 18 Jun 2013, Morten Rasmussen wrote: I don't think that you are passing nearly enough information around. A fairly simple example take a relatively modern 4-core system with turbo mode where speed controls affect two cores at a time (I don't know the details of the available CPUs to kn

Re: power-efficient scheduling design

2013-06-18 Thread Arjan van de Ven
On 6/14/2013 9:05 AM, Morten Rasmussen wrote: Looking at the discussion it seems that people have slightly different views, but most agree that the goal is an integrated scheduling, frequency, and idle policy like you pointed out from the beginning. ... except that such a solution does not re

Re: power-efficient scheduling design

2013-06-18 Thread Morten Rasmussen
On Tue, Jun 18, 2013 at 02:37:21AM +0100, David Lang wrote: > > On Fri, 14 Jun 2013, Morten Rasmussen wrote: > > > Looking at the discussion it seems that people have slightly different > > views, but most agree that the goal is an integrated scheduling, > > frequency, and idle policy like you po

Re: power-efficient scheduling design

2013-06-17 Thread David Lang
On Fri, 14 Jun 2013, Morten Rasmussen wrote: Looking at the discussion it seems that people have slightly different views, but most agree that the goal is an integrated scheduling, frequency, and idle policy like you pointed out from the beginning. What is less clear is how such design would l

Re: power-efficient scheduling design

2013-06-17 Thread Catalin Marinas
On Fri, Jun 14, 2013 at 05:05:22PM +0100, Morten Rasmussen wrote: > The intention is that the power scheduler will implement the (unified) > power policy. It gets the current load of the system from the scheduler. > Based on this information it will adjust the compute capacity available > to the sc

Re: power-efficient scheduling design

2013-06-14 Thread Morten Rasmussen
Hi, On Fri, May 31, 2013 at 11:52:04AM +0100, Ingo Molnar wrote: > > * Morten Rasmussen wrote: > > > Hi, > > > > A number of patch sets related to power-efficient scheduling have been > > posted over the last couple of months. Most of them do not have much > > data to back them up, so I deci

Re: power-efficient scheduling design

2013-06-12 Thread Preeti U Murthy
Hi, On 06/11/2013 06:20 AM, Rafael J. Wysocki wrote: > > OK, so let's try to take one step more and think about what part should belong > to the scheduler and what part should be taken care of by the "idle" driver. > > Do you have any specific view on that? I gave it some thought and went throu

Re: power-efficient scheduling design

2013-06-12 Thread David Lang
On Wed, 12 Jun 2013, Daniel Lezcano wrote: On Mon, 10 Jun 2013, Daniel Lezcano wrote: Some SoC can have a cluster of cpus sharing some resources, eg cache, so they must enter the same state at the same moment. Beside the synchronization mechanisms, that adds a dependency with the next event. F

Re: power-efficient scheduling design

2013-06-12 Thread David Lang
On Wed, 12 Jun 2013, Amit Kucheria wrote: On Wed, Jun 12, 2013 at 7:18 AM, Arjan van de Ven wrote: On 6/11/2013 5:27 PM, David Lang wrote: Nobody is saying that this sort of thing should be in the fastpath of the scheduler. But if the scheduler has a table that tells it the possible states

Re: power-efficient scheduling design

2013-06-12 Thread Catalin Marinas
On Wed, Jun 12, 2013 at 04:24:52PM +0100, Arjan van de Ven wrote: > >>> This isn't in the fastpath, it's in the rebalancing logic. > >> > >> the reality is much more complex unfortunately. > >> C and P states hang together tightly, and even C state on one core > >> impacts other cores' performance,

Re: power-efficient scheduling design

2013-06-12 Thread Arjan van de Ven
This isn't in the fastpath, it's in the rebalancing logic. the reality is much more complex unfortunately. C and P states hang together tightly, and even C state on one core impacts other cores' performance, just like P state selection on one core impacts other cores. (at least for x86, we shou

Re: power-efficient scheduling design

2013-06-12 Thread Catalin Marinas
Hi Arjan, On Wed, Jun 12, 2013 at 02:48:58AM +0100, Arjan van de Ven wrote: > On 6/11/2013 5:27 PM, David Lang wrote: > > Nobody is saying that this sort of thing should be in the fastpath > > of the scheduler. > > > > But if the scheduler has a table that tells it the possible states, > > and the

Re: power-efficient scheduling design

2013-06-12 Thread Amit Kucheria
On Wed, Jun 12, 2013 at 7:18 AM, Arjan van de Ven wrote: > On 6/11/2013 5:27 PM, David Lang wrote: >> >> >> Nobody is saying that this sort of thing should be in the fastpath of the >> scheduler. >> >> But if the scheduler has a table that tells it the possible states, and >> the cost to get from

Re: power-efficient scheduling design

2013-06-12 Thread Daniel Lezcano
On 06/12/2013 02:27 AM, David Lang wrote: > On Mon, 10 Jun 2013, Daniel Lezcano wrote: > >> Some SoC can have a cluster of cpus sharing some resources, eg cache, so >> they must enter the same state at the same moment. Beside the >> synchronization mechanisms, that adds a dependency with the next

Re: power-efficient scheduling design

2013-06-11 Thread Arjan van de Ven
On 6/11/2013 5:27 PM, David Lang wrote: Nobody is saying that this sort of thing should be in the fastpath of the scheduler. But if the scheduler has a table that tells it the possible states, and the cost to get from the current state to each of these states (and to get back and/or wake up

Re: power-efficient scheduling design

2013-06-11 Thread David Lang
On Mon, 10 Jun 2013, Daniel Lezcano wrote: Some SoC can have a cluster of cpus sharing some resources, eg cache, so they must enter the same state at the same moment. Beside the synchronization mechanisms, that adds a dependency with the next event. For example, the u8500 board has a couple of c

Re: power-efficient scheduling design

2013-06-10 Thread Rafael J. Wysocki
On Sunday, June 09, 2013 09:12:18 AM Preeti U Murthy wrote: > Hi Rafael, Hi Preeti, > On 06/08/2013 07:32 PM, Rafael J. Wysocki wrote: > > On Saturday, June 08, 2013 12:28:04 PM Catalin Marinas wrote: > >> On Fri, Jun 07, 2013 at 07:08:47PM +0100, Preeti U Murthy wrote: [...] > >> The scheduler

Re: power-efficient scheduling design

2013-06-10 Thread Daniel Lezcano
On 06/09/2013 05:42 AM, Preeti U Murthy wrote: > Hi Rafael, > > On 06/08/2013 07:32 PM, Rafael J. Wysocki wrote: >> On Saturday, June 08, 2013 12:28:04 PM Catalin Marinas wrote: >>> On Fri, Jun 07, 2013 at 07:08:47PM +0100, Preeti U Murthy wrote: On 06/07/2013 08:21 PM, Catalin Marinas wrote:

Re: power-efficient scheduling design

2013-06-09 Thread Catalin Marinas
Hi Preeti, (trimming lots of text, hopefully to make it easier to follow) On Sun, Jun 09, 2013 at 04:42:18AM +0100, Preeti U Murthy wrote: > On 06/08/2013 07:32 PM, Rafael J. Wysocki wrote: > > On Saturday, June 08, 2013 12:28:04 PM Catalin Marinas wrote: > >> On Fri, Jun 07, 2013 at 07:08:47PM +

Re: power-efficient scheduling design

2013-06-08 Thread Preeti U Murthy
Hi David, On 06/07/2013 11:06 PM, David Lang wrote: > On Fri, 7 Jun 2013, Preeti U Murthy wrote: > >> Hi Catalin, >> >> On 06/07/2013 08:21 PM, Catalin Marinas wrote: > >>> Take the cpuidle example, it uses the load average of the CPUs, >>> however this load average is currently controlled by th

Re: power-efficient scheduling design

2013-06-08 Thread Preeti U Murthy
Hi Catalin, On 06/08/2013 04:58 PM, Catalin Marinas wrote: > On Fri, Jun 07, 2013 at 07:08:47PM +0100, Preeti U Murthy wrote: >> On 06/07/2013 08:21 PM, Catalin Marinas wrote: >>> I think you are missing Ingo's point. It's not about the scheduler >>> complying with decisions made by various govern

Re: power-efficient scheduling design

2013-06-08 Thread Preeti U Murthy
Hi Rafael, On 06/08/2013 07:32 PM, Rafael J. Wysocki wrote: > On Saturday, June 08, 2013 12:28:04 PM Catalin Marinas wrote: >> On Fri, Jun 07, 2013 at 07:08:47PM +0100, Preeti U Murthy wrote: >>> On 06/07/2013 08:21 PM, Catalin Marinas wrote: I think you are missing Ingo's point. It's not abo

Re: power-efficient scheduling design

2013-06-08 Thread Rafael J. Wysocki
On Saturday, June 08, 2013 12:28:04 PM Catalin Marinas wrote: > On Fri, Jun 07, 2013 at 07:08:47PM +0100, Preeti U Murthy wrote: > > On 06/07/2013 08:21 PM, Catalin Marinas wrote: > > > I think you are missing Ingo's point. It's not about the scheduler > > > complying with decisions made by various

Re: power-efficient scheduling design

2013-06-08 Thread Catalin Marinas
On Fri, Jun 07, 2013 at 07:08:47PM +0100, Preeti U Murthy wrote: > On 06/07/2013 08:21 PM, Catalin Marinas wrote: > > I think you are missing Ingo's point. It's not about the scheduler > > complying with decisions made by various governors in the kernel > > (which may or may not have enough informa

Re: power-efficient scheduling design

2013-06-07 Thread David Lang
On Fri, 7 Jun 2013, Preeti U Murthy wrote: Hi Catalin, On 06/07/2013 08:21 PM, Catalin Marinas wrote: Take the cpuidle example, it uses the load average of the CPUs, however this load average is currently controlled by the scheduler (load balance). Rather than using a load average that degra

Re: power-efficient scheduling design

2013-06-07 Thread Preeti U Murthy
Hi Catalin, On 06/07/2013 08:21 PM, Catalin Marinas wrote: > I think you are missing Ingo's point. It's not about the scheduler > complying with decisions made by various governors in the kernel > (which may or may not have enough information) but rather the > scheduler being in a better position

Re: power-efficient scheduling design

2013-06-07 Thread Arjan van de Ven
On 6/6/2013 11:03 PM, Preeti U Murthy wrote: Hi, On 05/31/2013 04:22 PM, Ingo Molnar wrote: PeterZ and me tried to point out the design requirements previously, but it still does not appear to be clear enough to people, so let me spell it out again, in a hopefully clearer fashion. The schedule

Re: power-efficient scheduling design

2013-06-07 Thread Catalin Marinas
Hi Preeti, On 7 June 2013 07:03, Preeti U Murthy wrote: > On 05/31/2013 04:22 PM, Ingo Molnar wrote: >> PeterZ and me tried to point out the design requirements previously, but >> it still does not appear to be clear enough to people, so let me spell it >> out again, in a hopefully clearer fashio

Re: power-efficient scheduling design

2013-06-06 Thread Preeti U Murthy
Hi Morten, I have one point to make below. On 06/04/2013 08:33 PM, Morten Rasmussen wrote: > > Thanks for sharing your view. > > I agree with idea of having a high level user switch to change > power/performance policy trade-offs for the system. Not only for > scheduling. I also share your view

Re: power-efficient scheduling design

2013-06-06 Thread Preeti U Murthy
Hi, On 05/31/2013 04:22 PM, Ingo Molnar wrote: > PeterZ and me tried to point out the design requirements previously, but > it still does not appear to be clear enough to people, so let me spell it > out again, in a hopefully clearer fashion. > > The scheduler has valuable power saving informat

Re: power-efficient scheduling design

2013-06-05 Thread Amit Kucheria
On Fri, May 31, 2013 at 4:22 PM, Ingo Molnar wrote: > > * Morten Rasmussen wrote: > >> Hi, >> >> A number of patch sets related to power-efficient scheduling have been >> posted over the last couple of months. Most of them do not have much >> data to back them up, so I decided to do some testing.

Re: power-efficient scheduling design

2013-06-04 Thread Morten Rasmussen
On Fri, May 31, 2013 at 11:52:04AM +0100, Ingo Molnar wrote: > > * Morten Rasmussen wrote: > > > Hi, > > > > A number of patch sets related to power-efficient scheduling have been > > posted over the last couple of months. Most of them do not have much > > data to back them up, so I decided t

Re: power-efficient scheduling design

2013-06-03 Thread Ingo Molnar
* Arjan van de Ven wrote: > > > > - enumeration of idle states > > > > - how long it takes to enter+exit a particular idle state > > > > - [ perhaps information about how destructive to CPU caches that > > particular idle state is. ] > > > > - new driver entry point that allows the sche

Re: power-efficient scheduling design

2013-06-03 Thread Arjan van de Ven
- enumeration of idle states - how long it takes to enter+exit a particular idle state - [ perhaps information about how destructive to CPU caches that particular idle state is. ] - new driver entry point that allows the scheduler to enter any of the enumerated idle states. P

power-efficient scheduling design

2013-05-31 Thread Ingo Molnar
* Morten Rasmussen wrote: > Hi, > > A number of patch sets related to power-efficient scheduling have been > posted over the last couple of months. Most of them do not have much > data to back them up, so I decided to do some testing. Thanks, numbers are always welcome! > Measurement techni