Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-13 Thread Mike Galbraith
On Tue, 2014-05-13 at 10:08 -0400, Rik van Riel wrote: > OK, after doing some other NUMA stuff, and then looking at the scheduler > again with a fresh mind, I have drawn some more conclusions about what > the scheduler does, and how it breaks NUMA locality :) > > 1) If the node_distance between

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-13 Thread Rik van Riel
On 05/09/2014 11:54 PM, Mike Galbraith wrote: > On Fri, 2014-05-09 at 14:16 -0400, Rik van Riel wrote: > >> That leaves the big question: do we want to fall back to >> prev_cpu if it is not idle, and it has an idle sibling, >> or would it be better to find an idle sibling of prev_cpu >> when we

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-13 Thread Rik van Riel
On 05/09/2014 11:54 PM, Mike Galbraith wrote: On Fri, 2014-05-09 at 14:16 -0400, Rik van Riel wrote: That leaves the big question: do we want to fall back to prev_cpu if it is not idle, and it has an idle sibling, or would it be better to find an idle sibling of prev_cpu when we wake up a

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-13 Thread Mike Galbraith
On Tue, 2014-05-13 at 10:08 -0400, Rik van Riel wrote: OK, after doing some other NUMA stuff, and then looking at the scheduler again with a fresh mind, I have drawn some more conclusions about what the scheduler does, and how it breaks NUMA locality :) 1) If the node_distance between nodes

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Mike Galbraith
On Fri, 2014-05-09 at 14:16 -0400, Rik van Riel wrote: > That leaves the big question: do we want to fall back to > prev_cpu if it is not idle, and it has an idle sibling, > or would it be better to find an idle sibling of prev_cpu > when we wake up a task? Yes. If there was A correct answer,

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Rik van Riel
On 05/09/2014 01:55 PM, Mike Galbraith wrote: On Fri, 2014-05-09 at 11:24 -0400, Rik van Riel wrote: On 05/09/2014 11:24 AM, Mike Galbraith wrote: If no ->flags & SD_BALANCE_WAKE is encountered during traversal, sd remains NULL, we fall through to return prev_cpu. We do

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Mike Galbraith
On Fri, 2014-05-09 at 11:24 -0400, Rik van Riel wrote: > On 05/09/2014 11:24 AM, Mike Galbraith wrote: > > If no ->flags & SD_BALANCE_WAKE is encountered during traversal, sd > > remains NULL, we fall through to return prev_cpu. > We do fall through, but into this loop: >

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Rik van Riel
On 05/09/2014 11:24 AM, Mike Galbraith wrote: On Fri, 2014-05-09 at 10:22 -0400, Rik van Riel wrote: On 05/09/2014 03:34 AM, Mike Galbraith wrote: On Fri, 2014-05-09 at 01:27 -0400, Rik van Riel wrote: On Thu, 08 May 2014 22:20:25 -0400 Rik van Riel wrote: Looks like SD_BALANCE_WAKE is not

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Mike Galbraith
On Fri, 2014-05-09 at 10:22 -0400, Rik van Riel wrote: > On 05/09/2014 03:34 AM, Mike Galbraith wrote: > > On Fri, 2014-05-09 at 01:27 -0400, Rik van Riel wrote: > >> On Thu, 08 May 2014 22:20:25 -0400 > >> Rik van Riel wrote: > >> > >>> Looks like SD_BALANCE_WAKE is not gotten from the sd

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Rik van Riel
On 05/09/2014 03:34 AM, Mike Galbraith wrote: > On Fri, 2014-05-09 at 01:27 -0400, Rik van Riel wrote: >> On Thu, 08 May 2014 22:20:25 -0400 >> Rik van Riel wrote: >> >>> Looks like SD_BALANCE_WAKE is not gotten from the sd flags at >>> all, but passed into select_task_rq by try_to_wake_up, as a

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Mike Galbraith
On Fri, 2014-05-09 at 01:27 -0400, Rik van Riel wrote: > On Thu, 08 May 2014 22:20:25 -0400 > Rik van Riel wrote: > > > Looks like SD_BALANCE_WAKE is not gotten from the sd flags at > > all, but passed into select_task_rq by try_to_wake_up, as a > > hard coded sd_flags argument. > > > Should

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Mike Galbraith
On Fri, 2014-05-09 at 01:27 -0400, Rik van Riel wrote: On Thu, 08 May 2014 22:20:25 -0400 Rik van Riel r...@redhat.com wrote: Looks like SD_BALANCE_WAKE is not gotten from the sd flags at all, but passed into select_task_rq by try_to_wake_up, as a hard coded sd_flags argument.

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Rik van Riel
On 05/09/2014 03:34 AM, Mike Galbraith wrote: On Fri, 2014-05-09 at 01:27 -0400, Rik van Riel wrote: On Thu, 08 May 2014 22:20:25 -0400 Rik van Riel r...@redhat.com wrote: Looks like SD_BALANCE_WAKE is not gotten from the sd flags at all, but passed into select_task_rq by try_to_wake_up, as

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Mike Galbraith
On Fri, 2014-05-09 at 10:22 -0400, Rik van Riel wrote: On 05/09/2014 03:34 AM, Mike Galbraith wrote: On Fri, 2014-05-09 at 01:27 -0400, Rik van Riel wrote: On Thu, 08 May 2014 22:20:25 -0400 Rik van Riel r...@redhat.com wrote: Looks like SD_BALANCE_WAKE is not gotten from the sd flags

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Rik van Riel
On 05/09/2014 11:24 AM, Mike Galbraith wrote: On Fri, 2014-05-09 at 10:22 -0400, Rik van Riel wrote: On 05/09/2014 03:34 AM, Mike Galbraith wrote: On Fri, 2014-05-09 at 01:27 -0400, Rik van Riel wrote: On Thu, 08 May 2014 22:20:25 -0400 Rik van Riel r...@redhat.com wrote: Looks like

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Mike Galbraith
On Fri, 2014-05-09 at 11:24 -0400, Rik van Riel wrote: On 05/09/2014 11:24 AM, Mike Galbraith wrote: If no -flags SD_BALANCE_WAKE is encountered during traversal, sd remains NULL, we fall through to return prev_cpu. We do fall through, but into this loop:

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Rik van Riel
On 05/09/2014 01:55 PM, Mike Galbraith wrote: On Fri, 2014-05-09 at 11:24 -0400, Rik van Riel wrote: On 05/09/2014 11:24 AM, Mike Galbraith wrote: If no -flags SD_BALANCE_WAKE is encountered during traversal, sd remains NULL, we fall through to return prev_cpu. We do

Re: [PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-09 Thread Mike Galbraith
On Fri, 2014-05-09 at 14:16 -0400, Rik van Riel wrote: That leaves the big question: do we want to fall back to prev_cpu if it is not idle, and it has an idle sibling, or would it be better to find an idle sibling of prev_cpu when we wake up a task? Yes. If there was A correct answer, this

[PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-08 Thread Rik van Riel
On Thu, 08 May 2014 22:20:25 -0400 Rik van Riel wrote: > Looks like SD_BALANCE_WAKE is not gotten from the sd flags at > all, but passed into select_task_rq by try_to_wake_up, as a > hard coded sd_flags argument. > Should we do that, if SD_WAKE_BALANCE is not set for any sched domain? I

[PATCH] sched: wake up task on prev_cpu if not in SD_WAKE_AFFINE domain with cpu

2014-05-08 Thread Rik van Riel
On Thu, 08 May 2014 22:20:25 -0400 Rik van Riel r...@redhat.com wrote: Looks like SD_BALANCE_WAKE is not gotten from the sd flags at all, but passed into select_task_rq by try_to_wake_up, as a hard coded sd_flags argument. Should we do that, if SD_WAKE_BALANCE is not set for any sched