Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-13 Thread Michael wang
On 07/12/2014 12:11 AM, Rik van Riel wrote: > -BEGIN PGP SIGNED MESSAGE- [snip] >> >> That's full wake balance.. if that was cheap, >> select_idle_sibling() would not exist. > > Full wake balance iterates over all the groups in the system, > select_idle_sibling only over one LLC domain.

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-13 Thread Michael wang
On 07/12/2014 12:11 AM, Rik van Riel wrote: -BEGIN PGP SIGNED MESSAGE- [snip] That's full wake balance.. if that was cheap, select_idle_sibling() would not exist. Full wake balance iterates over all the groups in the system, select_idle_sibling only over one LLC domain.

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-11 Thread Rik van Riel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/02/2014 11:51 PM, Mike Galbraith wrote: > On Wed, 2014-07-02 at 10:47 -0400, Rik van Riel wrote: >> On 07/01/2014 04:38 AM, Michael wang wrote: >>> On 07/01/2014 04:20 PM, Peter Zijlstra wrote: [snip] > > Just wondering could we make

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-11 Thread Rik van Riel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/02/2014 11:51 PM, Mike Galbraith wrote: On Wed, 2014-07-02 at 10:47 -0400, Rik van Riel wrote: On 07/01/2014 04:38 AM, Michael wang wrote: On 07/01/2014 04:20 PM, Peter Zijlstra wrote: [snip] Just wondering could we make this another

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-03 Thread Nicolas Pitre
On Wed, 2 Jul 2014, Peter Zijlstra wrote: > On Wed, Jul 02, 2014 at 02:49:18PM +0200, Peter Zijlstra wrote: > > Clearly I need to go take out all these things because people don't seem > > to know this and SCHED_DEBUG isn't a big enough hint. Tedious. > > Maybe this would be enough clue? > >

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-03 Thread Nicolas Pitre
On Wed, 2 Jul 2014, Peter Zijlstra wrote: On Wed, Jul 02, 2014 at 02:49:18PM +0200, Peter Zijlstra wrote: Clearly I need to go take out all these things because people don't seem to know this and SCHED_DEBUG isn't a big enough hint. Tedious. Maybe this would be enough clue? diff --git

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Mike Galbraith
On Wed, 2014-07-02 at 10:47 -0400, Rik van Riel wrote: > On 07/01/2014 04:38 AM, Michael wang wrote: > > On 07/01/2014 04:20 PM, Peter Zijlstra wrote: > > [snip] > >>> > >>> Just wondering could we make this another scheduler feature? > >> > >> No; sched_feat() is for debugging, BIG CLUE: its

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Michael wang
On 07/02/2014 10:47 PM, Rik van Riel wrote: > On 07/01/2014 04:38 AM, Michael wang wrote: >> On 07/01/2014 04:20 PM, Peter Zijlstra wrote: >> [snip] Just wondering could we make this another scheduler feature? >>> >>> No; sched_feat() is for debugging, BIG CLUE: its guarded by >>>

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Michael wang
On 07/02/2014 08:49 PM, Peter Zijlstra wrote: > On Wed, Jul 02, 2014 at 10:47:34AM +0800, Michael wang wrote: >> The opinion on features actually make me a little confusing... I used to >> think the scheduler is willing on providing kinds of way to adapt itself >> to different situation, and some

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Rik van Riel
On 07/01/2014 04:38 AM, Michael wang wrote: > On 07/01/2014 04:20 PM, Peter Zijlstra wrote: > [snip] >>> >>> Just wondering could we make this another scheduler feature? >> >> No; sched_feat() is for debugging, BIG CLUE: its guarded by >> CONFIG_SCHED_DEBUG, anybody using it in production or

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Peter Zijlstra
On Wed, Jul 02, 2014 at 02:49:18PM +0200, Peter Zijlstra wrote: > Clearly I need to go take out all these things because people don't seem > to know this and SCHED_DEBUG isn't a big enough hint. Tedious. Maybe this would be enough clue? --- include/linux/kernel.h | 1 +

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Peter Zijlstra
On Wed, Jul 02, 2014 at 10:47:34AM +0800, Michael wang wrote: > The opinion on features actually make me a little confusing... I used to > think the scheduler is willing on providing kinds of way to adapt itself > to different situation, and some features do help on that, make them > only a debug

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Peter Zijlstra
On Wed, Jul 02, 2014 at 10:47:34AM +0800, Michael wang wrote: The opinion on features actually make me a little confusing... I used to think the scheduler is willing on providing kinds of way to adapt itself to different situation, and some features do help on that, make them only a debug

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Peter Zijlstra
On Wed, Jul 02, 2014 at 02:49:18PM +0200, Peter Zijlstra wrote: Clearly I need to go take out all these things because people don't seem to know this and SCHED_DEBUG isn't a big enough hint. Tedious. Maybe this would be enough clue? --- include/linux/kernel.h | 1 +

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Rik van Riel
On 07/01/2014 04:38 AM, Michael wang wrote: On 07/01/2014 04:20 PM, Peter Zijlstra wrote: [snip] Just wondering could we make this another scheduler feature? No; sched_feat() is for debugging, BIG CLUE: its guarded by CONFIG_SCHED_DEBUG, anybody using it in production or anywhere else is

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Michael wang
On 07/02/2014 08:49 PM, Peter Zijlstra wrote: On Wed, Jul 02, 2014 at 10:47:34AM +0800, Michael wang wrote: The opinion on features actually make me a little confusing... I used to think the scheduler is willing on providing kinds of way to adapt itself to different situation, and some

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Michael wang
On 07/02/2014 10:47 PM, Rik van Riel wrote: On 07/01/2014 04:38 AM, Michael wang wrote: On 07/01/2014 04:20 PM, Peter Zijlstra wrote: [snip] Just wondering could we make this another scheduler feature? No; sched_feat() is for debugging, BIG CLUE: its guarded by CONFIG_SCHED_DEBUG, anybody

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-02 Thread Mike Galbraith
On Wed, 2014-07-02 at 10:47 -0400, Rik van Riel wrote: On 07/01/2014 04:38 AM, Michael wang wrote: On 07/01/2014 04:20 PM, Peter Zijlstra wrote: [snip] Just wondering could we make this another scheduler feature? No; sched_feat() is for debugging, BIG CLUE: its guarded by

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Michael wang
On 07/01/2014 04:56 PM, Peter Zijlstra wrote: > On Tue, Jul 01, 2014 at 04:38:58PM +0800, Michael wang wrote: [snip] >> Currently when dbench running with stress, it could only gain one CPU, >> and cpu-cgroup cpu.shares is meaningless, is there any good methods to >> address that? > > So as far

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Peter Zijlstra
On Tue, Jul 01, 2014 at 04:38:58PM +0800, Michael wang wrote: > On 07/01/2014 04:20 PM, Peter Zijlstra wrote: > [snip] > >> > >> Just wondering could we make this another scheduler feature? > > > > No; sched_feat() is for debugging, BIG CLUE: its guarded by > > CONFIG_SCHED_DEBUG, anybody using

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Michael wang
On 07/01/2014 04:20 PM, Peter Zijlstra wrote: [snip] >> >> Just wondering could we make this another scheduler feature? > > No; sched_feat() is for debugging, BIG CLUE: its guarded by > CONFIG_SCHED_DEBUG, anybody using it in production or anywhere else is > broken. > > If people are using it, I

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Peter Zijlstra
On Tue, Jun 24, 2014 at 11:34:54AM +0800, Michael wang wrote: > On 06/23/2014 05:42 PM, Peter Zijlstra wrote: > [snip] > >> +} > > > > Still completely hate this, it doesn't make sense conceptual sense what > > so ever. > > Yeah... and now I agree your opinion that this could not address all the

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Michael wang
On 07/01/2014 01:41 PM, Mike Galbraith wrote: > On Tue, 2014-07-01 at 10:57 +0800, Michael wang wrote: > >> IMHO, currently the generic scheduler just try to take care both latency >> and throughput, both will take a little damage but won't be damaged too >> much, they just sacrificed for each

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Michael wang
On 07/01/2014 01:41 PM, Mike Galbraith wrote: On Tue, 2014-07-01 at 10:57 +0800, Michael wang wrote: IMHO, currently the generic scheduler just try to take care both latency and throughput, both will take a little damage but won't be damaged too much, they just sacrificed for each other...

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Peter Zijlstra
On Tue, Jun 24, 2014 at 11:34:54AM +0800, Michael wang wrote: On 06/23/2014 05:42 PM, Peter Zijlstra wrote: [snip] +} Still completely hate this, it doesn't make sense conceptual sense what so ever. Yeah... and now I agree your opinion that this could not address all the cases after

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Michael wang
On 07/01/2014 04:20 PM, Peter Zijlstra wrote: [snip] Just wondering could we make this another scheduler feature? No; sched_feat() is for debugging, BIG CLUE: its guarded by CONFIG_SCHED_DEBUG, anybody using it in production or anywhere else is broken. If people are using it, I should

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Peter Zijlstra
On Tue, Jul 01, 2014 at 04:38:58PM +0800, Michael wang wrote: On 07/01/2014 04:20 PM, Peter Zijlstra wrote: [snip] Just wondering could we make this another scheduler feature? No; sched_feat() is for debugging, BIG CLUE: its guarded by CONFIG_SCHED_DEBUG, anybody using it in

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-07-01 Thread Michael wang
On 07/01/2014 04:56 PM, Peter Zijlstra wrote: On Tue, Jul 01, 2014 at 04:38:58PM +0800, Michael wang wrote: [snip] Currently when dbench running with stress, it could only gain one CPU, and cpu-cgroup cpu.shares is meaningless, is there any good methods to address that? So as far as I

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Mike Galbraith
On Tue, 2014-07-01 at 10:57 +0800, Michael wang wrote: > IMHO, currently the generic scheduler just try to take care both latency > and throughput, both will take a little damage but won't be damaged too > much, they just sacrificed for each other... The cost can be large, so it's worth an

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Michael wang
On 06/30/2014 05:27 PM, Mike Galbraith wrote: > On Mon, 2014-06-30 at 16:47 +0800, Michael wang wrote: [snip] >>> While you're getting rid of the concept of 'GENTLE_FAIR_SLEEPERS', don't >>> forget to also get rid of the concept of 'over-scheduling' :) >> >> I'm new to this word... could you give

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Mike Galbraith
On Mon, 2014-06-30 at 16:47 +0800, Michael wang wrote: > Hi, Mike :) > > On 06/30/2014 04:06 PM, Mike Galbraith wrote: > > On Mon, 2014-06-30 at 15:36 +0800, Michael wang wrote: > >> On 06/18/2014 12:50 PM, Michael wang wrote: > >>> By testing we found that after put benchmark (dbench) in to

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Michael wang
Hi, Mike :) On 06/30/2014 04:06 PM, Mike Galbraith wrote: > On Mon, 2014-06-30 at 15:36 +0800, Michael wang wrote: >> On 06/18/2014 12:50 PM, Michael wang wrote: >>> By testing we found that after put benchmark (dbench) in to deep cpu-group, >>> tasks (dbench routines) start to gathered on one

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Mike Galbraith
On Mon, 2014-06-30 at 15:36 +0800, Michael wang wrote: > On 06/18/2014 12:50 PM, Michael wang wrote: > > By testing we found that after put benchmark (dbench) in to deep cpu-group, > > tasks (dbench routines) start to gathered on one CPU, which lead to that the > > benchmark could only get around

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Michael wang
On 06/18/2014 12:50 PM, Michael wang wrote: > By testing we found that after put benchmark (dbench) in to deep cpu-group, > tasks (dbench routines) start to gathered on one CPU, which lead to that the > benchmark could only get around 100% CPU whatever how big it's task-group's > share is, here is

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Michael wang
On 06/18/2014 12:50 PM, Michael wang wrote: By testing we found that after put benchmark (dbench) in to deep cpu-group, tasks (dbench routines) start to gathered on one CPU, which lead to that the benchmark could only get around 100% CPU whatever how big it's task-group's share is, here is the

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Mike Galbraith
On Mon, 2014-06-30 at 15:36 +0800, Michael wang wrote: On 06/18/2014 12:50 PM, Michael wang wrote: By testing we found that after put benchmark (dbench) in to deep cpu-group, tasks (dbench routines) start to gathered on one CPU, which lead to that the benchmark could only get around 100%

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Michael wang
Hi, Mike :) On 06/30/2014 04:06 PM, Mike Galbraith wrote: On Mon, 2014-06-30 at 15:36 +0800, Michael wang wrote: On 06/18/2014 12:50 PM, Michael wang wrote: By testing we found that after put benchmark (dbench) in to deep cpu-group, tasks (dbench routines) start to gathered on one CPU, which

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Mike Galbraith
On Mon, 2014-06-30 at 16:47 +0800, Michael wang wrote: Hi, Mike :) On 06/30/2014 04:06 PM, Mike Galbraith wrote: On Mon, 2014-06-30 at 15:36 +0800, Michael wang wrote: On 06/18/2014 12:50 PM, Michael wang wrote: By testing we found that after put benchmark (dbench) in to deep

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Michael wang
On 06/30/2014 05:27 PM, Mike Galbraith wrote: On Mon, 2014-06-30 at 16:47 +0800, Michael wang wrote: [snip] While you're getting rid of the concept of 'GENTLE_FAIR_SLEEPERS', don't forget to also get rid of the concept of 'over-scheduling' :) I'm new to this word... could you give more

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-30 Thread Mike Galbraith
On Tue, 2014-07-01 at 10:57 +0800, Michael wang wrote: IMHO, currently the generic scheduler just try to take care both latency and throughput, both will take a little damage but won't be damaged too much, they just sacrificed for each other... The cost can be large, so it's worth an

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-23 Thread Michael wang
On 06/23/2014 05:42 PM, Peter Zijlstra wrote: [snip] >> +} > > Still completely hate this, it doesn't make sense conceptual sense what > so ever. Yeah... and now I agree your opinion that this could not address all the cases after all the testing these days... Just wondering could we make this

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-23 Thread Peter Zijlstra
On Wed, Jun 18, 2014 at 12:50:17PM +0800, Michael wang wrote: > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index fea7d33..e1381cd 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -4409,6 +4409,62 @@ find_idlest_cpu(struct sched_group *group, struct > task_struct

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-23 Thread Michael wang
On 06/23/2014 05:42 PM, Peter Zijlstra wrote: [snip] +} Still completely hate this, it doesn't make sense conceptual sense what so ever. Yeah... and now I agree your opinion that this could not address all the cases after all the testing these days... Just wondering could we make this

Re: [PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-23 Thread Peter Zijlstra
On Wed, Jun 18, 2014 at 12:50:17PM +0800, Michael wang wrote: diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index fea7d33..e1381cd 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4409,6 +4409,62 @@ find_idlest_cpu(struct sched_group *group, struct task_struct *p, int

[PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-17 Thread Michael wang
By testing we found that after put benchmark (dbench) in to deep cpu-group, tasks (dbench routines) start to gathered on one CPU, which lead to that the benchmark could only get around 100% CPU whatever how big it's task-group's share is, here is the link of the way to reproduce the issue:

[PATCH] sched: select 'idle' cfs_rq per task-group to prevent tg-internal imbalance

2014-06-17 Thread Michael wang
By testing we found that after put benchmark (dbench) in to deep cpu-group, tasks (dbench routines) start to gathered on one CPU, which lead to that the benchmark could only get around 100% CPU whatever how big it's task-group's share is, here is the link of the way to reproduce the issue: