Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-11 Thread Yuyang Du
On Fri, Jul 11, 2014 at 08:52:01AM +0800, Yuyang Du wrote: > On Fri, Jul 11, 2014 at 10:47:09AM +0200, Peter Zijlstra wrote: > > On Fri, Jul 11, 2014 at 07:22:07AM +0800, Yuyang Du wrote: > > > On Thu, Jul 10, 2014 at 12:08:59PM +0200, Peter Zijlstra wrote: > > > > > > > > Since clock_task is the

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-11 Thread Yuyang Du
On Fri, Jul 11, 2014 at 10:47:09AM +0200, Peter Zijlstra wrote: > On Fri, Jul 11, 2014 at 07:22:07AM +0800, Yuyang Du wrote: > > On Thu, Jul 10, 2014 at 12:08:59PM +0200, Peter Zijlstra wrote: > > > > > > Since clock_task is the regular clock minus some local amount, the > > > difference between

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-11 Thread Peter Zijlstra
On Fri, Jul 11, 2014 at 07:22:07AM +0800, Yuyang Du wrote: > On Thu, Jul 10, 2014 at 12:08:59PM +0200, Peter Zijlstra wrote: > > > > Since clock_task is the regular clock minus some local amount, the > > difference between two regular clock reads is always a strict upper > > bound on clock_task

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-11 Thread Yuyang Du
On Thu, Jul 10, 2014 at 12:08:59PM +0200, Peter Zijlstra wrote: > > Since clock_task is the regular clock minus some local amount, the > difference between two regular clock reads is always a strict upper > bound on clock_task differences. > This is inspiring. Regarding the clock source in load

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-11 Thread Yuyang Du
On Thu, Jul 10, 2014 at 12:08:59PM +0200, Peter Zijlstra wrote: Since clock_task is the regular clock minus some local amount, the difference between two regular clock reads is always a strict upper bound on clock_task differences. This is inspiring. Regarding the clock source in load avg

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-11 Thread Peter Zijlstra
On Fri, Jul 11, 2014 at 07:22:07AM +0800, Yuyang Du wrote: On Thu, Jul 10, 2014 at 12:08:59PM +0200, Peter Zijlstra wrote: Since clock_task is the regular clock minus some local amount, the difference between two regular clock reads is always a strict upper bound on clock_task

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-11 Thread Yuyang Du
On Fri, Jul 11, 2014 at 10:47:09AM +0200, Peter Zijlstra wrote: On Fri, Jul 11, 2014 at 07:22:07AM +0800, Yuyang Du wrote: On Thu, Jul 10, 2014 at 12:08:59PM +0200, Peter Zijlstra wrote: Since clock_task is the regular clock minus some local amount, the difference between two regular

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-11 Thread Yuyang Du
On Fri, Jul 11, 2014 at 08:52:01AM +0800, Yuyang Du wrote: On Fri, Jul 11, 2014 at 10:47:09AM +0200, Peter Zijlstra wrote: On Fri, Jul 11, 2014 at 07:22:07AM +0800, Yuyang Du wrote: On Thu, Jul 10, 2014 at 12:08:59PM +0200, Peter Zijlstra wrote: Since clock_task is the regular clock

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread Yuyang Du
On Thu, Jul 10, 2014 at 10:06:27AM -0700, bseg...@google.com wrote: > So, sched_clock(_cpu) can be arbitrarily far off of cfs_rq_clock_task, so you > can't really do that. Ideally, yes, you would account for any time since > the last update and account that time as !runnable. However, I don't >

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread Yuyang Du
On Thu, Jul 10, 2014 at 10:01:42AM -0700, bseg...@google.com wrote: > > The problem with that is that last_update_time is measured in > > clock_task, and you cannot transfer these values between CPUs. > > clock_task can drift unbounded between CPUs. > > Yes, but we don't need to - we just use the

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread bsegall
Yuyang Du writes: > Thanks, Peter. > > On Wed, Jul 09, 2014 at 08:45:43PM +0200, Peter Zijlstra wrote: > >> Nope :-).. we got rid of that lock for a good reason. >> >> Also, this is one area where I feel performance really trumps >> correctness, we can fudge the blocked load a little. So the >>

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread bsegall
Peter Zijlstra writes: > On Wed, Jul 09, 2014 at 12:07:08PM -0700, bseg...@google.com wrote: >> Peter Zijlstra writes: >> >> > On Wed, Jul 09, 2014 at 09:07:53AM +0800, Yuyang Du wrote: >> >> That is chalenging... Can someone (Peter) grant us a lock of the remote >> >> rq? :) >> > >> > Nope

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread Peter Zijlstra
On Wed, Jul 09, 2014 at 12:07:08PM -0700, bseg...@google.com wrote: > Peter Zijlstra writes: > > > On Wed, Jul 09, 2014 at 09:07:53AM +0800, Yuyang Du wrote: > >> That is chalenging... Can someone (Peter) grant us a lock of the remote > >> rq? :) > > > > Nope :-).. we got rid of that lock for a

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread Yuyang Du
Thanks, Peter. On Wed, Jul 09, 2014 at 08:45:43PM +0200, Peter Zijlstra wrote: > Nope :-).. we got rid of that lock for a good reason. > > Also, this is one area where I feel performance really trumps > correctness, we can fudge the blocked load a little. So the > sched_clock_cpu() difference

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread Yuyang Du
Thanks, Peter. On Wed, Jul 09, 2014 at 08:45:43PM +0200, Peter Zijlstra wrote: Nope :-).. we got rid of that lock for a good reason. Also, this is one area where I feel performance really trumps correctness, we can fudge the blocked load a little. So the sched_clock_cpu() difference is a

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread Peter Zijlstra
On Wed, Jul 09, 2014 at 12:07:08PM -0700, bseg...@google.com wrote: Peter Zijlstra pet...@infradead.org writes: On Wed, Jul 09, 2014 at 09:07:53AM +0800, Yuyang Du wrote: That is chalenging... Can someone (Peter) grant us a lock of the remote rq? :) Nope :-).. we got rid of that lock

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread bsegall
Peter Zijlstra pet...@infradead.org writes: On Wed, Jul 09, 2014 at 12:07:08PM -0700, bseg...@google.com wrote: Peter Zijlstra pet...@infradead.org writes: On Wed, Jul 09, 2014 at 09:07:53AM +0800, Yuyang Du wrote: That is chalenging... Can someone (Peter) grant us a lock of the remote

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread bsegall
Yuyang Du yuyang...@intel.com writes: Thanks, Peter. On Wed, Jul 09, 2014 at 08:45:43PM +0200, Peter Zijlstra wrote: Nope :-).. we got rid of that lock for a good reason. Also, this is one area where I feel performance really trumps correctness, we can fudge the blocked load a little. So

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread Yuyang Du
On Thu, Jul 10, 2014 at 10:01:42AM -0700, bseg...@google.com wrote: The problem with that is that last_update_time is measured in clock_task, and you cannot transfer these values between CPUs. clock_task can drift unbounded between CPUs. Yes, but we don't need to - we just use the remote

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-10 Thread Yuyang Du
On Thu, Jul 10, 2014 at 10:06:27AM -0700, bseg...@google.com wrote: So, sched_clock(_cpu) can be arbitrarily far off of cfs_rq_clock_task, so you can't really do that. Ideally, yes, you would account for any time since the last update and account that time as !runnable. However, I don't

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread Yuyang Du
On Wed, Jul 09, 2014 at 10:08:42AM -0700, bseg...@google.com wrote: > > That I believe is not a problem. It is a matter of when cfs_rq->load.weight > > changes and when we look at it to contribute to the cfs_rq's load_avg. > > Fortunately, we will not miss any change of cfs_rq->load.weight,

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread bsegall
Peter Zijlstra writes: > On Wed, Jul 09, 2014 at 09:07:53AM +0800, Yuyang Du wrote: >> That is chalenging... Can someone (Peter) grant us a lock of the remote rq? >> :) > > Nope :-).. we got rid of that lock for a good reason. > > Also, this is one area where I feel performance really trumps >

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread Peter Zijlstra
On Wed, Jul 09, 2014 at 09:07:53AM +0800, Yuyang Du wrote: > That is chalenging... Can someone (Peter) grant us a lock of the remote rq? :) Nope :-).. we got rid of that lock for a good reason. Also, this is one area where I feel performance really trumps correctness, we can fudge the blocked

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread bsegall
Yuyang Du writes: > Thanks, Ben. > > On Tue, Jul 08, 2014 at 10:04:22AM -0700, bseg...@google.com wrote: > >> > The sampling of cfs_rq->load.weight should be equivalent to the current >> > code in that at the end of day cfs_rq->load.weight worth of runnable would >> > contribute to

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread Yuyang Du
Thanks, Ben. On Tue, Jul 08, 2014 at 10:04:22AM -0700, bseg...@google.com wrote: > > The sampling of cfs_rq->load.weight should be equivalent to the current > > code in that at the end of day cfs_rq->load.weight worth of runnable would > > contribute to runnable_load_avg/blocked_load_avg for

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread Yuyang Du
Thanks, Ben. On Tue, Jul 08, 2014 at 10:04:22AM -0700, bseg...@google.com wrote: The sampling of cfs_rq-load.weight should be equivalent to the current code in that at the end of day cfs_rq-load.weight worth of runnable would contribute to runnable_load_avg/blocked_load_avg for both the

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread bsegall
Yuyang Du yuyang...@intel.com writes: Thanks, Ben. On Tue, Jul 08, 2014 at 10:04:22AM -0700, bseg...@google.com wrote: The sampling of cfs_rq-load.weight should be equivalent to the current code in that at the end of day cfs_rq-load.weight worth of runnable would contribute to

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread Peter Zijlstra
On Wed, Jul 09, 2014 at 09:07:53AM +0800, Yuyang Du wrote: That is chalenging... Can someone (Peter) grant us a lock of the remote rq? :) Nope :-).. we got rid of that lock for a good reason. Also, this is one area where I feel performance really trumps correctness, we can fudge the blocked

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread bsegall
Peter Zijlstra pet...@infradead.org writes: On Wed, Jul 09, 2014 at 09:07:53AM +0800, Yuyang Du wrote: That is chalenging... Can someone (Peter) grant us a lock of the remote rq? :) Nope :-).. we got rid of that lock for a good reason. Also, this is one area where I feel performance

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-09 Thread Yuyang Du
On Wed, Jul 09, 2014 at 10:08:42AM -0700, bseg...@google.com wrote: That I believe is not a problem. It is a matter of when cfs_rq-load.weight changes and when we look at it to contribute to the cfs_rq's load_avg. Fortunately, we will not miss any change of cfs_rq-load.weight, always

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-08 Thread bsegall
Yuyang Du writes: > Thanks, Ben, > > On Mon, Jul 07, 2014 at 03:25:07PM -0700, bseg...@google.com wrote: >> Also something to note is that cfs_rq->load_avg just takes samples of >> load.weight every 1us, which seems unfortunate. We thought this was ok >> for p->se.load.weight, because it isn't

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-08 Thread Peter Zijlstra
On Mon, Jul 07, 2014 at 03:25:07PM -0700, bseg...@google.com wrote: > >> +static inline void enqueue_entity_load_avg(struct sched_entity *se) > >> { > >> + struct sched_avg *sa = >avg; > >> + struct cfs_rq *cfs_rq = cfs_rq_of(se); > >> + u64 now = cfs_rq_clock_task(cfs_rq); > >> + u32

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-08 Thread Yuyang Du
Thanks, Ben, On Mon, Jul 07, 2014 at 03:25:07PM -0700, bseg...@google.com wrote: > > Yeah, while this is technically limited to 1/us (per cpu), it is still > much higher - the replaced code would do updates generally only on > period overflow (1ms) and even then only with nontrivial delta. >

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-08 Thread Yuyang Du
Thanks, Ben, On Mon, Jul 07, 2014 at 03:25:07PM -0700, bseg...@google.com wrote: Yeah, while this is technically limited to 1/us (per cpu), it is still much higher - the replaced code would do updates generally only on period overflow (1ms) and even then only with nontrivial delta. Will

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-08 Thread Peter Zijlstra
On Mon, Jul 07, 2014 at 03:25:07PM -0700, bseg...@google.com wrote: +static inline void enqueue_entity_load_avg(struct sched_entity *se) { + struct sched_avg *sa = se-avg; + struct cfs_rq *cfs_rq = cfs_rq_of(se); + u64 now = cfs_rq_clock_task(cfs_rq); + u32 old_load_avg =

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-08 Thread bsegall
Yuyang Du yuyang...@intel.com writes: Thanks, Ben, On Mon, Jul 07, 2014 at 03:25:07PM -0700, bseg...@google.com wrote: Also something to note is that cfs_rq-load_avg just takes samples of load.weight every 1us, which seems unfortunate. We thought this was ok for p-se.load.weight, because it

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-07 Thread Yuyang Du
Thanks, Peter. On Mon, Jul 07, 2014 at 12:46:46PM +0200, Peter Zijlstra wrote: > > That tg->load_avg cacheline is already red hot glowing, and you've just > increased the amount of updates to it.. That's not going to be pleasant. > Logically, this rewrite updates tg->load_avg as soon as it is

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-07 Thread bsegall
Peter Zijlstra writes: > On Wed, Jul 02, 2014 at 10:30:56AM +0800, Yuyang Du wrote: >> The idea of per entity runnable load average (aggregated to cfs_rq and >> task_group load) >> was proposed by Paul Turner, and it is still followed by this rewrite. But >> this rewrite >> is made due to the

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-07 Thread Peter Zijlstra
On Wed, Jul 02, 2014 at 10:30:56AM +0800, Yuyang Du wrote: > The idea of per entity runnable load average (aggregated to cfs_rq and > task_group load) > was proposed by Paul Turner, and it is still followed by this rewrite. But > this rewrite > is made due to the following ends: > > (1).

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-07 Thread Peter Zijlstra
On Wed, Jul 02, 2014 at 10:30:56AM +0800, Yuyang Du wrote: > The idea of per entity runnable load average (aggregated to cfs_rq and > task_group load) > was proposed by Paul Turner, and it is still followed by this rewrite. But > this rewrite > is made due to the following ends: > > (1).

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-07 Thread Peter Zijlstra
On Wed, Jul 02, 2014 at 10:30:56AM +0800, Yuyang Du wrote: The idea of per entity runnable load average (aggregated to cfs_rq and task_group load) was proposed by Paul Turner, and it is still followed by this rewrite. But this rewrite is made due to the following ends: (1). cfs_rq's load

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-07 Thread Peter Zijlstra
On Wed, Jul 02, 2014 at 10:30:56AM +0800, Yuyang Du wrote: The idea of per entity runnable load average (aggregated to cfs_rq and task_group load) was proposed by Paul Turner, and it is still followed by this rewrite. But this rewrite is made due to the following ends: (1). cfs_rq's load

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-07 Thread bsegall
Peter Zijlstra pet...@infradead.org writes: On Wed, Jul 02, 2014 at 10:30:56AM +0800, Yuyang Du wrote: The idea of per entity runnable load average (aggregated to cfs_rq and task_group load) was proposed by Paul Turner, and it is still followed by this rewrite. But this rewrite is made

Re: [PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-07 Thread Yuyang Du
Thanks, Peter. On Mon, Jul 07, 2014 at 12:46:46PM +0200, Peter Zijlstra wrote: That tg-load_avg cacheline is already red hot glowing, and you've just increased the amount of updates to it.. That's not going to be pleasant. Logically, this rewrite updates tg-load_avg as soon as it is

[PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-02 Thread Yuyang Du
The idea of per entity runnable load average (aggregated to cfs_rq and task_group load) was proposed by Paul Turner, and it is still followed by this rewrite. But this rewrite is made due to the following ends: (1). cfs_rq's load average (namely runnable_load_avg and blocked_load_avg) is

[PATCH 2/2] sched: Rewrite per entity runnable load average tracking

2014-07-02 Thread Yuyang Du
The idea of per entity runnable load average (aggregated to cfs_rq and task_group load) was proposed by Paul Turner, and it is still followed by this rewrite. But this rewrite is made due to the following ends: (1). cfs_rq's load average (namely runnable_load_avg and blocked_load_avg) is