Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-27 Thread Joel Fernandes (Google)
On Thu, Jul 27, 2017 at 12:21 AM, Juri Lelli  wrote:
[..]
>> >
>> > But even without that, if you see the routine
>> > init_entity_runnable_average() in fair.c, the new tasks are
>> > initialized in a way that they are seen as heavy tasks. And so even
>> > for the first time they run, freq should normally increase on the
>> > target CPU (at least with above application).i
>>
>> Ok, but the "heavy" in init_entity_runnable_average means for load,
>> not the util_avg. The util_avg is what's used for frequency scaling
>> IIUC and is set to 0 in that function no?
>>
>
> True for init_entity_runnable_average(), but for new task post_init_
> entity_util_avg() is then also called (from wake_up_new_task()), which
> modifies the initial util_avg value (depending on current rq {util,
> load}_avg.

Got it. That makes sense, thank you!

-Joel


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-27 Thread Joel Fernandes (Google)
On Thu, Jul 27, 2017 at 12:21 AM, Juri Lelli  wrote:
[..]
>> >
>> > But even without that, if you see the routine
>> > init_entity_runnable_average() in fair.c, the new tasks are
>> > initialized in a way that they are seen as heavy tasks. And so even
>> > for the first time they run, freq should normally increase on the
>> > target CPU (at least with above application).i
>>
>> Ok, but the "heavy" in init_entity_runnable_average means for load,
>> not the util_avg. The util_avg is what's used for frequency scaling
>> IIUC and is set to 0 in that function no?
>>
>
> True for init_entity_runnable_average(), but for new task post_init_
> entity_util_avg() is then also called (from wake_up_new_task()), which
> modifies the initial util_avg value (depending on current rq {util,
> load}_avg.

Got it. That makes sense, thank you!

-Joel


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-27 Thread Juri Lelli
Hi,

On 26/07/17 23:23, Joel Fernandes (Google) wrote:
> Hi Viresh,
> 
> On Wed, Jul 26, 2017 at 10:46 PM, Viresh Kumar  
> wrote:
> > On 26-07-17, 22:14, Joel Fernandes (Google) wrote:

[...]

> >
> > But even without that, if you see the routine
> > init_entity_runnable_average() in fair.c, the new tasks are
> > initialized in a way that they are seen as heavy tasks. And so even
> > for the first time they run, freq should normally increase on the
> > target CPU (at least with above application).i
> 
> Ok, but the "heavy" in init_entity_runnable_average means for load,
> not the util_avg. The util_avg is what's used for frequency scaling
> IIUC and is set to 0 in that function no?
> 

True for init_entity_runnable_average(), but for new task post_init_
entity_util_avg() is then also called (from wake_up_new_task()), which
modifies the initial util_avg value (depending on current rq {util,
load}_avg.


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-27 Thread Juri Lelli
Hi,

On 26/07/17 23:23, Joel Fernandes (Google) wrote:
> Hi Viresh,
> 
> On Wed, Jul 26, 2017 at 10:46 PM, Viresh Kumar  
> wrote:
> > On 26-07-17, 22:14, Joel Fernandes (Google) wrote:

[...]

> >
> > But even without that, if you see the routine
> > init_entity_runnable_average() in fair.c, the new tasks are
> > initialized in a way that they are seen as heavy tasks. And so even
> > for the first time they run, freq should normally increase on the
> > target CPU (at least with above application).i
> 
> Ok, but the "heavy" in init_entity_runnable_average means for load,
> not the util_avg. The util_avg is what's used for frequency scaling
> IIUC and is set to 0 in that function no?
> 

True for init_entity_runnable_average(), but for new task post_init_
entity_util_avg() is then also called (from wake_up_new_task()), which
modifies the initial util_avg value (depending on current rq {util,
load}_avg.


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-27 Thread Viresh Kumar
On 26-07-17, 23:23, Joel Fernandes (Google) wrote:
> Ok, but the "heavy" in init_entity_runnable_average means for load,
> not the util_avg. The util_avg is what's used for frequency scaling
> IIUC and is set to 0 in that function no?

That's because the task isn't enqueued yet and so don't have any
utilization. The last line of that routine is a comment which says:

/* when this task enqueue'ed, it will contribute to its cfs_rq's load_avg */

But once the task is enqueued, this load_avg will get considered for
sure :)

> > The application was written by Steve (all credit goes to him) before
> > he left Linaro, but I did test it with ftrace. What I saw with ftrace
> > was that the freq isn't reevaluated for almost an entire tick many
> > times because we enqueued the task remotely. And that changes with
> > this series.
> >
> >> > The reason being that this patchset only targets a corner case, where
> >> > following are required to be true to improve performance and that
> >> > doesn't happen too often with these tests:
> >> >
> >> > - Task is migrated to another CPU.
> >> > - The task has maximum demand initially, and should take the CPU to
> >>
> >> Just to make the cover-letter more clear and also confirming with you
> >> I understand the above usecase, maybe in the future this can reworded
> >> from "initially" to "before the migration" and "take the CPU" to "take
> >> the target CPU of the migration" ?
> >
> > I can reword it a bit, but the test case wasn't really migrating
> > anything and was looking only at the initial loads.
> 
> Ok, I wasn't talking about the synthetic test in the second part of my
> email above but about the explanation you gave about Galleryfling
> improvement (that the migration of a task with high utilization
> doesn't update the target frequency) which makes sense to me so we are
> on the same page about that.

Okay, great.

-- 
viresh


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-27 Thread Viresh Kumar
On 26-07-17, 23:23, Joel Fernandes (Google) wrote:
> Ok, but the "heavy" in init_entity_runnable_average means for load,
> not the util_avg. The util_avg is what's used for frequency scaling
> IIUC and is set to 0 in that function no?

That's because the task isn't enqueued yet and so don't have any
utilization. The last line of that routine is a comment which says:

/* when this task enqueue'ed, it will contribute to its cfs_rq's load_avg */

But once the task is enqueued, this load_avg will get considered for
sure :)

> > The application was written by Steve (all credit goes to him) before
> > he left Linaro, but I did test it with ftrace. What I saw with ftrace
> > was that the freq isn't reevaluated for almost an entire tick many
> > times because we enqueued the task remotely. And that changes with
> > this series.
> >
> >> > The reason being that this patchset only targets a corner case, where
> >> > following are required to be true to improve performance and that
> >> > doesn't happen too often with these tests:
> >> >
> >> > - Task is migrated to another CPU.
> >> > - The task has maximum demand initially, and should take the CPU to
> >>
> >> Just to make the cover-letter more clear and also confirming with you
> >> I understand the above usecase, maybe in the future this can reworded
> >> from "initially" to "before the migration" and "take the CPU" to "take
> >> the target CPU of the migration" ?
> >
> > I can reword it a bit, but the test case wasn't really migrating
> > anything and was looking only at the initial loads.
> 
> Ok, I wasn't talking about the synthetic test in the second part of my
> email above but about the explanation you gave about Galleryfling
> improvement (that the migration of a task with high utilization
> doesn't update the target frequency) which makes sense to me so we are
> on the same page about that.

Okay, great.

-- 
viresh


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-27 Thread Joel Fernandes (Google)
Hi Viresh,

On Wed, Jul 26, 2017 at 10:46 PM, Viresh Kumar  wrote:
> On 26-07-17, 22:14, Joel Fernandes (Google) wrote:

>> Also one more comment about this usecase:
>>
>> You mentioned in our discussion at [2] sometime back, about the
>> question of initial utilization,
>>
>> "We don't have any such configurable way possible right
>> now, but there were discussions on how much utilization should a new
>> task be assigned when it first comes up."
>
> We still initialize it to a value, just that it isn't configurable.
> See below..
>
>> But, then in your cover letter above, you mentioned "This is verified
>> using ftrace". So my question is how has this been verified with
>> ftrace if the new initial utilization as you said in [2] is currently
>> still under discussion? Basically how could you verify with ftrace
>> that the target CPU frequency isn't increasing immediately on spawning
>> of a new task remotely, if the initial utilization of a new task isn't
>> something we set/configure with current code? Am I missing something?
>>
>> [2] https://lists.linaro.org/pipermail/eas-dev/2017-January/000785.html
>
> The statement "new tasks should receive maximum demand initially" is
> used to represent tasks which have high demand every time they run.
> For example scrolling of a web page or gallery on our phones. Yes,
> maybe I can use the work "migrated" (as you suggested later) as the
> history of its utilization will move with it then to the new CPU.
>
> But even without that, if you see the routine
> init_entity_runnable_average() in fair.c, the new tasks are
> initialized in a way that they are seen as heavy tasks. And so even
> for the first time they run, freq should normally increase on the
> target CPU (at least with above application).i

Ok, but the "heavy" in init_entity_runnable_average means for load,
not the util_avg. The util_avg is what's used for frequency scaling
IIUC and is set to 0 in that function no?

>
> The application was written by Steve (all credit goes to him) before
> he left Linaro, but I did test it with ftrace. What I saw with ftrace
> was that the freq isn't reevaluated for almost an entire tick many
> times because we enqueued the task remotely. And that changes with
> this series.
>
>> > The reason being that this patchset only targets a corner case, where
>> > following are required to be true to improve performance and that
>> > doesn't happen too often with these tests:
>> >
>> > - Task is migrated to another CPU.
>> > - The task has maximum demand initially, and should take the CPU to
>>
>> Just to make the cover-letter more clear and also confirming with you
>> I understand the above usecase, maybe in the future this can reworded
>> from "initially" to "before the migration" and "take the CPU" to "take
>> the target CPU of the migration" ?
>
> I can reword it a bit, but the test case wasn't really migrating
> anything and was looking only at the initial loads.

Ok, I wasn't talking about the synthetic test in the second part of my
email above but about the explanation you gave about Galleryfling
improvement (that the migration of a task with high utilization
doesn't update the target frequency) which makes sense to me so we are
on the same page about that.

thanks,

-Joel


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-27 Thread Joel Fernandes (Google)
Hi Viresh,

On Wed, Jul 26, 2017 at 10:46 PM, Viresh Kumar  wrote:
> On 26-07-17, 22:14, Joel Fernandes (Google) wrote:

>> Also one more comment about this usecase:
>>
>> You mentioned in our discussion at [2] sometime back, about the
>> question of initial utilization,
>>
>> "We don't have any such configurable way possible right
>> now, but there were discussions on how much utilization should a new
>> task be assigned when it first comes up."
>
> We still initialize it to a value, just that it isn't configurable.
> See below..
>
>> But, then in your cover letter above, you mentioned "This is verified
>> using ftrace". So my question is how has this been verified with
>> ftrace if the new initial utilization as you said in [2] is currently
>> still under discussion? Basically how could you verify with ftrace
>> that the target CPU frequency isn't increasing immediately on spawning
>> of a new task remotely, if the initial utilization of a new task isn't
>> something we set/configure with current code? Am I missing something?
>>
>> [2] https://lists.linaro.org/pipermail/eas-dev/2017-January/000785.html
>
> The statement "new tasks should receive maximum demand initially" is
> used to represent tasks which have high demand every time they run.
> For example scrolling of a web page or gallery on our phones. Yes,
> maybe I can use the work "migrated" (as you suggested later) as the
> history of its utilization will move with it then to the new CPU.
>
> But even without that, if you see the routine
> init_entity_runnable_average() in fair.c, the new tasks are
> initialized in a way that they are seen as heavy tasks. And so even
> for the first time they run, freq should normally increase on the
> target CPU (at least with above application).i

Ok, but the "heavy" in init_entity_runnable_average means for load,
not the util_avg. The util_avg is what's used for frequency scaling
IIUC and is set to 0 in that function no?

>
> The application was written by Steve (all credit goes to him) before
> he left Linaro, but I did test it with ftrace. What I saw with ftrace
> was that the freq isn't reevaluated for almost an entire tick many
> times because we enqueued the task remotely. And that changes with
> this series.
>
>> > The reason being that this patchset only targets a corner case, where
>> > following are required to be true to improve performance and that
>> > doesn't happen too often with these tests:
>> >
>> > - Task is migrated to another CPU.
>> > - The task has maximum demand initially, and should take the CPU to
>>
>> Just to make the cover-letter more clear and also confirming with you
>> I understand the above usecase, maybe in the future this can reworded
>> from "initially" to "before the migration" and "take the CPU" to "take
>> the target CPU of the migration" ?
>
> I can reword it a bit, but the test case wasn't really migrating
> anything and was looking only at the initial loads.

Ok, I wasn't talking about the synthetic test in the second part of my
email above but about the explanation you gave about Galleryfling
improvement (that the migration of a task with high utilization
doesn't update the target frequency) which makes sense to me so we are
on the same page about that.

thanks,

-Joel


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-26 Thread Viresh Kumar
On 26-07-17, 22:14, Joel Fernandes (Google) wrote:
> I think you dropped [1] in your cover-letter. May be you meant to add
> it at the end of the cover letter?
> 
> I noticed from your v2 that its:
> https://pastebin.com/7LkMSRxE

Yeah, I missed it. Thanks :)

> Also one more comment about this usecase:
> 
> You mentioned in our discussion at [2] sometime back, about the
> question of initial utilization,
> 
> "We don't have any such configurable way possible right
> now, but there were discussions on how much utilization should a new
> task be assigned when it first comes up."

We still initialize it to a value, just that it isn't configurable.
See below..

> But, then in your cover letter above, you mentioned "This is verified
> using ftrace". So my question is how has this been verified with
> ftrace if the new initial utilization as you said in [2] is currently
> still under discussion? Basically how could you verify with ftrace
> that the target CPU frequency isn't increasing immediately on spawning
> of a new task remotely, if the initial utilization of a new task isn't
> something we set/configure with current code? Am I missing something?
> 
> [2] https://lists.linaro.org/pipermail/eas-dev/2017-January/000785.html

The statement "new tasks should receive maximum demand initially" is
used to represent tasks which have high demand every time they run.
For example scrolling of a web page or gallery on our phones. Yes,
maybe I can use the work "migrated" (as you suggested later) as the
history of its utilization will move with it then to the new CPU.

But even without that, if you see the routine
init_entity_runnable_average() in fair.c, the new tasks are
initialized in a way that they are seen as heavy tasks. And so even
for the first time they run, freq should normally increase on the
target CPU (at least with above application).

The application was written by Steve (all credit goes to him) before
he left Linaro, but I did test it with ftrace. What I saw with ftrace
was that the freq isn't reevaluated for almost an entire tick many
times because we enqueued the task remotely. And that changes with
this series.

> > The reason being that this patchset only targets a corner case, where
> > following are required to be true to improve performance and that
> > doesn't happen too often with these tests:
> >
> > - Task is migrated to another CPU.
> > - The task has maximum demand initially, and should take the CPU to
> 
> Just to make the cover-letter more clear and also confirming with you
> I understand the above usecase, maybe in the future this can reworded
> from "initially" to "before the migration" and "take the CPU" to "take
> the target CPU of the migration" ?

I can reword it a bit, but the test case wasn't really migrating
anything and was looking only at the initial loads.

> >   higher OPPs.
> > - And the target CPU doesn't call into schedutil until the next tick.
> 
> I found this usecase to be more plausible and can see this patch
> series being useful there.
> 
> Could you also keep me in CC on these patches (at joe...@google.com)?
> I'm interested in this series.

Sure.

-- 
viresh


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-26 Thread Viresh Kumar
On 26-07-17, 22:14, Joel Fernandes (Google) wrote:
> I think you dropped [1] in your cover-letter. May be you meant to add
> it at the end of the cover letter?
> 
> I noticed from your v2 that its:
> https://pastebin.com/7LkMSRxE

Yeah, I missed it. Thanks :)

> Also one more comment about this usecase:
> 
> You mentioned in our discussion at [2] sometime back, about the
> question of initial utilization,
> 
> "We don't have any such configurable way possible right
> now, but there were discussions on how much utilization should a new
> task be assigned when it first comes up."

We still initialize it to a value, just that it isn't configurable.
See below..

> But, then in your cover letter above, you mentioned "This is verified
> using ftrace". So my question is how has this been verified with
> ftrace if the new initial utilization as you said in [2] is currently
> still under discussion? Basically how could you verify with ftrace
> that the target CPU frequency isn't increasing immediately on spawning
> of a new task remotely, if the initial utilization of a new task isn't
> something we set/configure with current code? Am I missing something?
> 
> [2] https://lists.linaro.org/pipermail/eas-dev/2017-January/000785.html

The statement "new tasks should receive maximum demand initially" is
used to represent tasks which have high demand every time they run.
For example scrolling of a web page or gallery on our phones. Yes,
maybe I can use the work "migrated" (as you suggested later) as the
history of its utilization will move with it then to the new CPU.

But even without that, if you see the routine
init_entity_runnable_average() in fair.c, the new tasks are
initialized in a way that they are seen as heavy tasks. And so even
for the first time they run, freq should normally increase on the
target CPU (at least with above application).

The application was written by Steve (all credit goes to him) before
he left Linaro, but I did test it with ftrace. What I saw with ftrace
was that the freq isn't reevaluated for almost an entire tick many
times because we enqueued the task remotely. And that changes with
this series.

> > The reason being that this patchset only targets a corner case, where
> > following are required to be true to improve performance and that
> > doesn't happen too often with these tests:
> >
> > - Task is migrated to another CPU.
> > - The task has maximum demand initially, and should take the CPU to
> 
> Just to make the cover-letter more clear and also confirming with you
> I understand the above usecase, maybe in the future this can reworded
> from "initially" to "before the migration" and "take the CPU" to "take
> the target CPU of the migration" ?

I can reword it a bit, but the test case wasn't really migrating
anything and was looking only at the initial loads.

> >   higher OPPs.
> > - And the target CPU doesn't call into schedutil until the next tick.
> 
> I found this usecase to be more plausible and can see this patch
> series being useful there.
> 
> Could you also keep me in CC on these patches (at joe...@google.com)?
> I'm interested in this series.

Sure.

-- 
viresh


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-26 Thread Joel Fernandes (Google)
Hi Viresh,

On Wed, Jul 26, 2017 at 2:22 AM, Viresh Kumar  wrote:

>
> With Android UI and benchmarks the latency of cpufreq response to
> certain scheduling events can become very critical. Currently, callbacks
> into schedutil are only made from the scheduler if the target CPU of the
> event is the same as the current CPU. This means there are certain
> situations where a target CPU may not run schedutil for some time.
>
> One testcase to show this behavior is where a task starts running on
> CPU0, then a new task is also spawned on CPU0 by a task on CPU1. If the
> system is configured such that new tasks should receive maximum demand
> initially, this should result in CPU0 increasing frequency immediately.
> Because of the above mentioned limitation though this does not occur.
> This is verified using ftrace with the sample [1] application.

I think you dropped [1] in your cover-letter. May be you meant to add
it at the end of the cover letter?

I noticed from your v2 that its:
https://pastebin.com/7LkMSRxE

Also one more comment about this usecase:

You mentioned in our discussion at [2] sometime back, about the
question of initial utilization,

"We don't have any such configurable way possible right
now, but there were discussions on how much utilization should a new
task be assigned when it first comes up."

But, then in your cover letter above, you mentioned "This is verified
using ftrace". So my question is how has this been verified with
ftrace if the new initial utilization as you said in [2] is currently
still under discussion? Basically how could you verify with ftrace
that the target CPU frequency isn't increasing immediately on spawning
of a new task remotely, if the initial utilization of a new task isn't
something we set/configure with current code? Am I missing something?

[2] https://lists.linaro.org/pipermail/eas-dev/2017-January/000785.html

>
> Maybe the ideal solution is to always allow remote callbacks but that
> has its own challenges:
>
> o There is no protection required for single CPU per policy case today,
>   and adding any kind of locking there, to supply remote callbacks,
>   isn't really a good idea.
>
> o If is local CPU isn't part of the same cpufreq policy as the target
>   CPU, then we wouldn't be able to do fast switching at all and have to
>   use some kind of bottom half to schedule work on the target CPU to do
>   real switching. That may be overkill as well.
>
>
> And so this series only allows remote callbacks for target CPUs that
> share the cpufreq policy with the local CPU.
>
> This series is tested with couple of usecases (Android: hackbench,
> recentfling, galleryfling, vellamo, Ubuntu: hackbench) on ARM hikey
> board (64 bit octa-core, single policy). Only galleryfling showed minor
> improvements, while others didn't had much deviation.
>
> The reason being that this patchset only targets a corner case, where
> following are required to be true to improve performance and that
> doesn't happen too often with these tests:
>
> - Task is migrated to another CPU.
> - The task has maximum demand initially, and should take the CPU to

Just to make the cover-letter more clear and also confirming with you
I understand the above usecase, maybe in the future this can reworded
from "initially" to "before the migration" and "take the CPU" to "take
the target CPU of the migration" ?

>   higher OPPs.
> - And the target CPU doesn't call into schedutil until the next tick.

I found this usecase to be more plausible and can see this patch
series being useful there.

Could you also keep me in CC on these patches (at joe...@google.com)?
I'm interested in this series.

thanks!

-Joel


Re: [Eas-dev] [PATCH V4 0/3] sched: cpufreq: Allow remote callbacks

2017-07-26 Thread Joel Fernandes (Google)
Hi Viresh,

On Wed, Jul 26, 2017 at 2:22 AM, Viresh Kumar  wrote:

>
> With Android UI and benchmarks the latency of cpufreq response to
> certain scheduling events can become very critical. Currently, callbacks
> into schedutil are only made from the scheduler if the target CPU of the
> event is the same as the current CPU. This means there are certain
> situations where a target CPU may not run schedutil for some time.
>
> One testcase to show this behavior is where a task starts running on
> CPU0, then a new task is also spawned on CPU0 by a task on CPU1. If the
> system is configured such that new tasks should receive maximum demand
> initially, this should result in CPU0 increasing frequency immediately.
> Because of the above mentioned limitation though this does not occur.
> This is verified using ftrace with the sample [1] application.

I think you dropped [1] in your cover-letter. May be you meant to add
it at the end of the cover letter?

I noticed from your v2 that its:
https://pastebin.com/7LkMSRxE

Also one more comment about this usecase:

You mentioned in our discussion at [2] sometime back, about the
question of initial utilization,

"We don't have any such configurable way possible right
now, but there were discussions on how much utilization should a new
task be assigned when it first comes up."

But, then in your cover letter above, you mentioned "This is verified
using ftrace". So my question is how has this been verified with
ftrace if the new initial utilization as you said in [2] is currently
still under discussion? Basically how could you verify with ftrace
that the target CPU frequency isn't increasing immediately on spawning
of a new task remotely, if the initial utilization of a new task isn't
something we set/configure with current code? Am I missing something?

[2] https://lists.linaro.org/pipermail/eas-dev/2017-January/000785.html

>
> Maybe the ideal solution is to always allow remote callbacks but that
> has its own challenges:
>
> o There is no protection required for single CPU per policy case today,
>   and adding any kind of locking there, to supply remote callbacks,
>   isn't really a good idea.
>
> o If is local CPU isn't part of the same cpufreq policy as the target
>   CPU, then we wouldn't be able to do fast switching at all and have to
>   use some kind of bottom half to schedule work on the target CPU to do
>   real switching. That may be overkill as well.
>
>
> And so this series only allows remote callbacks for target CPUs that
> share the cpufreq policy with the local CPU.
>
> This series is tested with couple of usecases (Android: hackbench,
> recentfling, galleryfling, vellamo, Ubuntu: hackbench) on ARM hikey
> board (64 bit octa-core, single policy). Only galleryfling showed minor
> improvements, while others didn't had much deviation.
>
> The reason being that this patchset only targets a corner case, where
> following are required to be true to improve performance and that
> doesn't happen too often with these tests:
>
> - Task is migrated to another CPU.
> - The task has maximum demand initially, and should take the CPU to

Just to make the cover-letter more clear and also confirming with you
I understand the above usecase, maybe in the future this can reworded
from "initially" to "before the migration" and "take the CPU" to "take
the target CPU of the migration" ?

>   higher OPPs.
> - And the target CPU doesn't call into schedutil until the next tick.

I found this usecase to be more plausible and can see this patch
series being useful there.

Could you also keep me in CC on these patches (at joe...@google.com)?
I'm interested in this series.

thanks!

-Joel