Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Tue, Jul 29, 2014 at 02:12:37PM +0200, Peter Zijlstra wrote:
> On Mon, Jul 28, 2014 at 07:37:33PM +0200, Frederic Weisbecker wrote:
> > The timekeeper gets initialized to the value of the CPU where the
> > first clockevent device is setup. This works well because the timekeeper
> > can be any online CPU in most configs.
> >
> > Full dynticks has its own requirement though and needs the timekeeper
> > to always be 0. And this requirement seem to accomodate pretty well with
> > the above described boot timekeeper setting because the first clockevent
> > device happens to be initialized, most of the time, on the boot CPU
> > (which should be CPU 0).
>
> This isn't true in general, Voyager (which we dropped support for iirc)
> had a boot cpu != 0, and I think there's ARM platforms where the same
> can be true.
Ah I didn't know that. But I heard before that assuming CPU0 as the boot
CPU is a mistake.
>
> > However there is no mention of such a guarantee anywhere. This assumption
> > might well be defeated on some corner case now or in the future.
>
> Right..
>
> > So lets wipe out the FUD and force tick_do_timer_cpu to CPU 0 on boot
> > when full dynticks is used.
> >
> > This way we can even remove some corner case code that handled scenarios
> > where all clockevent devices were setup on full dynticks CPUs.
>
> > diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
> > index 0a0608e..cb57bce 100644
> > --- a/kernel/time/tick-common.c
> > +++ b/kernel/time/tick-common.c
> > @@ -179,10 +179,10 @@ static void tick_setup_device(struct tick_device *td,
> > * this cpu:
> > */
> > if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
> > - if (!tick_nohz_full_cpu(cpu))
> > - tick_do_timer_cpu = cpu;
> > + if (tick_nohz_full_enabled())
> > + tick_do_timer_cpu = TICK_DO_TIMER_DEFAULT;
> > else
> > - tick_do_timer_cpu = TICK_DO_TIMER_NONE;
> > + tick_do_timer_cpu = cpu;
> > tick_next_period = ktime_get();
> > tick_period = ktime_set(0, NSEC_PER_SEC / HZ);
> > }
>
> So from what I can tell this code can get called before SMP setup, which
> would mean we could get here before CPU0 is online?
If CPU0 is a secondary CPU then yeah we have a problem.
Look, the most important for me is to determine the boot CPU and make sure
we don't need to offline it for suspend.
Looking at disable_nonboot_cpus(), it seems that the lowest number from the
online set
is considered the boot CPU. But if CPU0 boots after the boot CPU, this
assumption is defeated,
right?
Probably disable_nonboot_cpus() doesn't look for the real boot CPU but
rather it just tries to keep at least one CPU alive for stopping and resuming
the
suspend process.
It looks like the only sane thing I can do is:
1) Pick up the boot CPU (the one that calls tick_nohz_init() for example) as the
default timekeeper.
2) Let it be offlined but then handover its timekeeping duty to any online CPU,
whether full dynticks or not (of course then the full dynticks timekeeper won't
be
ever allowed to enter in full dynticks mode), we don't have the choice anyway.
But then we need a way to reset the sysidle state when we handover the
timekeeping
duty.
Big headaches on the horizon!
Why haven't we kept CPU0 as the boot CPU for everyone? Things were just simple
that way...
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Tue, Jul 29, 2014 at 02:12:37PM +0200, Peter Zijlstra wrote:
On Mon, Jul 28, 2014 at 07:37:33PM +0200, Frederic Weisbecker wrote:
The timekeeper gets initialized to the value of the CPU where the
first clockevent device is setup. This works well because the timekeeper
can be any online CPU in most configs.
Full dynticks has its own requirement though and needs the timekeeper
to always be 0. And this requirement seem to accomodate pretty well with
the above described boot timekeeper setting because the first clockevent
device happens to be initialized, most of the time, on the boot CPU
(which should be CPU 0).
This isn't true in general, Voyager (which we dropped support for iirc)
had a boot cpu != 0, and I think there's ARM platforms where the same
can be true.
Ah I didn't know that. But I heard before that assuming CPU0 as the boot
CPU is a mistake.
However there is no mention of such a guarantee anywhere. This assumption
might well be defeated on some corner case now or in the future.
Right..
So lets wipe out the FUD and force tick_do_timer_cpu to CPU 0 on boot
when full dynticks is used.
This way we can even remove some corner case code that handled scenarios
where all clockevent devices were setup on full dynticks CPUs.
diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
index 0a0608e..cb57bce 100644
--- a/kernel/time/tick-common.c
+++ b/kernel/time/tick-common.c
@@ -179,10 +179,10 @@ static void tick_setup_device(struct tick_device *td,
* this cpu:
*/
if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
- if (!tick_nohz_full_cpu(cpu))
- tick_do_timer_cpu = cpu;
+ if (tick_nohz_full_enabled())
+ tick_do_timer_cpu = TICK_DO_TIMER_DEFAULT;
else
- tick_do_timer_cpu = TICK_DO_TIMER_NONE;
+ tick_do_timer_cpu = cpu;
tick_next_period = ktime_get();
tick_period = ktime_set(0, NSEC_PER_SEC / HZ);
}
So from what I can tell this code can get called before SMP setup, which
would mean we could get here before CPU0 is online?
If CPU0 is a secondary CPU then yeah we have a problem.
Look, the most important for me is to determine the boot CPU and make sure
we don't need to offline it for suspend.
Looking at disable_nonboot_cpus(), it seems that the lowest number from the
online set
is considered the boot CPU. But if CPU0 boots after the boot CPU, this
assumption is defeated,
right?
Probably disable_nonboot_cpus() doesn't look for the real boot CPU but
rather it just tries to keep at least one CPU alive for stopping and resuming
the
suspend process.
It looks like the only sane thing I can do is:
1) Pick up the boot CPU (the one that calls tick_nohz_init() for example) as the
default timekeeper.
2) Let it be offlined but then handover its timekeeping duty to any online CPU,
whether full dynticks or not (of course then the full dynticks timekeeper won't
be
ever allowed to enter in full dynticks mode), we don't have the choice anyway.
But then we need a way to reset the sysidle state when we handover the
timekeeping
duty.
Big headaches on the horizon!
Why haven't we kept CPU0 as the boot CPU for everyone? Things were just simple
that way...
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Mon, Jul 28, 2014 at 07:37:33PM +0200, Frederic Weisbecker wrote:
> The timekeeper gets initialized to the value of the CPU where the
> first clockevent device is setup. This works well because the timekeeper
> can be any online CPU in most configs.
>
> Full dynticks has its own requirement though and needs the timekeeper
> to always be 0. And this requirement seem to accomodate pretty well with
> the above described boot timekeeper setting because the first clockevent
> device happens to be initialized, most of the time, on the boot CPU
> (which should be CPU 0).
This isn't true in general, Voyager (which we dropped support for iirc)
had a boot cpu != 0, and I think there's ARM platforms where the same
can be true.
> However there is no mention of such a guarantee anywhere. This assumption
> might well be defeated on some corner case now or in the future.
Right..
> So lets wipe out the FUD and force tick_do_timer_cpu to CPU 0 on boot
> when full dynticks is used.
>
> This way we can even remove some corner case code that handled scenarios
> where all clockevent devices were setup on full dynticks CPUs.
> diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
> index 0a0608e..cb57bce 100644
> --- a/kernel/time/tick-common.c
> +++ b/kernel/time/tick-common.c
> @@ -179,10 +179,10 @@ static void tick_setup_device(struct tick_device *td,
>* this cpu:
>*/
> if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
> - if (!tick_nohz_full_cpu(cpu))
> - tick_do_timer_cpu = cpu;
> + if (tick_nohz_full_enabled())
> + tick_do_timer_cpu = TICK_DO_TIMER_DEFAULT;
> else
> - tick_do_timer_cpu = TICK_DO_TIMER_NONE;
> + tick_do_timer_cpu = cpu;
> tick_next_period = ktime_get();
> tick_period = ktime_set(0, NSEC_PER_SEC / HZ);
> }
So from what I can tell this code can get called before SMP setup, which
would mean we could get here before CPU0 is online?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Mon, Jul 28, 2014 at 07:37:33PM +0200, Frederic Weisbecker wrote:
The timekeeper gets initialized to the value of the CPU where the
first clockevent device is setup. This works well because the timekeeper
can be any online CPU in most configs.
Full dynticks has its own requirement though and needs the timekeeper
to always be 0. And this requirement seem to accomodate pretty well with
the above described boot timekeeper setting because the first clockevent
device happens to be initialized, most of the time, on the boot CPU
(which should be CPU 0).
This isn't true in general, Voyager (which we dropped support for iirc)
had a boot cpu != 0, and I think there's ARM platforms where the same
can be true.
However there is no mention of such a guarantee anywhere. This assumption
might well be defeated on some corner case now or in the future.
Right..
So lets wipe out the FUD and force tick_do_timer_cpu to CPU 0 on boot
when full dynticks is used.
This way we can even remove some corner case code that handled scenarios
where all clockevent devices were setup on full dynticks CPUs.
diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
index 0a0608e..cb57bce 100644
--- a/kernel/time/tick-common.c
+++ b/kernel/time/tick-common.c
@@ -179,10 +179,10 @@ static void tick_setup_device(struct tick_device *td,
* this cpu:
*/
if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
- if (!tick_nohz_full_cpu(cpu))
- tick_do_timer_cpu = cpu;
+ if (tick_nohz_full_enabled())
+ tick_do_timer_cpu = TICK_DO_TIMER_DEFAULT;
else
- tick_do_timer_cpu = TICK_DO_TIMER_NONE;
+ tick_do_timer_cpu = cpu;
tick_next_period = ktime_get();
tick_period = ktime_set(0, NSEC_PER_SEC / HZ);
}
So from what I can tell this code can get called before SMP setup, which
would mean we could get here before CPU0 is online?
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH 07/10] nohz: Enforce timekeeping on CPU 0
The timekeeper gets initialized to the value of the CPU where the
first clockevent device is setup. This works well because the timekeeper
can be any online CPU in most configs.
Full dynticks has its own requirement though and needs the timekeeper
to always be 0. And this requirement seem to accomodate pretty well with
the above described boot timekeeper setting because the first clockevent
device happens to be initialized, most of the time, on the boot CPU
(which should be CPU 0).
However there is no mention of such a guarantee anywhere. This assumption
might well be defeated on some corner case now or in the future.
So lets wipe out the FUD and force tick_do_timer_cpu to CPU 0 on boot
when full dynticks is used.
This way we can even remove some corner case code that handled scenarios
where all clockevent devices were setup on full dynticks CPUs.
Cc: Ingo Molnar
Cc: Nicolas Pitre
Cc: Paul E. McKenney
Cc: Peter Zijlstra
Cc: Steven Rostedt
Cc: Thomas Gleixner
Cc: Viresh Kumar
Signed-off-by: Frederic Weisbecker
---
kernel/time/tick-common.c | 6 +++---
kernel/time/tick-sched.c | 6 --
2 files changed, 3 insertions(+), 9 deletions(-)
diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
index 0a0608e..cb57bce 100644
--- a/kernel/time/tick-common.c
+++ b/kernel/time/tick-common.c
@@ -179,10 +179,10 @@ static void tick_setup_device(struct tick_device *td,
* this cpu:
*/
if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
- if (!tick_nohz_full_cpu(cpu))
- tick_do_timer_cpu = cpu;
+ if (tick_nohz_full_enabled())
+ tick_do_timer_cpu = TICK_DO_TIMER_DEFAULT;
else
- tick_do_timer_cpu = TICK_DO_TIMER_NONE;
+ tick_do_timer_cpu = cpu;
tick_next_period = ktime_get();
tick_period = ktime_set(0, NSEC_PER_SEC / HZ);
}
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 3d63944..2ea2143 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -741,12 +741,6 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched
*ts)
*/
if (tick_do_timer_cpu == cpu)
return false;
- /*
-* Boot safety: make sure the timekeeping duty has been
-* assigned before entering dyntick-idle mode,
-*/
- if (tick_do_timer_cpu == TICK_DO_TIMER_NONE)
- return false;
}
return true;
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH 07/10] nohz: Enforce timekeeping on CPU 0
The timekeeper gets initialized to the value of the CPU where the
first clockevent device is setup. This works well because the timekeeper
can be any online CPU in most configs.
Full dynticks has its own requirement though and needs the timekeeper
to always be 0. And this requirement seem to accomodate pretty well with
the above described boot timekeeper setting because the first clockevent
device happens to be initialized, most of the time, on the boot CPU
(which should be CPU 0).
However there is no mention of such a guarantee anywhere. This assumption
might well be defeated on some corner case now or in the future.
So lets wipe out the FUD and force tick_do_timer_cpu to CPU 0 on boot
when full dynticks is used.
This way we can even remove some corner case code that handled scenarios
where all clockevent devices were setup on full dynticks CPUs.
Cc: Ingo Molnar mi...@kernel.org
Cc: Nicolas Pitre nicolas.pi...@linaro.org
Cc: Paul E. McKenney paul...@linux.vnet.ibm.com
Cc: Peter Zijlstra pet...@infradead.org
Cc: Steven Rostedt rost...@goodmis.org
Cc: Thomas Gleixner t...@linutronix.de
Cc: Viresh Kumar viresh.ku...@linaro.org
Signed-off-by: Frederic Weisbecker fweis...@gmail.com
---
kernel/time/tick-common.c | 6 +++---
kernel/time/tick-sched.c | 6 --
2 files changed, 3 insertions(+), 9 deletions(-)
diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
index 0a0608e..cb57bce 100644
--- a/kernel/time/tick-common.c
+++ b/kernel/time/tick-common.c
@@ -179,10 +179,10 @@ static void tick_setup_device(struct tick_device *td,
* this cpu:
*/
if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
- if (!tick_nohz_full_cpu(cpu))
- tick_do_timer_cpu = cpu;
+ if (tick_nohz_full_enabled())
+ tick_do_timer_cpu = TICK_DO_TIMER_DEFAULT;
else
- tick_do_timer_cpu = TICK_DO_TIMER_NONE;
+ tick_do_timer_cpu = cpu;
tick_next_period = ktime_get();
tick_period = ktime_set(0, NSEC_PER_SEC / HZ);
}
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 3d63944..2ea2143 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -741,12 +741,6 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched
*ts)
*/
if (tick_do_timer_cpu == cpu)
return false;
- /*
-* Boot safety: make sure the timekeeping duty has been
-* assigned before entering dyntick-idle mode,
-*/
- if (tick_do_timer_cpu == TICK_DO_TIMER_NONE)
- return false;
}
return true;
--
1.8.3.1
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, Jul 19, 2014 at 01:31:25PM -0400, Nicolas Pitre wrote: > On Sat, 19 Jul 2014, Frederic Weisbecker wrote: > > > The timekeeper gets initialized to the value of the CPU where the > > first clockevent device is setup. This works well because the timekeeper > > can be any online CPU in most configs. > > > > Full dynticks has its own requirement though and needs the timekeeper > > to always be 0. And this requirement seem to accomodate pretty well with > > the above described boot timekeeper setting because the first clockevent > > device happens to be initialized, most of the time, on the boot CPU > > (which should be CPU 0). > > This might have been discussed before... but this isn't ARM big.LITTLE > friendly at all. > > Could we accommodate for any arbitrary CPU instead of making CPU 0 > "special" other than its role as the boot CPU please? It doesn't have > to be completely dynamic, but CPU 0 might be a really bad choice for > ongoing periodic duties in some configurations. For example, we might > highly prefer to do this on CPU 4 for power efficiency reasons once it > is online and keep CPU 0 in a deep C-state as much as possible. I can certainly arrange for setting user defined timekeeper on boot time. Just one constraint: since the timekeeper is fixed, we can't offline it. That's usually fine with CPU 0 but on other CPUs, rejecting a CPU offlining blocks the suspend process. I imagine I can handover the timekeeping to another CPU in this case and accept the offlining. But there are chances that the only online CPUs remaining are full dynticks and it's not tempting to give them a timekeeping duty. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, Jul 19, 2014 at 02:46:56PM -0400, Nicolas Pitre wrote: > People think all sorts of things. And it becomes very irritating when > thoughtful assumptions get burned into ROM for example. We should be > able to do better in the kernel. Agreed. There used to be an x86 subarch where the boot cpu wasn't cpu0, that always gave a lot of 'joy' too :-) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, 19 Jul 2014, Peter Zijlstra wrote: > On Sat, Jul 19, 2014 at 01:31:25PM -0400, Nicolas Pitre wrote: > > On Sat, 19 Jul 2014, Frederic Weisbecker wrote: > > > > > The timekeeper gets initialized to the value of the CPU where the > > > first clockevent device is setup. This works well because the timekeeper > > > can be any online CPU in most configs. > > > > > > Full dynticks has its own requirement though and needs the timekeeper > > > to always be 0. And this requirement seem to accomodate pretty well with > > > the above described boot timekeeper setting because the first clockevent > > > device happens to be initialized, most of the time, on the boot CPU > > > (which should be CPU 0). > > > > This might have been discussed before... but this isn't ARM big.LITTLE > > friendly at all. > > > > Could we accommodate for any arbitrary CPU instead of making CPU 0 > > "special" other than its role as the boot CPU please? It doesn't have > > to be completely dynamic, but CPU 0 might be a really bad choice for > > ongoing periodic duties in some configurations. For example, we might > > highly prefer to do this on CPU 4 for power efficiency reasons once it > > is online and keep CPU 0 in a deep C-state as much as possible. > > This is because CPU0 can be a big core, right? IIRC this is done because > a big core as boot cpu, boots faster and some people think boot time is > relevant. People think all sorts of things. And it becomes very irritating when thoughtful assumptions get burned into ROM for example. We should be able to do better in the kernel. Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, Jul 19, 2014 at 01:31:25PM -0400, Nicolas Pitre wrote: > On Sat, 19 Jul 2014, Frederic Weisbecker wrote: > > > The timekeeper gets initialized to the value of the CPU where the > > first clockevent device is setup. This works well because the timekeeper > > can be any online CPU in most configs. > > > > Full dynticks has its own requirement though and needs the timekeeper > > to always be 0. And this requirement seem to accomodate pretty well with > > the above described boot timekeeper setting because the first clockevent > > device happens to be initialized, most of the time, on the boot CPU > > (which should be CPU 0). > > This might have been discussed before... but this isn't ARM big.LITTLE > friendly at all. > > Could we accommodate for any arbitrary CPU instead of making CPU 0 > "special" other than its role as the boot CPU please? It doesn't have > to be completely dynamic, but CPU 0 might be a really bad choice for > ongoing periodic duties in some configurations. For example, we might > highly prefer to do this on CPU 4 for power efficiency reasons once it > is online and keep CPU 0 in a deep C-state as much as possible. This is because CPU0 can be a big core, right? IIRC this is done because a big core as boot cpu, boots faster and some people think boot time is relevant. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, 19 Jul 2014, Frederic Weisbecker wrote: > The timekeeper gets initialized to the value of the CPU where the > first clockevent device is setup. This works well because the timekeeper > can be any online CPU in most configs. > > Full dynticks has its own requirement though and needs the timekeeper > to always be 0. And this requirement seem to accomodate pretty well with > the above described boot timekeeper setting because the first clockevent > device happens to be initialized, most of the time, on the boot CPU > (which should be CPU 0). This might have been discussed before... but this isn't ARM big.LITTLE friendly at all. Could we accommodate for any arbitrary CPU instead of making CPU 0 "special" other than its role as the boot CPU please? It doesn't have to be completely dynamic, but CPU 0 might be a really bad choice for ongoing periodic duties in some configurations. For example, we might highly prefer to do this on CPU 4 for power efficiency reasons once it is online and keep CPU 0 in a deep C-state as much as possible. Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, 19 Jul 2014, Frederic Weisbecker wrote: The timekeeper gets initialized to the value of the CPU where the first clockevent device is setup. This works well because the timekeeper can be any online CPU in most configs. Full dynticks has its own requirement though and needs the timekeeper to always be 0. And this requirement seem to accomodate pretty well with the above described boot timekeeper setting because the first clockevent device happens to be initialized, most of the time, on the boot CPU (which should be CPU 0). This might have been discussed before... but this isn't ARM big.LITTLE friendly at all. Could we accommodate for any arbitrary CPU instead of making CPU 0 special other than its role as the boot CPU please? It doesn't have to be completely dynamic, but CPU 0 might be a really bad choice for ongoing periodic duties in some configurations. For example, we might highly prefer to do this on CPU 4 for power efficiency reasons once it is online and keep CPU 0 in a deep C-state as much as possible. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, Jul 19, 2014 at 01:31:25PM -0400, Nicolas Pitre wrote: On Sat, 19 Jul 2014, Frederic Weisbecker wrote: The timekeeper gets initialized to the value of the CPU where the first clockevent device is setup. This works well because the timekeeper can be any online CPU in most configs. Full dynticks has its own requirement though and needs the timekeeper to always be 0. And this requirement seem to accomodate pretty well with the above described boot timekeeper setting because the first clockevent device happens to be initialized, most of the time, on the boot CPU (which should be CPU 0). This might have been discussed before... but this isn't ARM big.LITTLE friendly at all. Could we accommodate for any arbitrary CPU instead of making CPU 0 special other than its role as the boot CPU please? It doesn't have to be completely dynamic, but CPU 0 might be a really bad choice for ongoing periodic duties in some configurations. For example, we might highly prefer to do this on CPU 4 for power efficiency reasons once it is online and keep CPU 0 in a deep C-state as much as possible. This is because CPU0 can be a big core, right? IIRC this is done because a big core as boot cpu, boots faster and some people think boot time is relevant. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, 19 Jul 2014, Peter Zijlstra wrote: On Sat, Jul 19, 2014 at 01:31:25PM -0400, Nicolas Pitre wrote: On Sat, 19 Jul 2014, Frederic Weisbecker wrote: The timekeeper gets initialized to the value of the CPU where the first clockevent device is setup. This works well because the timekeeper can be any online CPU in most configs. Full dynticks has its own requirement though and needs the timekeeper to always be 0. And this requirement seem to accomodate pretty well with the above described boot timekeeper setting because the first clockevent device happens to be initialized, most of the time, on the boot CPU (which should be CPU 0). This might have been discussed before... but this isn't ARM big.LITTLE friendly at all. Could we accommodate for any arbitrary CPU instead of making CPU 0 special other than its role as the boot CPU please? It doesn't have to be completely dynamic, but CPU 0 might be a really bad choice for ongoing periodic duties in some configurations. For example, we might highly prefer to do this on CPU 4 for power efficiency reasons once it is online and keep CPU 0 in a deep C-state as much as possible. This is because CPU0 can be a big core, right? IIRC this is done because a big core as boot cpu, boots faster and some people think boot time is relevant. People think all sorts of things. And it becomes very irritating when thoughtful assumptions get burned into ROM for example. We should be able to do better in the kernel. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, Jul 19, 2014 at 02:46:56PM -0400, Nicolas Pitre wrote: People think all sorts of things. And it becomes very irritating when thoughtful assumptions get burned into ROM for example. We should be able to do better in the kernel. Agreed. There used to be an x86 subarch where the boot cpu wasn't cpu0, that always gave a lot of 'joy' too :-) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 07/10] nohz: Enforce timekeeping on CPU 0
On Sat, Jul 19, 2014 at 01:31:25PM -0400, Nicolas Pitre wrote: On Sat, 19 Jul 2014, Frederic Weisbecker wrote: The timekeeper gets initialized to the value of the CPU where the first clockevent device is setup. This works well because the timekeeper can be any online CPU in most configs. Full dynticks has its own requirement though and needs the timekeeper to always be 0. And this requirement seem to accomodate pretty well with the above described boot timekeeper setting because the first clockevent device happens to be initialized, most of the time, on the boot CPU (which should be CPU 0). This might have been discussed before... but this isn't ARM big.LITTLE friendly at all. Could we accommodate for any arbitrary CPU instead of making CPU 0 special other than its role as the boot CPU please? It doesn't have to be completely dynamic, but CPU 0 might be a really bad choice for ongoing periodic duties in some configurations. For example, we might highly prefer to do this on CPU 4 for power efficiency reasons once it is online and keep CPU 0 in a deep C-state as much as possible. I can certainly arrange for setting user defined timekeeper on boot time. Just one constraint: since the timekeeper is fixed, we can't offline it. That's usually fine with CPU 0 but on other CPUs, rejecting a CPU offlining blocks the suspend process. I imagine I can handover the timekeeping to another CPU in this case and accept the offlining. But there are chances that the only online CPUs remaining are full dynticks and it's not tempting to give them a timekeeping duty. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 07/10] nohz: Enforce timekeeping on CPU 0
The timekeeper gets initialized to the value of the CPU where the
first clockevent device is setup. This works well because the timekeeper
can be any online CPU in most configs.
Full dynticks has its own requirement though and needs the timekeeper
to always be 0. And this requirement seem to accomodate pretty well with
the above described boot timekeeper setting because the first clockevent
device happens to be initialized, most of the time, on the boot CPU
(which should be CPU 0).
However there is no mention of such a guarantee anywhere. This assumption
might well be defeated on some corner case now or in the future.
So lets wipe out the FUD and force tick_do_timer_cpu to CPU 0 on boot
when full dynticks is used.
This way we can even remove some corner case code that handled scenarios
where all clockevent devices were setup on full dynticks CPUs.
Cc: Ingo Molnar
Cc: Paul E. McKenney
Cc: Peter Zijlstra
Cc: Steven Rostedt
Cc: Thomas Gleixner
Cc: Viresh Kumar
Signed-off-by: Frederic Weisbecker
---
kernel/time/tick-common.c | 6 +++---
kernel/time/tick-sched.c | 6 --
2 files changed, 3 insertions(+), 9 deletions(-)
diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
index 0a0608e..cb57bce 100644
--- a/kernel/time/tick-common.c
+++ b/kernel/time/tick-common.c
@@ -179,10 +179,10 @@ static void tick_setup_device(struct tick_device *td,
* this cpu:
*/
if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
- if (!tick_nohz_full_cpu(cpu))
- tick_do_timer_cpu = cpu;
+ if (tick_nohz_full_enabled())
+ tick_do_timer_cpu = TICK_DO_TIMER_DEFAULT;
else
- tick_do_timer_cpu = TICK_DO_TIMER_NONE;
+ tick_do_timer_cpu = cpu;
tick_next_period = ktime_get();
tick_period = ktime_set(0, NSEC_PER_SEC / HZ);
}
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 3d63944..2ea2143 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -741,12 +741,6 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched
*ts)
*/
if (tick_do_timer_cpu == cpu)
return false;
- /*
-* Boot safety: make sure the timekeeping duty has been
-* assigned before entering dyntick-idle mode,
-*/
- if (tick_do_timer_cpu == TICK_DO_TIMER_NONE)
- return false;
}
return true;
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH 07/10] nohz: Enforce timekeeping on CPU 0
The timekeeper gets initialized to the value of the CPU where the
first clockevent device is setup. This works well because the timekeeper
can be any online CPU in most configs.
Full dynticks has its own requirement though and needs the timekeeper
to always be 0. And this requirement seem to accomodate pretty well with
the above described boot timekeeper setting because the first clockevent
device happens to be initialized, most of the time, on the boot CPU
(which should be CPU 0).
However there is no mention of such a guarantee anywhere. This assumption
might well be defeated on some corner case now or in the future.
So lets wipe out the FUD and force tick_do_timer_cpu to CPU 0 on boot
when full dynticks is used.
This way we can even remove some corner case code that handled scenarios
where all clockevent devices were setup on full dynticks CPUs.
Cc: Ingo Molnar mi...@kernel.org
Cc: Paul E. McKenney paul...@linux.vnet.ibm.com
Cc: Peter Zijlstra pet...@infradead.org
Cc: Steven Rostedt rost...@goodmis.org
Cc: Thomas Gleixner t...@linutronix.de
Cc: Viresh Kumar viresh.ku...@linaro.org
Signed-off-by: Frederic Weisbecker fweis...@gmail.com
---
kernel/time/tick-common.c | 6 +++---
kernel/time/tick-sched.c | 6 --
2 files changed, 3 insertions(+), 9 deletions(-)
diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
index 0a0608e..cb57bce 100644
--- a/kernel/time/tick-common.c
+++ b/kernel/time/tick-common.c
@@ -179,10 +179,10 @@ static void tick_setup_device(struct tick_device *td,
* this cpu:
*/
if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
- if (!tick_nohz_full_cpu(cpu))
- tick_do_timer_cpu = cpu;
+ if (tick_nohz_full_enabled())
+ tick_do_timer_cpu = TICK_DO_TIMER_DEFAULT;
else
- tick_do_timer_cpu = TICK_DO_TIMER_NONE;
+ tick_do_timer_cpu = cpu;
tick_next_period = ktime_get();
tick_period = ktime_set(0, NSEC_PER_SEC / HZ);
}
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 3d63944..2ea2143 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -741,12 +741,6 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched
*ts)
*/
if (tick_do_timer_cpu == cpu)
return false;
- /*
-* Boot safety: make sure the timekeeping duty has been
-* assigned before entering dyntick-idle mode,
-*/
- if (tick_do_timer_cpu == TICK_DO_TIMER_NONE)
- return false;
}
return true;
--
1.8.3.1
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
