Re: [PATCH 6/6] sched/isolation: Update nohz documentation to explain tick offload

[Frederic Weisbecker]

On Sat, Feb 17, 2018 at 11:53:22AM +0100, Thomas Gleixner wrote:
> On Thu, 15 Feb 2018, Frederic Weisbecker wrote:
> 
> > Update the documentation to reflect the 1Hz tick offload changes.
> > 
> > Signed-off-by: Frederic Weisbecker 
> > Cc: Chris Metcalf 
> > Cc: Christoph Lameter 
> > Cc: Luiz Capitulino 
> > Cc: Mike Galbraith 
> > Cc: Paul E. McKenney 
> > Cc: Peter Zijlstra 
> > Cc: Rik van Riel 
> > Cc: Thomas Gleixner 
> > Cc: Wanpeng Li 
> > Cc: Ingo Molnar 
> > ---
> >  Documentation/admin-guide/kernel-parameters.txt | 11 +++
> >  1 file changed, 11 insertions(+)
> > 
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt 
> > b/Documentation/admin-guide/kernel-parameters.txt
> > index 1d1d53f..50b9837 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -1766,6 +1766,17 @@
> >  
> > nohz
> >   Disable the tick when a single task runs.
> > +
> > + A residual 1Hz tick is offloaded to workqueues, which 
> > you
> > + need to affine to housekeeping through the global
> > + workqueue's affinity configured via the
> > + /sys/devices/virtual/workqueue/cpumask sysfs file, or
> > + by using the 'domain' flag described below.
> > +
> > + NOTE: by default the global workqueue runs on all 
> > CPUs,
> > + so to protect individual CPUs the 'cpumask' file has 
> > to
> > + be configured manually after bootup.
> 
> Hmm. Why don't you set that to the housekeeping cpus by default?

"nohz_full=" does affine to housekeeping by default but "isolcpus=nohz" doesn't
as it is designed for more finegrained settings.

In fact nohz_full= is what people want to use right now as isolcpus= is still
under construction. I'm waiting to be able to make its effects mutable through
cpusets before extending it further. But in the end I would like to have things
such as:

 isolcpus=nohz,async

This would affine the tick and everything that is async (workqueues, unbound 
timers,
unbound kthreads, etc...) to housekeeping.

And an even simpler toggle to fully isolate a CPU:

 isolcpus=all

This is to avoid that nohz alone implies side effects. For example if we affine
workqueues right now and later the 1hz offloaded tick isn't necessary anymore,
we'll stick with affined workqueues as a side effect without reason, and 
reverting
that would end up into regression.

Thanks.

Re: [PATCH 6/6] sched/isolation: Update nohz documentation to explain tick offload

[Frederic Weisbecker]

On Sat, Feb 17, 2018 at 11:53:22AM +0100, Thomas Gleixner wrote:
> On Thu, 15 Feb 2018, Frederic Weisbecker wrote:
> 
> > Update the documentation to reflect the 1Hz tick offload changes.
> > 
> > Signed-off-by: Frederic Weisbecker 
> > Cc: Chris Metcalf 
> > Cc: Christoph Lameter 
> > Cc: Luiz Capitulino 
> > Cc: Mike Galbraith 
> > Cc: Paul E. McKenney 
> > Cc: Peter Zijlstra 
> > Cc: Rik van Riel 
> > Cc: Thomas Gleixner 
> > Cc: Wanpeng Li 
> > Cc: Ingo Molnar 
> > ---
> >  Documentation/admin-guide/kernel-parameters.txt | 11 +++
> >  1 file changed, 11 insertions(+)
> > 
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt 
> > b/Documentation/admin-guide/kernel-parameters.txt
> > index 1d1d53f..50b9837 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -1766,6 +1766,17 @@
> >  
> > nohz
> >   Disable the tick when a single task runs.
> > +
> > + A residual 1Hz tick is offloaded to workqueues, which 
> > you
> > + need to affine to housekeeping through the global
> > + workqueue's affinity configured via the
> > + /sys/devices/virtual/workqueue/cpumask sysfs file, or
> > + by using the 'domain' flag described below.
> > +
> > + NOTE: by default the global workqueue runs on all 
> > CPUs,
> > + so to protect individual CPUs the 'cpumask' file has 
> > to
> > + be configured manually after bootup.
> 
> Hmm. Why don't you set that to the housekeeping cpus by default?

"nohz_full=" does affine to housekeeping by default but "isolcpus=nohz" doesn't
as it is designed for more finegrained settings.

In fact nohz_full= is what people want to use right now as isolcpus= is still
under construction. I'm waiting to be able to make its effects mutable through
cpusets before extending it further. But in the end I would like to have things
such as:

 isolcpus=nohz,async

This would affine the tick and everything that is async (workqueues, unbound 
timers,
unbound kthreads, etc...) to housekeeping.

And an even simpler toggle to fully isolate a CPU:

 isolcpus=all

This is to avoid that nohz alone implies side effects. For example if we affine
workqueues right now and later the 1hz offloaded tick isn't necessary anymore,
we'll stick with affined workqueues as a side effect without reason, and 
reverting
that would end up into regression.

Thanks.

Re: [PATCH 6/6] sched/isolation: Update nohz documentation to explain tick offload

[Thomas Gleixner]

On Thu, 15 Feb 2018, Frederic Weisbecker wrote:

> Update the documentation to reflect the 1Hz tick offload changes.
> 
> Signed-off-by: Frederic Weisbecker 
> Cc: Chris Metcalf 
> Cc: Christoph Lameter 
> Cc: Luiz Capitulino 
> Cc: Mike Galbraith 
> Cc: Paul E. McKenney 
> Cc: Peter Zijlstra 
> Cc: Rik van Riel 
> Cc: Thomas Gleixner 
> Cc: Wanpeng Li 
> Cc: Ingo Molnar 
> ---
>  Documentation/admin-guide/kernel-parameters.txt | 11 +++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt 
> b/Documentation/admin-guide/kernel-parameters.txt
> index 1d1d53f..50b9837 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -1766,6 +1766,17 @@
>  
>   nohz
> Disable the tick when a single task runs.
> +
> +   A residual 1Hz tick is offloaded to workqueues, which 
> you
> +   need to affine to housekeeping through the global
> +   workqueue's affinity configured via the
> +   /sys/devices/virtual/workqueue/cpumask sysfs file, or
> +   by using the 'domain' flag described below.
> +
> +   NOTE: by default the global workqueue runs on all 
> CPUs,
> +   so to protect individual CPUs the 'cpumask' file has 
> to
> +   be configured manually after bootup.

Hmm. Why don't you set that to the housekeeping cpus by default?

Thanks,

tglx

Re: [PATCH 6/6] sched/isolation: Update nohz documentation to explain tick offload

[Thomas Gleixner]

On Thu, 15 Feb 2018, Frederic Weisbecker wrote:

> Update the documentation to reflect the 1Hz tick offload changes.
> 
> Signed-off-by: Frederic Weisbecker 
> Cc: Chris Metcalf 
> Cc: Christoph Lameter 
> Cc: Luiz Capitulino 
> Cc: Mike Galbraith 
> Cc: Paul E. McKenney 
> Cc: Peter Zijlstra 
> Cc: Rik van Riel 
> Cc: Thomas Gleixner 
> Cc: Wanpeng Li 
> Cc: Ingo Molnar 
> ---
>  Documentation/admin-guide/kernel-parameters.txt | 11 +++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt 
> b/Documentation/admin-guide/kernel-parameters.txt
> index 1d1d53f..50b9837 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -1766,6 +1766,17 @@
>  
>   nohz
> Disable the tick when a single task runs.
> +
> +   A residual 1Hz tick is offloaded to workqueues, which 
> you
> +   need to affine to housekeeping through the global
> +   workqueue's affinity configured via the
> +   /sys/devices/virtual/workqueue/cpumask sysfs file, or
> +   by using the 'domain' flag described below.
> +
> +   NOTE: by default the global workqueue runs on all 
> CPUs,
> +   so to protect individual CPUs the 'cpumask' file has 
> to
> +   be configured manually after bootup.

Hmm. Why don't you set that to the housekeeping cpus by default?

Thanks,

tglx

[PATCH 6/6] sched/isolation: Update nohz documentation to explain tick offload

[Frederic Weisbecker]

Update the documentation to reflect the 1Hz tick offload changes.

Signed-off-by: Frederic Weisbecker 
Cc: Chris Metcalf 
Cc: Christoph Lameter 
Cc: Luiz Capitulino 
Cc: Mike Galbraith 
Cc: Paul E. McKenney 
Cc: Peter Zijlstra 
Cc: Rik van Riel 
Cc: Thomas Gleixner 
Cc: Wanpeng Li 
Cc: Ingo Molnar 
---
 Documentation/admin-guide/kernel-parameters.txt | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt 
b/Documentation/admin-guide/kernel-parameters.txt
index 1d1d53f..50b9837 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1766,6 +1766,17 @@
 
nohz
  Disable the tick when a single task runs.
+
+ A residual 1Hz tick is offloaded to workqueues, which 
you
+ need to affine to housekeeping through the global
+ workqueue's affinity configured via the
+ /sys/devices/virtual/workqueue/cpumask sysfs file, or
+ by using the 'domain' flag described below.
+
+ NOTE: by default the global workqueue runs on all 
CPUs,
+ so to protect individual CPUs the 'cpumask' file has 
to
+ be configured manually after bootup.
+
domain
  Isolate from the general SMP balancing and scheduling
  algorithms. Note that performing domain isolation 
this way
-- 
2.7.4

[PATCH 6/6] sched/isolation: Update nohz documentation to explain tick offload

[Frederic Weisbecker]

Update the documentation to reflect the 1Hz tick offload changes.

Signed-off-by: Frederic Weisbecker 
Cc: Chris Metcalf 
Cc: Christoph Lameter 
Cc: Luiz Capitulino 
Cc: Mike Galbraith 
Cc: Paul E. McKenney 
Cc: Peter Zijlstra 
Cc: Rik van Riel 
Cc: Thomas Gleixner 
Cc: Wanpeng Li 
Cc: Ingo Molnar 
---
 Documentation/admin-guide/kernel-parameters.txt | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt 
b/Documentation/admin-guide/kernel-parameters.txt
index 1d1d53f..50b9837 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1766,6 +1766,17 @@
 
nohz
  Disable the tick when a single task runs.
+
+ A residual 1Hz tick is offloaded to workqueues, which 
you
+ need to affine to housekeeping through the global
+ workqueue's affinity configured via the
+ /sys/devices/virtual/workqueue/cpumask sysfs file, or
+ by using the 'domain' flag described below.
+
+ NOTE: by default the global workqueue runs on all 
CPUs,
+ so to protect individual CPUs the 'cpumask' file has 
to
+ be configured manually after bootup.
+
domain
  Isolate from the general SMP balancing and scheduling
  algorithms. Note that performing domain isolation 
this way
-- 
2.7.4