On Tue, 2017-06-20 at 13:42 -0400, Rik van Riel wrote:
> On Mon, 2017-06-19 at 04:12 +0200, Frederic Weisbecker wrote:
> > Although idle load balancing obviously only concern idle CPUs, it can
> > be a disturbance on a busy nohz_full CPU. Indeed a CPU can only get
> > rid
> > of an idle load balancing duty once a tick fires while it runs a task
> > and this can take a while in a nohz_full CPU.
> >
> > We could fix that and escape the idle load balancing duty from the
> > very
> > idle exit path but that would bring unecessary overhead. Lets just
> > not
> > bother and leave that job to housekeeping CPUs (those outside
> > nohz_full
> > range). The nohz_full CPUs simply don't want any disturbance.
> >
> > Signed-off-by: Frederic Weisbecker <fweis...@gmail.com>
> > Cc: Thomas Gleixner <t...@linutronix.de>
> > Cc: Ingo Molnar <mi...@kernel.org>
> > Cc: Rik van Riel <r...@redhat.com>
> > Cc: Peter Zijlstra <pet...@infradead.org>
> > ---
> > kernel/sched/fair.c | 4 ++++
> > 1 file changed, 4 insertions(+)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index d711093..cfca960 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -8659,6 +8659,10 @@ void nohz_balance_enter_idle(int cpu)
> > if (!cpu_active(cpu))
> > return;
> >
> > + /* Spare idle load balancing on CPUs that don't want to be
> > disturbed */
> > + if (!is_housekeeping_cpu(cpu))
> > + return;
> > +
> > if (test_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu)))
> > return;
>
> I am not entirely convinced on this one.
>
> Doesn't the if (on_null_domain(cpu_rq(cpu)) test
> a few lines down take care of this already?
>
> Do we want nohz_full to always automatically
> imply that no idle balancing will happen, like
> on isolated CPUs?
IMO, nohz_full capable CPUs that are not isolated should automatically
become housekeepers, and nohz_full _active_ upon becoming isolated.
When a used as a housekeeper, you still pay a price for having the
nohz_full capability available, but it doesn't have to be as high.
In my kernels, I use cpusets to turn nohz on/off set wise, so CPUs can
be ticking, dyntick, nohz_full or housekeeper, RT load balancing and
cpupri on/off as well if you want to assume full responsibility. It's
a tad (from box of xxl tads) ugly, but more flexible.
-Mike
