On Tue, Apr 05, 2016 at 12:37:07PM -0700, John Johansen wrote: > Enable dynamically scaling max jobs if new resources are brought online > > BugLink: http://bugs.launchpad.net/bugs/1566490 > > This patch enables to parser to scale the max jobs if new resources are > being brought online by the scheduler. > > It only enables the scaling check if there is a difference between the > maximum number of cpus (CONF) and the number of online (ONLN) cpus. > > Instead of checking for more resources regardless, of whether the online > cpu count is increasing it limits its checking to a maximum of > MAX CPUS + 1 - ONLN cpus times. With each check coming after fork spawns a > new work unit, giving the scheduler a chance to bring new cpus online > before the next check. The +1 ensures the checks will be done at least > once after the scheduling task sleeps waiting for its children giving > the scheduler an extra chance to bring cpus online. > > Signed-off-by: John Johansen <[email protected]>
This feels more complicated than it could be but I must admit I can't suggest any modifications to the algorithm to simplify it. Acked-by: Seth Arnold <[email protected]> Thanks > > === modified file 'parser/parser_main.c' > --- a/parser/parser_main.c 2016-01-15 01:26:26 +0000 > +++ b/parser/parser_main.c 2016-04-05 07:27:06 +0000 > @@ -89,6 +89,12 @@ > long jobs_max = -8; /* 8 * cpus */ > long jobs = JOBS_AUTO; /* default: number of processor > cores */ > long njobs = 0; > +long jobs_scale = 0; /* number of chance to resample online > + * cpus. This allows jobs spawning to > + * scale when scheduling policy is > + * taking cpus off line, and brings > + * them back with load > + */ > bool debug_jobs = false; > > struct timespec cache_tstamp, mru_policy_tstamp; > @@ -898,6 +904,16 @@ > RESULT(WORK); \ > break; \ > }*/ \ > + if (jobs_scale) { \ > + long n = sysconf(_SC_NPROCESSORS_ONLN); \ > + if (n > jobs) { \ > + /* reset sample chances - potentially reduce to 0 */ \ > + jobs_scale = jobs_max + 1 - n; \ > + jobs = n; \ > + } else \ > + /* reduce scaling chance by 1 */ \ > + jobs_scale--; \ > + } \ > if (njobs == jobs) { \ > /* wait for a child */ \ > if (debug_jobs) \ > @@ -959,11 +975,18 @@ > { > /* jobs and paralell_max set by default, config or args */ > long n = sysconf(_SC_NPROCESSORS_ONLN); > + long maxn = sysconf(_SC_NPROCESSORS_CONF); > if (n == -1) > /* unable to determine number of processors, default to 1 */ > n = 1; > + if (maxn == -1) > + /* unable to determine number of processors, default to 1 */ > + maxn = 1; > + if (n < maxn) > + /* the bigger the difference the more sample chances given */ > + jobs_scale = maxn + 1 - n; > jobs = compute_jobs(n, jobs); > - jobs_max = compute_jobs(n, jobs_max); > + jobs_max = compute_jobs(maxn, jobs_max); > > if (jobs > jobs_max) { > pwarn("%s: Warning capping number of jobs to %ld * # of cpus == > '%ld'", >
signature.asc
Description: Digital signature
-- AppArmor mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
