Resending to portage dev ml to solicit comments... Thoughts welcome, since it's an issue that must be dealt with to sanely do efficient parallelization (monkeying with make.conf and hardcoding vals isn't much of a solution).
Zac- comments? You seem to have totally missed the original email ;)
-------------------------------------
Date: Sun, 1 Oct 2006 12:27:13 -0700
To: gentoo-dev@lists.gentoo.org
From: Brian Harring <[EMAIL PROTECTED]>
Subject: Re: [gentoo-dev] Setting number of parallel builds for other
build-systems than 'make'
On Sun, Oct 01, 2006 at 09:52:14AM -0700, Donnie Berkholz wrote:
> Tiziano Müller wrote:
> >-----BEGIN PGP SIGNED MESSAGE-----
> >Hash: SHA1
> >
> >Hi everyone
> >
> >It seems that setting the number of parallel builds using '-jN' does not
> >only work for make, but also for scons and bjam (and maybe others as well).
> >Since it isn't save to assume that '-jN' is the only option in MAKEOPTS,
> >some filtering is needed.
> >Now, SCONSOPTS (BJAMOPTS respectively) could be added to make.conf and used
> >whenever one of those build-systems is being used. But we would probably
> >have to add a ...OPTS for every build-system.
> >
> >What do you think? Better ideas?
>
> I think adding it for each build system is probably a good idea,
> nobody's guaranteeing option-level compatibility with make. Optionally
> falling back to using the few valid MAKEOPTS might be a nice addition.
I might be daft (likely), but why not just introduce a var indicating
max parallelization instead? Tweak portage to push that setting into
MAKEOPTS="${MAKEOPTS+${MAKEOPTS} } -j${PARALLELIZATION}".
Might sound daft, but -j is hardcoded parallelization; if you're
trying to run 3 build processes, the original var of -j isn't all that
useful, nor will the hardcoded -j# var set for 3 package build
processes be useful if you're doing a single build.
Depending on number of seperate package build processes possible, the
# of build processes allocated per build process needs to vary
(essentially).
Now... that's a bit ahead of what portage is doing atm, but folks
*will* parallelize portage building (see bug 147516 if in doubt), so
its not too far out.
Getting back to the actual topic, set this var, it's a raw int that a
scons/bjam eclass can use to easily set appropriate var with the
parallelization requested, thus unifying this particular knob; more
importantly, it gives them a way to do what they're after while
exposing a knob that the pkg manager can easily fiddle with for global
parallelization needs.
Only downside to it I see is that it requires mangling the pkg manager
to translate the parallelization setting into MAKEOPTS+=-j#; can't
really get around that however due to the fact MAKEOPTS is already
forced in installed configuration though.
Thoughts?
~harring
pgpZOWWRmNkmO.pgp
Description: PGP signature
