On Wed, May 20, 2020 at 5:37 PM Michael 'veremitz' Everitt <[email protected]> wrote: > > On 21/05/20 01:18, Matt Turner wrote: > > On Wed, May 20, 2020 at 4:00 PM Michael 'veremitz' Everitt > > <[email protected]> wrote: > >> On 20/05/20 04:42, Matt Turner wrote: > >>> We currently have two mechanisms of setting MAKEOPTS: in spec files and > >>> in catalystrc. > >>> > >>> Setting makeopts in spec files doesn't make sense. The spec should > >>> describe the thing that's being built and not contain options that are > >>> specific to the build system. > >>> > >>> Setting makeopts via catalystrc is better, but it only applies to the > >>> actual build system invocations, leaving emerge to run jobs serially or > >>> again requiring configuration specific to the build machine to be put > >>> into the spec file. For example: > >>> > >>> update_seed_command: ... --jobs 5 --load-average 5 > >>> > >>> With jobs and load-average specified in catalyst.conf, catalyst has the > >>> information required to configure both emerge and the build systems > >>> emerge executes. > >>> > >>> This removes the undocumented makeopts spec file option and replaces it > >>> with jobs and load-average settings in catalyst.conf. > >>> > >>> Signed-off-by: Matt Turner <[email protected]> > >>> --- > >>> catalyst/base/stagebase.py | 12 +++++------- > >>> catalyst/defaults.py | 2 ++ > >>> doc/catalyst-config.5.txt | 15 ++++++++++++--- > >>> etc/catalyst.conf | 8 ++++++++ > >>> etc/catalystrc | 3 --- > >>> targets/support/chroot-functions.sh | 8 ++++++++ > >>> 6 files changed, 35 insertions(+), 13 deletions(-) > >> NACK. This and patch 20 make it impossible to customise specs for different > >> arches/subarches/libc/etc using different distcc_hosts with varying numbers > >> of hosts and/or cores available. In many cases, you may not be able to set > >> up identical distcc 'farms' or 'clusters' with the complete set of > >> toolchains/chroots applicable to a 'standard' installation, due to hardware > >> limitations. > > I don't follow. You mention varying distcc hosts per arches/subarches/libc. > > > > Let's say there are 4 systems on the network. 3x amd64 systems and 1 > > mips system. > > > > If I'm compiling for amd64 on system and I'm using distcc, there's a > > fixed number of distcc hosts on the network, aren't there? > Yes, but do you feed your amd64 jobs to your mips system? how do you filter > between them?
I wouldn't... but I don't think that matters? You just set distcc_hosts to not send jobs to that system. > > If I'm building for mips on the mips system with cross compilers > > installed on, say two of the amd64 systems, then there's a fixed > > number of distcc hosts on the network, aren't there? > > > > So, I don't think arches (or subarches) has any bearing. > > > > And for different libcs, distcc doesn't have any bearing on that, does > > it? distcc (without pump mode, which has been removed from Gentoo) > > sends preprocessed source files across the network, so what libc you > > have doesn't have any bearing either. > I have one or two systems with musl toolchains, but not all, again, how do > I choose between them, from a common build host, for each different spec > file variant I wish to run? Like I said, distcc sends a preprocessed file across the network, so there's not a dependence on the libc. I.e., a musl system running distccd can compile files just fine for a system running glibc. > > You seem to be thinking that setting a single system-wide distcc_hosts > > is somehow limiting, but I can't see how. Please explain. > > > I fail to see how you can select from a pool of distcc slaves for a given > job type/configuration. > > I'm assuming you've used distcc before? With cluster project's help I > actually have several distcc daemons running on a given host, if I should > require eg. gcc-8, 9 or (in the near future) 10. These run on different > ports on each slave system. > How would you propose selecting between these separate daemons for a given > build in your new schema? Yes, I use distcc to cross compile for slow mips and arm systems. I don't understand why it takes so much more effort on the part of the receiver to interpret your messages, but good grief. I think the case Michael is thinking of is, e.g. a system building both rel_type: default and rel_type: hardened stages. The set of distcc slaves that have a non-hardened toolchain might be different from the set of distcc slaves with a hardened toolchain. I'm unsure how this works, so I'm asking #gentoo-toolchain. So all of the previous email exchange was a collection of red herrings...
