On Tue, Jul 04, 2017 at 01:22:05PM -0700, Steve Langasek wrote: > On Tue, Jul 04, 2017 at 04:31:32PM +0000, James Page wrote: > > Mathias and I where discussing some challenges around fitting the Ceph > > build with any level of parallelism in to a virtualized launchpad builder > > (as used for the majority of architectures today). > > > Basically the current memory spec only permits a max-parallel of 2 (on > > Xenial amd64). Ceph is a big C++ project so takes some time to build with > > such a low level of parallelism. > > > Would it be possible to nudge up the default memory allocation on the > > builders? I use a 16G machine for test builds of i386/amd64 and I've not > > seen any memory exhaustion with parallel=8. > > > Or maybe have a mechanism to allow a source package to request a higher > > spec of builder? > > You should probably consult the Launchpad team on this, since the Launchpad > builder configuration is owned by them and not by the Ubuntu developers > directly. > > As far as raising the default memory allocation, understand that there is a > tradeoff between being able to parallelize /within/ a build and being able > to parallelize across /all/ builds. If the sole justification for this > request is that it would speed up the build of one particular package, I > don't think that's a sufficient reason to reduce the capacity of the > launchpad build farm as a whole.
Indeed - increasing the size of a single builder would mean reducing the number of builders we can fit on scalingstack, and I don't think that's a good trade-off globally. The current size seems to be a sweet spot, even if it means that some individual packages are a bit slower to build. The build farm spends a lot of time on lots of relatively small builds, so parallelising across all builders tends to win as a strategy. > Providing a mechanism to request a builder of a different size seems ok, but > would need to be designed/implemented in Launchpad; and how do you prevent > this from being abused? Our experience is that the build farm delivers best overall performance when it's divided into as small a number of pools as possible, and as Dimitri says we pre-boot builder VMs before knowing what build is going to be dispatched to them, so having some larger builders would require dividing the build farm into more pools: we'd have to have amd64+i386 (small) and amd64+i386 (large) rather than just amd64+i386, for instance. We'd be very resistant to this. -- Colin Watson [[email protected]] -- ubuntu-devel mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
