On 4 July 2017 at 21:22, Steve Langasek <[email protected]> wrote: > > Hi James, > > On Tue, Jul 04, 2017 at 04:31:32PM +0000, James Page wrote: > > Hi All > > > Mathias and I where discussing some challenges around fitting the Ceph > > build with any level of parallelism in to a virtualized launchpad builder > > (as used for the majority of architectures today). > > > Basically the current memory spec only permits a max-parallel of 2 (on > > Xenial amd64). Ceph is a big C++ project so takes some time to build with > > such a low level of parallelism. > > > Would it be possible to nudge up the default memory allocation on the > > builders? I use a 16G machine for test builds of i386/amd64 and I've not > > seen any memory exhaustion with parallel=8. > > > Or maybe have a mechanism to allow a source package to request a higher > > spec of builder? > > You should probably consult the Launchpad team on this, since the Launchpad > builder configuration is owned by them and not by the Ubuntu developers > directly. > > As far as raising the default memory allocation, understand that there is a > tradeoff between being able to parallelize /within/ a build and being able > to parallelize across /all/ builds. If the sole justification for this > request is that it would speed up the build of one particular package, I > don't think that's a sufficient reason to reduce the capacity of the > launchpad build farm as a whole. > > Providing a mechanism to request a builder of a different size seems ok, but > would need to be designed/implemented in Launchpad; and how do you prevent > this from being abused? >
Yeah, if we make all builders x4 the size, it may mean we will have 4x less /builders/. However debhelper 10 defaults to parallel builds, so I wonder if having bigger instances will be a net gain or not. As far as I understand, from previous face to face conversations, the builder VMs are pre-booted (allocated) and join Launchpad pool of builders idling. Thus a builder is already there, before it knows which package it will build and whether the package is a distro package or a ppa one. My concern was that launchpad builders should scale with the queue sizes, as their utilisation is not 100%. Such that e.g. launchpad allocation of builders quota could be potentially shared with the ADT testing quota. As in ADT we frequently see the tests failing due to hitting ADT quotas, at the same time launchpad builders sit there idling. If there was some capacity to auto scale builders, when the queue is short it would be nicer probably to have less builders, but maybe a few beefier instances. But this is very hand-wavy, as changes to builders will need a chunk of engineering on launchpad builder side. -- Regards, Dimitri. -- ubuntu-devel mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
