On Thu, 2021-09-09 at 09:10 -0400, Trevor Gamblin wrote: > This prioritizeBuilders function sorts builders by the length of their > associated worker lists, so that builders that can only be assigned to a > small number of workers are assigned to those workers before other > builds that don't have specific needs when resources are limited. An > example might be when a slot is available on an Ubuntu-based worker, and > "oe-selftest-ubuntu" and "genericx86-64" build requests exist in the > queue. Since oe-selftest-ubuntu requires an Ubuntu-based worker and > genericx86-64 does not, genericx86-64 will be assigned a higher value > (lower priority) so that oe-selftest-ubuntu is assigned to that worker > first. > > Signed-off-by: Trevor Gamblin <[email protected]> > --- > builders.py | 13 +++++++++++++ > master.cfg | 6 ++++++ > 2 files changed, 19 insertions(+) >
We did merge and start using this. Unfortunately whilst I thought it would work, it has some unfortunate side effects and we'll probably have to disable it. If you have two build requests for say "a-full" builds, it will allocate oe- selftest-XXX and reproducible-XXX to the XXX workers, starving generic targets. You therefore end up with two builds running, both with half their builder targets and it will hurt completion. I did talk with upstream buildbot a little on irc about this challenge. The default prioritizeBuilders function (_defaultSorter in buildrequestdistributor.py) sorts by build request time. They were suggesting some things which are fairly invasive like adding a estimated completion time to the defaultSorter so we could weight slow builds. To be honest it's beyond my comfort level with twisted defereds trying to query and obtain the right data. I have started wondering if we can in fact cheat and tweak the order we trigger the builds in the a-full and a-quick targets. I think we could influence the order of builderNames in the Triggerable and that would then change the buildrequest timestamps so the default sorting method would work better for us again. I think that translates to adding your sorting method in this patch to builderNamesFromConfigQuick and builderNamesFromConfigFull. Probably worth a try? Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54702): https://lists.yoctoproject.org/g/yocto/message/54702 Mute This Topic: https://lists.yoctoproject.org/mt/85483506/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
