On Sat, 2021-09-11 at 14:07 +0100, Richard Purdie via lists.yoctoproject.org
wrote:
> On Thu, 2021-09-09 at 09:10 -0400, Trevor Gamblin wrote:
> > This prioritizeBuilders function sorts builders by the length of their
> > associated worker lists, so that builders that can only be assigned to a
> > small number of workers are assigned to those workers before other
> > builds that don't have specific needs when resources are limited. An
> > example might be when a slot is available on an Ubuntu-based worker, and
> > "oe-selftest-ubuntu" and "genericx86-64" build requests exist in the
> > queue. Since oe-selftest-ubuntu requires an Ubuntu-based worker and
> > genericx86-64 does not, genericx86-64 will be assigned a higher value
> > (lower priority) so that oe-selftest-ubuntu is assigned to that worker
> > first.
> >
> > Signed-off-by: Trevor Gamblin <[email protected]>
> > ---
> > builders.py | 13 +++++++++++++
> > master.cfg | 6 ++++++
> > 2 files changed, 19 insertions(+)
> >
>
> We did merge and start using this. Unfortunately whilst I thought it would
> work,
> it has some unfortunate side effects and we'll probably have to disable it.
>
> If you have two build requests for say "a-full" builds, it will allocate oe-
> selftest-XXX and reproducible-XXX to the XXX workers, starving generic
> targets.
> You therefore end up with two builds running, both with half their builder
> targets and it will hurt completion.
>
> I did talk with upstream buildbot a little on irc about this challenge.
>
> The default prioritizeBuilders function (_defaultSorter in
> buildrequestdistributor.py) sorts by build request time.
>
> They were suggesting some things which are fairly invasive like adding a
> estimated completion time to the defaultSorter so we could weight slow builds.
> To be honest it's beyond my comfort level with twisted defereds trying to
> query
> and obtain the right data.
>
> I have started wondering if we can in fact cheat and tweak the order we
> trigger
> the builds in the a-full and a-quick targets. I think we could influence the
> order of builderNames in the Triggerable and that would then change the
> buildrequest timestamps so the default sorting method would work better for us
> again.
>
> I think that translates to adding your sorting method in this patch to
> builderNamesFromConfigQuick and builderNamesFromConfigFull. Probably worth a
> try?
Upstream gave this as an example which I think we can probably do something
with:
from datetime import datetime, timedelta
from dateutil.tz import tzutc
from twisted.internet import defer
# this stores bonuses for builders, which takes in account longer build time
builder_bonuses = {
'bldr1': timedelta(hours=1)
}
def BuilderSorter(master, builders):
# perform an asynchronous schwarzian transform, transforming None
# into a really big date, so that any
# date set to 'None' will appear at the
# end of the list during comparisons.
max_time = datetime.max
# Need to set the timezone on the date, in order
# to perform comparisons with other dates which
# have the time zone set.
max_time = max_time.replace(tzinfo=tzutc())
@defer.inlineCallbacks
def transform(bldr):
time = yield bldr.getOldestRequestTime()
if time is None:
time = max_time
else:
if bldr.name in builder_bonuses:
time = time + builder_bonuses[bldr.name]
defer.returnValue((time, bldr))
transformed = yield defer.gatherResults(
[transform(bldr) for bldr in builders])
# sort the transformed list synchronously, comparing None to the end of
# the list
def transformedKey(a):
(date, builder) = a
return (date, builder.name)
transformed.sort(key=transformedKey)
# and reverse the transform
rv = [xf[1] for xf in transformed]
return rv
c['prioritizeBuilders'] = BuilderSorter
and then we're not relying on internal bebehaviours with the sorting. It was
pointed out to me that the timestamps are only second accuracy so the global
builders list sorting would probably have more effect.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#54703): https://lists.yoctoproject.org/g/yocto/message/54703
Mute This Topic: https://lists.yoctoproject.org/mt/85483506/21656
Group Owner: [email protected]
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
