On Apr 12, 2018, at 02:53, Mojca Miklavec wrote:

> On 12 April 2018 at 04:23, Ryan Schmidt wrote:
>> On Apr 11, 2018, at 21:22, Joshua Root wrote:
>>> On 2018-4-12 11:49 , Ryan Schmidt wrote:
>>>> 
>>>> These performance problems predate the existence of our 
>>>> getNextBuildOnPortBuilder function.
>>>> 
>>>> It smells to me like a missing database index problem. I don't know how to 
>>>> determine that though.
>>> 
>>> I'm guessing we probably run into trouble when the number of pending
>>> builds exceeds our cachedbuildrequests number. Is that at the default
>>> 1000 as set in master.cfg?
>> 
>> Aha! Sure, I don't know why its value would be anything other than what 
>> we've told it to be in master.cfg. So we should increase that to a much 
>> higher number.
> 
> Joshua, thank you very much for figuring out what the problem was and
> saving our buildbot.
> 
>> Along with maybe some of the other values.
> 
> You might want to increase the number of builds retained in history. I
> suspect that currently the builds get deleted after roughly 10.000 new
> builds come in.

Yes, we do only keep the 10,000 most recent builds per builder. Do you really 
think we need to keep more than that?


> I owe a deep apology. I did commit perl updates (with more than 1k
> ports) a couple of times without ever experience a problem as big as
> this one. This list seemed super short in comparison, but I didn't
> anticipate that so many perl subports would be broken and that the
> list would end up containing more than 1k ports per builder.

No worries. I appreciate the work you're doing, and we need to fix 
infrastructure such as buildbot to be compatible with how we work, so I'm glad 
we figured out how to let the buildbot deal with a larger number of pending 
builds.

I'm not sure why the mass perl changes before worked better...


> The queue for 10.13 has finally been digested, so I'll go over the
> list of failed ports and probably open a ticket to track progress on
> fixing the issues.

Dave has been adding some missing select files and missing setuptools 
dependencies.

Reply via email to