Timothy Arceri <tarc...@itsqueeze.com> writes:

> I'd just like to point out that as an outside user of Intels CI I have 
> missed regressions on a couple of occasions. However this was not due to 
> "Lazy behaviour", having a CI system is fantastic and I'm very grateful 
> to have access to it. However, it's not uncommon to run into issues and 
> have no idea what is going on with the system.
> Some examples are getting no emails back from the system after pushing, 
> results that look like a successful run even though things have failed 
> or just no results email at all, results with tonnes and tonnes of fails 
> which are clearly unrelated to my latest branch, on occasion the system 
> seems to have crashed? and been unresponsive for a whole weekend (which 
> means down for me on a Monday in Australia).

There is some down-time that we can't control.  Intel has turned off our
lab for days at a time to perform power upgrades.

In cases where our jenkins automation is at fault, we do work to improve

Other failure modes for CI are similar to what happened this past
weekend, where we were forced to update to linux 4.14 which generated
thousands of vulkan failures (fixed in 4.15, and the CI is back).

Would it help to have an announcement list where we can indicate any
current issues affecting CI?  If devs have examples of CI that are more
reliable than ours, I'm eager to learn how to improve.

> If the change is for i965 I either wait or try bug someone at Intel (if 
> anyone happens to be around) to find out what is going on, but for core 
> mesa changes having run piglit on radeonsi locally I tend to push my 
> changes. I get that regressions are frustrating at times but using the 
> CI system as an outside user can be frustrating also when you have no 
> idea whats going on in the black box after pushing a branch, especially 
> when you need to wait an hour or so to try again in between runs.

I hear you.  We are prioritizing better visibility for external users,
and have added staff to implement that.

> I gave feedback at first on ways the system could be better, or errors I 
> seemed to hit but was told there wouldn't be and improvements made for 
> the foreseeable future so I stopped giving feedback and instead switched 
> to relying on my own local testing when the CI system seemed to have 
> lost its mind.
> Anyway this is not meant to be a criticism, I just wanted to share my 
> experience as an outside user.

The feedback is appreciated.
mesa-dev mailing list

Reply via email to