1-4 all seem uncontroversial, especially with Tim's suggested
improvement to immediately leave a "some queues failed" comment.
On Sat, Jun 15, 2019 at 11:13 PM, Aakash Jain <aakash_j...@apple.com>
wrote:
5) Do not comment on bugzilla bug at all, instead send email to the
author of the patch.
Pros: less noisy, also this will allow to include more detailed
information about the failure in email.
Cons: reviewers would have to click status-bubbles to see the
failures, failure information is not immediately present in the
comments.
Well I think this would be OK, but next we'll be complaining about the
emails. The underlying problem is excessive test flakiness. We're just
not doing a very good job of tracking flaky tests and EWS isn't good at
detecting flaky tests automatically. (Often a flaky test will fail
twice in the same run, and that gets detected as a test failure.)
Instead of reporting bugs for such tests, we're ignoring them because
there are so many and we're so used to it.
commit-queue is able to report bugs when it detects a flaky patch, but
EWS doesn't do this. That might be a positive change. Of course, EWS
would need to be smart enough to not report a bug if the flakiness is
triggered by the patch itself, which might not be simple to determine.
I don't have any concrete suggestions here. Just brainstorming.
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
