I may be alone here, but I actually found build/test(s) failure emails to be 
super useful.  The part that I did not find useful was that each time a 
build/test(s) would fail on a particular platform, there would be two 
comments/emails, which furthermore split the useful information into two 
pieces.  IIRC, the first comment/email would contain information about what 
build/test(s) failed, and the second comment/email was just an archive upload 
of the layout test results (if applicable).

I would find it very useful to get an email each time one of the bots had 
something go "wrong", and ideally include a link directly to the 'errors' log 
that Aakash mention in the beginning of this thread (or to the layout test 
results in the case of a test(s) failure).  Right now, if something goes 
"wrong", I have to be the one to discover it and then click through a few pages 
until I find the data that will actually help me solve the problem.

As a patch author, this is useful in letting me not have to remember to check 
my patch once a day in order to see if there are any red bubbles (something 
which I've forgotten to do, leaving a broken patch up for review thinking all 
was fine).  I can upload a patch and ideally leave it be, as I will be notified 
if the build/test(s) fails.

As a patch reviewer, I'm not sure this would be as useful, as I have a saved 
query that I use for finding all patches that need to be reviewed, and on a 
search result page there are no EWS bubbles anyways, so I can't see the info 
there.  I have to click into the bug regardless.  To Ryosuke's point, it may 
still be useful to get an email as a reviewer simply so I know that something 
failed and can potentially remember that fact before I click on a bug in the 
search results page, but I know personally that my mind doesn't really work 
that way :P


> On Nov 5, 2019, at 09:04, Alexey Proskuryakov <a...@webkit.org> wrote:
>> 4 нояб. 2019 г., в 1:37 PM, Ryosuke Niwa <rn...@webkit.org 
>> <mailto:rn...@webkit.org>> написал(а):
>> On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov <a...@webkit.org 
>> <mailto:a...@webkit.org>> wrote:
>> Can you elaborate on that, how exactly is e-mailing on first failure useful 
>> to reviewers?
>> Getting rid of Bugzilla comments was one of the goals of EWS rewrite, based 
>> on engineering feedback about noise in bugs and in e-mail, and I 
>> wholeheartedly agree with this feedback. So I think that comments are 
>> generally undesirable.
>> Since I don't understand what your precise scenario is, I may be make straw 
>> man arguments below, but here are some things that I think make the proposed 
>> behavior unhelpful (add a comment on first failure, or when all EWSes pass).
>> 1. EWS comments in Bugzilla are so annoying that some people take the 
>> radical step of manually hiding them. EWS history is archived anyway, there 
>> is no need to look into comments for it.
>> 2. There are often many people CC'ed on the bug to whom EWS data is 
>> irrelevant or even mysterious (e.g. reporters, web developers or 
>> non-reviewers). The noise is a slight annoyance, discouraging further 
>> participation in the project.
>> 3. I believe that for most reviewers, the mode of operation is one of the 
>> two: (1) do it when pinged directly, or (2) go over the review queue when 
>> one has the time. Getting EWS comments helps neither.
>> 4. Commenting when all EWSes pass is not very practical - it's too often 
>> that we have some stragglers that take days (or forever). I don't think that 
>> we can make it reliable even if we start actively policing EWS 
>> responsiveness.
>> 5. The reviewer likely wants to know the state of multiple EWSes if they are 
>> going to wait for EWS at all. What exactly are they going to do after 
>> getting an e-mail that one EWS failed?
>> I often use a EWS failure as a signal to wait reviewing a patch. Otherwise, 
>> a bug mail will stay in my inbox as one of items to get to.
>> I can see the usefulness in the somewhat unusual case of a super urgent 
>> patch. We may want multiple people to watch it, so that members of CC list 
>> would go and ask the patch author to update it with more urgency than e-mail 
>> allows for. I think that opt-in is a better mechanism for that, so that 
>> people who opted in would receive information about each EWS data point.
>> I think there is a value in knowing that a patch isn't ready instead of 
>> having to open the bug to realize that.
> So just to clarify, 
> - a major part of how you get to review bugs is by being CC'ed, and you 
> review them when you have the time to read bugmail;
> - and you don't open the bug in Bugzilla if there is already an EWS failure 
> by the time you read the e-mail where review is requested?
> That's clearly a valid benefit. In my mind, it probably doesn't outweigh the 
> downsides. On the other hand, yours is a voice of someone who reviews way 
> more patches than Maciej and me combined these days, so maybe more e-mail is 
> an overall benefit to many of the reviewers.
> - Alexey
>> - R. Niwa
>>> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak <m...@apple.com 
>>> <mailto:m...@apple.com>> написал(а):
>>> I think they are useful to actual and potential reviewers. Direct email to 
>>> the patch author is not something anyone can Cc themselves on, and is not 
>>> archived, so seems like a strictly worse form of communication.
>>>> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov <a...@apple.com 
>>>> <mailto:a...@apple.com>> wrote:
>>>> My preference is still e-mailing the patch author directly (possibly, also 
>>>> having an option to opt in for anyone). Bugzilla comments will always be 
>>>> irrelevant for most people CC'ed on the bug, and they are almost always 
>>>> undesirable to keep within the discussion flow.
>>>> - Alexey
>>>>> 1 нояб. 2019 г., в 18:28, Aakash Jain <aakash_j...@apple.com 
>>>>> <mailto:aakash_j...@apple.com>> написал(а):
>>>>> Sounds good. I prefer the single comment when the first failure occur. 
>>>>> That way notification would be sent as soon as the first failure happens.
>>>>> I'll implement that (assuming it's acceptable to everyone).
>>>>> Thanks
>>>>> Aakash
>>>>>> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak <m...@apple.com 
>>>>>> <mailto:m...@apple.com>> wrote:
>>>>>> How about only a single comment when the first failure occurs? (Or else 
>>>>>> when all bots pass, if there is never a failure.)
>>>>>> This should help the author, the reviewer, and anyone else cc’d, without 
>>>>>> being too spammy.
>>>>>>> On Nov 1, 2019, at 5:20 PM, Aakash Jain <aakash_j...@apple.com 
>>>>>>> <mailto:aakash_j...@apple.com>> wrote:
>>>>>>> Hi Ryosuke,
>>>>>>> Many people didn't like the noise by the EWS comments, and we removed 
>>>>>>> the comments as per previous discussion in: 
>>>>>>> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html 
>>>>>>> <https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html>.
>>>>>>> I agree with your point that having some kind of notification might be 
>>>>>>> useful.
>>>>>>> I proposed some ideas in 
>>>>>>> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html
>>>>>>> <https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html>,
>>>>>>>  but didn't get much feedback. If we can all agree on a solution, I can 
>>>>>>> look into implementing it.
>>>>>>> Thanks
>>>>>>> Aakash
>>>>>>>> On Oct 30, 2019, at 1:03 AM,
>>>>>>>> - R. Niwa
>>>>>>>> <rn...@webkit.org <mailto:rn...@webkit.org>> wrote:
>>>>>>>> These enhancements are great. There is one feature of the old EWS that 
>>>>>>>> I really miss, which is that I used to get emails when some EWS 
>>>>>>>> failed. With new EWS, I have to keep checking back the bugzilla to see 
>>>>>>>> if any of them have failed periodically.
>>>>>>>> Can we add a feature to opt into such an email notification? Maybe a 
>>>>>>>> flag on a patch or JSON configuration file somewhere.
>>>>>>>> - R. Niwa
>>>>>>>> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain <aakash_j...@apple.com 
>>>>>>>> <mailto:aakash_j...@apple.com>> wrote:
>>>>>>>> Hi Everyone,
>>>>>>>> I am happy to announce another EWS feature.
>>>>>>>> From now on, in case of build failure, EWS will parse the errors and 
>>>>>>>> display them in a separate 'errors' log. You wouldn't have to search 
>>>>>>>> through thousands of lines of logs to find the error message.
>>>>>>>> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054 
>>>>>>>> <https://ews-build.webkit.org/#/builders/16/builds/6054>, in step #7 
>>>>>>>> WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, and 
>>>>>>>> the error is not at the end of the logs. Normally, it requires some 
>>>>>>>> searching through the logs to find the relevant errors. But now, there 
>>>>>>>> is another 'errors' log, which contains just the relevant 11 lines 
>>>>>>>> (containing error and few related lines to provide additional context).
>>>>>>>> Hopefully this would save some time and efforts previously spent on 
>>>>>>>> searching through the large logs.
>>>>>>>> Note that this information is not displayed in status-bubble tool-tip, 
>>>>>>>> since this might be lot of text to display in the tooltip. My further 
>>>>>>>> plan is to make this information more readily available, by adding it 
>>>>>>>> to a custom designed page which will open on clicking the status 
>>>>>>>> bubble https://webkit.org/b/197522 <https://webkit.org/b/197522>
>>>>>>>> Please let me know if you notice any issues or have any feedback.
>>>>>>>> Thanks
>>>>>>>> Aakash
>>>>>>>> Reference: https://webkit.org/b/203418 <https://webkit.org/b/203418>
>>>>>>>> _______________________________________________
>>>>>>>> webkit-dev mailing list
>>>>>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
>>>>>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev 
>>>>>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev>
>>>>>>>> -- 
>>>>>>>> - R. Niwa
>>>>>>> _______________________________________________
>>>>>>> webkit-dev mailing list
>>>>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
>>>>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev 
>>>>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev>
>>>>> _______________________________________________
>>>>> webkit-dev mailing list
>>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
>>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev 
>>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev>
>>>> - Alexey
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
>> https://lists.webkit.org/mailman/listinfo/webkit-dev 
>> <https://lists.webkit.org/mailman/listinfo/webkit-dev>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

webkit-dev mailing list

Reply via email to