Re: [webkit-dev] Requesting feedback about EWS comments on Bugzilla bugs

[David Kilzer]

I actually “missed” output from the stylebot insofar as I was expecting the 
errors to be posted, and then I was going to reply to that automated message.

I think it’s too easy to ignore style errors if you have to click to load 
another website, though, so I filed this bug with some thoughts:

Bug 201993: EWS style bot should show error output via semi-modal dialog rather 
than requiring to click through
>

Maybe we could do something similar for layout test failures (a semi-modal 
dialog that lists the errors, and links directly to the results rather than 
having to click (twice) through the buildbot landing page?

Dave


On Sep 19, 2019, at 9:45 AM, Aakash Jain  wrote:

> On Jun 16, 2019, at 2:14 PM, Darin Adler  > wrote:
> 
>> On Jun 15, 2019, at 9:13 PM, Aakash Jain > > wrote:
>> 
>>> 1) Do not upload archive (for layout-test-results) on bugzilla, instead 
>>> upload it to another server, unzip it and post a link to the results.html.
>>> Pros:
>>> a) Engineers won't have to download the attachment, unzip it, look for 
>>> failures, and then delete it from their disk. They can simply click the url 
>>> to view the results. 
>>> b) This approach will also reduce 2 comments per failure to 1 comment. 
>>> Currently there are two comments per failure, one for failure details, 
>>> second for bugzilla attachment.
>> 
>> Great improvement to do this.
> 
> We have implemented this in the new EWS. Layout test results are no longer 
> added to EWS as attachments. Instead they are available to view in browser or 
> download from the Buildbot build page.
> 
>> The most confusing thing about build bot comments is all the “creation of 
>> attachments” extra text with things like “attachment number” and “patch".
>> 
>> However, it’s really nice that I can download a directory full of test 
>> results easily. I’d like to see the EWS website still have that feature.
>> 
>>> 4) When a patch becomes 'obsolete', tag the corresponding EWS comments as 
>>> 'obsolete', so that they will be hidden.
>> 
>> Incredibly valuable.
>> 
>>> 5) Do not comment on bugzilla bug at all
>> 
>> I think this makes sense. I don’t see a reason that test results need to be 
>> comments. I think the “red bubble” in EWS already calls someone’s attention 
>> to failures.
>> 
>> If we want to augment it, we should think of what we are aiming at. I do 
>> find it useful to see which tests are failing, and when I click on the red 
>> bubble I don’t see that information. I have to click once to see the “log of 
>> activities” then click on “results”, then see a confusing giant file with 
>> lots of other information. At the bottom of that file the one thing I want 
>> to know.
>> 
>> A better hierarchy is to put that “what new tests are failing” summary right 
>> t the top and let the logs be fallbacks, not the primary place to see the 
>> features.
>> 
>>> instead send email to the author of the patch.
>> 
>> Why? I don’t think this should send any emails at all, unless the person 
>> requested it.
>> 
>>> Pros: less noisy, also this will allow to include more detailed information 
>>> about the failure in email.
>> 
>> I think the more detailed information should be on the webpage, not in an 
>> email.
>> 
>>> Cons: reviewers would have to click status-bubbles to see the failures, 
>>> failure information is not immediately present in the comments.
>> 
>> I think we should start with this approach, eliminating the comments 
>> entirely.
> 
> Following this suggestion, we eliminated the comments entirely in the new 
> EWS. However, some people mentioned that since there are no comments, they do 
> not get email notifications on failure, e.g.: https://webkit.org/b/200399 
> . Maybe we should have some kind of comments by 
> EWS. Few ideas:
> 
> 1) Comment on first failure for a patch, e.g.: "Some failures were noticed by 
> EWS, please check the status bubbles".
> 
> 2) Comment about success on all queues, e.g.: "Patch passed all EWS queues".
> 
> 
> What do you guys think?
> 
>> 
>> — Darin
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Requesting feedback about EWS comments on Bugzilla bugs

[Aakash Jain]

> On Jun 16, 2019, at 2:14 PM, Darin Adler  wrote:
> 
>> On Jun 15, 2019, at 9:13 PM, Aakash Jain > > wrote:
>> 
>> 1) Do not upload archive (for layout-test-results) on bugzilla, instead 
>> upload it to another server, unzip it and post a link to the results.html.
>> Pros:
>> a) Engineers won't have to download the attachment, unzip it, look for 
>> failures, and then delete it from their disk. They can simply click the url 
>> to view the results. 
>> b) This approach will also reduce 2 comments per failure to 1 comment. 
>> Currently there are two comments per failure, one for failure details, 
>> second for bugzilla attachment.
> 
> Great improvement to do this.

We have implemented this in the new EWS. Layout test results are no longer 
added to EWS as attachments. Instead they are available to view in browser or 
download from the Buildbot build page.

> The most confusing thing about build bot comments is all the “creation of 
> attachments” extra text with things like “attachment number” and “patch".
> 
> However, it’s really nice that I can download a directory full of test 
> results easily. I’d like to see the EWS website still have that feature.
> 
>> 4) When a patch becomes 'obsolete', tag the corresponding EWS comments as 
>> 'obsolete', so that they will be hidden.
> 
> Incredibly valuable.
> 
>> 5) Do not comment on bugzilla bug at all
> 
> I think this makes sense. I don’t see a reason that test results need to be 
> comments. I think the “red bubble” in EWS already calls someone’s attention 
> to failures.
> 
> If we want to augment it, we should think of what we are aiming at. I do find 
> it useful to see which tests are failing, and when I click on the red bubble 
> I don’t see that information. I have to click once to see the “log of 
> activities” then click on “results”, then see a confusing giant file with 
> lots of other information. At the bottom of that file the one thing I want to 
> know.
> 
> A better hierarchy is to put that “what new tests are failing” summary right 
> t the top and let the logs be fallbacks, not the primary place to see the 
> features.
> 
>> instead send email to the author of the patch.
> 
> Why? I don’t think this should send any emails at all, unless the person 
> requested it.
> 
>> Pros: less noisy, also this will allow to include more detailed information 
>> about the failure in email.
> 
> I think the more detailed information should be on the webpage, not in an 
> email.
> 
>> Cons: reviewers would have to click status-bubbles to see the failures, 
>> failure information is not immediately present in the comments.
> 
> I think we should start with this approach, eliminating the comments entirely.

Following this suggestion, we eliminated the comments entirely in the new EWS. 
However, some people mentioned that since there are no comments, they do not 
get email notifications on failure, e.g.: https://webkit.org/b/200399 
. Maybe we should have some kind of comments by 
EWS. Few ideas:

1) Comment on first failure for a patch, e.g.: "Some failures were noticed by 
EWS, please check the status bubbles".

2) Comment about success on all queues, e.g.: "Patch passed all EWS queues".


What do you guys think?

> 
> — Darin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Questions about JSC EWS queues

[Aakash Jain]

> On Sep 19, 2019, at 7:22 AM, Guillaume Emont  wrote:
> 
> Quoting Aakash Jain (2019-09-18 19:21:02)
>> Hi All,
>> 
>> I am working on moving JSC EWS queues from old EWS to new EWS. I am trying to
>> clearly understand various JSC EWS queues. I have few questions:
>> 
>> 1) What does 'jsc-only' port represent? From https://trac.webkit.org/browser/
>> webkit/trunk/Tools/Scripts/webkitpy/common/config/ews.json#L45 it seems like
>> Apple JSC EWS uses 'mac' port, while linux jsc EWSes use 'jsc-only' port. Is
>> jsc-only port specific to linux? Is there a corresponding jsc port for mac?
>> 
>> 2) Is there any difference between the three linux JSC EWSes (JSC i386, JSC
>> MIPS32el, JSC Armv7) apart from the architecture and few queues running/
>> skipping tests? 
> 
> I think the main difference is indeed the different architecture. There
> is also the fact that the JSC i386 EWS runs the tests natively, but the
> mips and armv7 EWSes cross-compile and run the tests on a set of remote
> devices.
> 
>> 
>> 3) Where is the architecture for various linux JSC EWS queues configured in 
>> the
>> old EWS code (e.g.: I don't see the architecture being added in https://
>> trac.webkit.org/changeset/237001/webkit)? Is the queue configuration for
>> various JSC EWS queues same, and the device connected to the queue actually
>> decides the architecture?
> 
> I guess it's not defined anywhere indeed. The scripts that start the
> queues for the arm and mips EWSes define a bunch of env variables to
> point to the cross-compiler and also add a few options in
> JSCTESTS_OPTIONS, such as a  --remote-config-file and --memory-limited.
> I can share these scripts with you if you need more details.

Thanks for the reply.

Yeah, I would need the details. It would be nice if you can share those scripts 
with me.

> 
> Hope this helps.
> 
> Cheers,
> 
> Guij
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Questions about JSC EWS queues

[Guillaume Emont]

Quoting Aakash Jain (2019-09-18 19:21:02)
> Hi All,
> 
> I am working on moving JSC EWS queues from old EWS to new EWS. I am trying to
> clearly understand various JSC EWS queues. I have few questions:
> 
> 1) What does 'jsc-only' port represent? From https://trac.webkit.org/browser/
> webkit/trunk/Tools/Scripts/webkitpy/common/config/ews.json#L45 it seems like
> Apple JSC EWS uses 'mac' port, while linux jsc EWSes use 'jsc-only' port. Is
> jsc-only port specific to linux? Is there a corresponding jsc port for mac?
> 
> 2) Is there any difference between the three linux JSC EWSes (JSC i386, JSC
> MIPS32el, JSC Armv7) apart from the architecture and few queues running/
> skipping tests? 

I think the main difference is indeed the different architecture. There
is also the fact that the JSC i386 EWS runs the tests natively, but the
mips and armv7 EWSes cross-compile and run the tests on a set of remote
devices.

> 
> 3) Where is the architecture for various linux JSC EWS queues configured in 
> the
> old EWS code (e.g.: I don't see the architecture being added in https://
> trac.webkit.org/changeset/237001/webkit)? Is the queue configuration for
> various JSC EWS queues same, and the device connected to the queue actually
> decides the architecture?

I guess it's not defined anywhere indeed. The scripts that start the
queues for the arm and mips EWSes define a bunch of env variables to
point to the cross-compiler and also add a few options in
JSCTESTS_OPTIONS, such as a  --remote-config-file and --memory-limited.
I can share these scripts with you if you need more details.

Hope this helps.

Cheers,

Guij

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev