Re: [webkit-dev] Planned EWS improvements

[Aakash Jain]

Hi Everyone,

I am very happy to announce that I have added the ability to retry the patches 
in EWS :)

From now on, for any patch, whenever any of the EWS status-bubble is red, you 
will see another button named 'Retry failed builds' next to the status-bubbles. 
Clicking on 'Retry failed builds' button would cause all the failed EWS builds 
for that patch to be retried. Please click on this button only when needed, as 
unnecessary retries might put load on the system and others patches might have 
to wait longer.

As always, please feel free to let me know if you notice any issue or have any 
feedback.

Known UI issues: https://bugs.webkit.org/show_bug.cgi?id=203249 


Thanks
Aakash


> On Sep 26, 2019, at 7:56 PM, Saam Barati  wrote:
> 
> Hi Aakash,
> 
> Thanks for doing this work.
> 
>> On Sep 26, 2019, at 11:27 AM, Aakash Jain > > wrote:
>> 
>> ...
>> 
>> 3) Add a way to retry a patch in EWS. This would allow engineers to confirm 
>> that the failures indicated by EWS aren't flaky/incorrect. Maybe a good 
>> place to add the 'retry' button would be the status-bubble's tool-tip 
>> (visible only if the bubble is red) https://webkit.org/b/196599 
>> This would be amazing. (My 2c: I'd vote for the 
>> button being visible without tooltip.)
> 
> - Saam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Planned EWS improvements

[Aakash Jain]

> On Sep 30, 2019, at 9:32 AM, Adrian Perez de Castro  wrote:
> 
> Hello Aakash (and everybody else),
> 
> On Thu, 26 Sep 2019 14:27:25 -0400, Aakash Jain  wrote:
> 
>> I have been making number of improvements to EWS. I also have various
>> planned improvements to EWS […]
> 
> Thanks once again for your continued work on improving the EWS, it is
> much appreciated =)

Thank You for the feedback. I am hoping that Improved EWS would increase 
overall team's productivity as people have to deal less with build/test 
breakages and rollouts.

> 
>> Here is the list of improvements (in no particular order):
>> 
>> […]
>> 
>> 3) Add a way to retry a patch in EWS. This would allow engineers to confirm
>> that the failures indicated by EWS aren't flaky/incorrect. Maybe a good
>> place to add the 'retry' button would be the status-bubble's tool-tip
>> (visible only if the bubble is red) https://webkit.org/b/196599
>> 
> 
> Being able to retry a patch without having to re-upload it, and more
> importantly without having to ping another person, would be neat. I
> don't have a strong opinion about which tasks to prioritize first, tho.
> 
>> […]
>> 
>> 6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests)
>> https://webkit.org/b/189206 ,
>> https://webkit.org/b/201928 
> 
> In this line of adding more things that the EWS could test, among the
> developers of the GTK/WPE ports we think it would be manageable for us
> to have bots run API tests. There is some work we need to do, and the
> intention is to start with GTK first, see how it goes, and then add WPE
> afterwards. Carlos opened a bug for this, I took the liberty of CC'ing
> you assuming that you might want to at least keep tabs on it (feel free
> to unsubscribe):
> 
>  https://bugs.webkit.org/show_bug.cgi?id=202361

Sure, I will work with you and Carlos on this.

> 
> Best regards,
> —Adrián

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Planned EWS improvements

[Adrian Perez de Castro]

Hello Aakash (and everybody else),

On Thu, 26 Sep 2019 14:27:25 -0400, Aakash Jain  wrote:
 
> I have been making number of improvements to EWS. I also have various
> planned improvements to EWS […]

Thanks once again for your continued work on improving the EWS, it is
much appreciated =)

> Here is the list of improvements (in no particular order):
> 
> […]
>
> 3) Add a way to retry a patch in EWS. This would allow engineers to confirm
> that the failures indicated by EWS aren't flaky/incorrect. Maybe a good
> place to add the 'retry' button would be the status-bubble's tool-tip
> (visible only if the bubble is red) https://webkit.org/b/196599
> 

Being able to retry a patch without having to re-upload it, and more
importantly without having to ping another person, would be neat. I
don't have a strong opinion about which tasks to prioritize first, tho.

> […]
> 
> 6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests)
> https://webkit.org/b/189206 ,
> https://webkit.org/b/201928 

In this line of adding more things that the EWS could test, among the
developers of the GTK/WPE ports we think it would be manageable for us
to have bots run API tests. There is some work we need to do, and the
intention is to start with GTK first, see how it goes, and then add WPE
afterwards. Carlos opened a bug for this, I took the liberty of CC'ing
you assuming that you might want to at least keep tabs on it (feel free
to unsubscribe):

  https://bugs.webkit.org/show_bug.cgi?id=202361

Best regards,
—Adrián


signature.asc
Description: PGP signature
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Planned EWS improvements

[Simon Fraser]

> On Sep 27, 2019, at 10:27 PM, Konstantin Tokarev  wrote:
> 
> 27.09.2019, 08:15, "Simon Fraser" :
>>> On Sep 27, 2019, at 3:27 AM, Aakash Jain  wrote:
>>> 
>>> Hi Everyone,
>>> 
>>> I have been making number of improvements to EWS. I also have various 
>>> planned improvements to EWS. I wanted to reach out to you guys to see if 
>>> anyone
>> 
>> Not everyone on this list is a guy.
> 
> FWIW, plural form of "guy" is often used as gender-neutral vocative, see e.g. 
> [1,2]
> 
> [1] https://www.merriam-webster.com/dictionary/guy
> [2] https://dictionary.cambridge.org/us/dictionary/english/guy

I apologize to Aakash for calling this out publicly.

However, ‘guys’ clearly has gender-specific roots, and to my ears, it sounds 
male-biased, There are many other words that can be used in its place which are 
more inclusive (“folks”, “everyone”, “you all” etc).

Simon

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Planned EWS improvements

[Aakash Jain]

Thank You everyone (who responded) for the feedback.

I will prioritize the features some of you mentioned, especially adding a way 
to retry the patch.

Thanks
Aakash

> On Sep 27, 2019, at 9:27 AM, Konstantin Tokarev  wrote:
> 
> 
> 
> 27.09.2019, 08:15, "Simon Fraser" :
>>> On Sep 27, 2019, at 3:27 AM, Aakash Jain  wrote:
>>> 
>>> Hi Everyone,
>>> 
>>> I have been making number of improvements to EWS. I also have various 
>>> planned improvements to EWS. I wanted to reach out to you guys to see if 
>>> anyone
>> 
>> Not everyone on this list is a guy.
> 
> FWIW, plural form of "guy" is often used as gender-neutral vocative, see e.g. 
> [1,2]
> 
> [1] https://www.merriam-webster.com/dictionary/guy
> [2] https://dictionary.cambridge.org/us/dictionary/english/guy
> 
> 
> -- 
> Regards,
> Konstantin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Planned EWS improvements

[Simon Fraser]

> On Sep 27, 2019, at 3:27 AM, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> I have been making number of improvements to EWS. I also have various planned 
> improvements to EWS. I wanted to reach out to you guys to see if anyone

Not everyone on this list is a guy.

> wants me to prioritize any particular improvement(s). If there is any 
> improvement which you want to see and is not listed below, please feel free 
> to let me know. Also most of the queues have been transitioned from old to 
> new EWS and I am working on the remaining ones (jsc, windows and 
> commit-queue).
> 
> Here is the list of improvements (in no particular order):
> 
> 1) Develop a webpage showing summary of EWS builds for a patch. This page 
> would provide the summary of important build-steps, high-level details about 
> the failure (e.g.: name of the tests which failed, or possibly relevant build 
> failure logs), and include link(s) to the Buildbot page(s). This page will 
> open on clicking the status-bubbles (and would be replacement of old EWS 
> status page like https://webkit-queues.webkit.org/patch/379563/win-ews 
> ). Currently clicking 
> the status-bubble opens the Buildbot build page, which contains a lot of 
> infrastructure details, and probably is information-overload for many 
> engineers, so this summary page should help with that. 
> https://webkit.org/b/197522 
> 
> 2) Redesign status-bubble tooltip to include more detailed information about 
> failures (e.g.: each test failure name along-with url to flakiness dashboard, 
> and url to complete results.html file, as suggested by David Kilzer in 
> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html 
> ). 
> We should also add the tooltip support for iPad/iPhone 
> https://webkit.org/b/201940 

#1

> 
> 3) Add a way to retry a patch in EWS. This would allow engineers to confirm 
> that the failures indicated by EWS aren't flaky/incorrect. Maybe a good place 
> to add the 'retry' button would be the status-bubble's tool-tip (visible only 
> if the bubble is red) https://webkit.org/b/196599 
> 
#2

> 
> 4) Parse the relevant build failure message from build logs (and display in 
> summary page) https://webkit.org/b/201941 
> 
> 5) Style failure should be displayed in-line on the review page along-with 
> the code, just like the reviewer's comments https://webkit.org/b/202252 
> 
> 
> 6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests) 
> https://webkit.org/b/189206 , 
> https://webkit.org/b/201928 
> 
> 7) Add commit-queue support for security bugs https://webkit.org/b/201939 
> 
> 
> 8) API tests should upload crashlogs https://webkit.org/b/201929 
> 

Simon

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Planned EWS improvements

[Ryosuke Niwa]

On Fri, Sep 27, 2019 at 7:57 AM Saam Barati  wrote:

> Hi Aakash,
>
> Thanks for doing this work.
>
> On Sep 26, 2019, at 11:27 AM, Aakash Jain  wrote:
>
> Hi Everyone,
>
> I have been making number of improvements to EWS. I also have various
> planned improvements to EWS. I wanted to reach out to you guys to see if
> anyone wants me to prioritize any particular improvement(s). If there is
> any improvement which you want to see and is not listed below, please feel
> free to let me know. Also most of the queues have been transitioned from
> old to new EWS and I am working on the remaining ones (jsc, windows and
> commit-queue).
>
> Here is the list of improvements (in no particular order):
>
> 1) Develop a webpage showing summary of EWS builds for a patch. This page
> would provide the summary of important build-steps, high-level details
> about the failure (e.g.: name of the tests which failed, or possibly
> relevant build failure logs), and include link(s) to the Buildbot page(s).
> This page will open on clicking the status-bubbles (and would be
> replacement of old EWS status page like
> https://webkit-queues.webkit.org/patch/379563/win-ews). Currently
> clicking the status-bubble opens the Buildbot build page, which contains a
> lot of infrastructure details, and probably is information-overload for
> many engineers, so this summary page should help with that.
> https://webkit.org/b/197522
>
> 2) Redesign status-bubble tooltip to include more detailed information
> about failures (e.g.: each test failure name along-with url to flakiness
> dashboard, and url to complete results.html file, as suggested by David
> Kilzer in
> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html).
> We should also add the tooltip support for iPad/iPhone
> https://webkit.org/b/201940
>
> 3) Add a way to retry a patch in EWS. This would allow engineers to
> confirm that the failures indicated by EWS aren't flaky/incorrect. Maybe a
> good place to add the 'retry' button would be the status-bubble's tool-tip
> (visible only if the bubble is red) https://webkit.org/b/196599
>
> This would be amazing. (My 2c: I'd vote for the button being visible
> without tooltip.)
>

Yeah, this would be the most useful addition to EWS.

4) Parse the relevant build failure message from build logs (and display in
> summary page) https://webkit.org/b/201941
>
> 5) Style failure should be displayed in-line on the review page along-with
> the code, just like the reviewer's comments https://webkit.org/b/202252
>
> 6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests)
> https://webkit.org/b/189206, https://webkit.org/b/201928
>
> 7) Add commit-queue support for security bugs https://webkit.org/b/201939
>
> 8) API tests should upload crashlogs https://webkit.org/b/201929
>
>
Along with this one.

- R. Niwa

-- 
- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Planned EWS improvements

[Saam Barati]

Hi Aakash,

Thanks for doing this work.

> On Sep 26, 2019, at 11:27 AM, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> I have been making number of improvements to EWS. I also have various planned 
> improvements to EWS. I wanted to reach out to you guys to see if anyone wants 
> me to prioritize any particular improvement(s). If there is any improvement 
> which you want to see and is not listed below, please feel free to let me 
> know. Also most of the queues have been transitioned from old to new EWS and 
> I am working on the remaining ones (jsc, windows and commit-queue).
> 
> Here is the list of improvements (in no particular order):
> 
> 1) Develop a webpage showing summary of EWS builds for a patch. This page 
> would provide the summary of important build-steps, high-level details about 
> the failure (e.g.: name of the tests which failed, or possibly relevant build 
> failure logs), and include link(s) to the Buildbot page(s). This page will 
> open on clicking the status-bubbles (and would be replacement of old EWS 
> status page like https://webkit-queues.webkit.org/patch/379563/win-ews 
> ). Currently clicking 
> the status-bubble opens the Buildbot build page, which contains a lot of 
> infrastructure details, and probably is information-overload for many 
> engineers, so this summary page should help with that. 
> https://webkit.org/b/197522 
> 
> 2) Redesign status-bubble tooltip to include more detailed information about 
> failures (e.g.: each test failure name along-with url to flakiness dashboard, 
> and url to complete results.html file, as suggested by David Kilzer in 
> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html 
> ). 
> We should also add the tooltip support for iPad/iPhone 
> https://webkit.org/b/201940 
> 
> 3) Add a way to retry a patch in EWS. This would allow engineers to confirm 
> that the failures indicated by EWS aren't flaky/incorrect. Maybe a good place 
> to add the 'retry' button would be the status-bubble's tool-tip (visible only 
> if the bubble is red) https://webkit.org/b/196599 
> This would be amazing. (My 2c: I'd vote for the 
> button being visible without tooltip.)

- Saam

> 
> 4) Parse the relevant build failure message from build logs (and display in 
> summary page) https://webkit.org/b/201941 
> 
> 5) Style failure should be displayed in-line on the review page along-with 
> the code, just like the reviewer's comments https://webkit.org/b/202252 
> 
> 
> 6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests) 
> https://webkit.org/b/189206 , 
> https://webkit.org/b/201928 
> 
> 7) Add commit-queue support for security bugs https://webkit.org/b/201939 
> 
> 
> 8) API tests should upload crashlogs https://webkit.org/b/201929 
> 
> 
> Thanks
> Aakash
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Planned EWS improvements

[Aakash Jain]

Hi Everyone,

I have been making number of improvements to EWS. I also have various planned 
improvements to EWS. I wanted to reach out to you guys to see if anyone wants 
me to prioritize any particular improvement(s). If there is any improvement 
which you want to see and is not listed below, please feel free to let me know. 
Also most of the queues have been transitioned from old to new EWS and I am 
working on the remaining ones (jsc, windows and commit-queue).

Here is the list of improvements (in no particular order):

1) Develop a webpage showing summary of EWS builds for a patch. This page would 
provide the summary of important build-steps, high-level details about the 
failure (e.g.: name of the tests which failed, or possibly relevant build 
failure logs), and include link(s) to the Buildbot page(s). This page will open 
on clicking the status-bubbles (and would be replacement of old EWS status page 
like https://webkit-queues.webkit.org/patch/379563/win-ews 
). Currently clicking 
the status-bubble opens the Buildbot build page, which contains a lot of 
infrastructure details, and probably is information-overload for many 
engineers, so this summary page should help with that. 
https://webkit.org/b/197522 

2) Redesign status-bubble tooltip to include more detailed information about 
failures (e.g.: each test failure name along-with url to flakiness dashboard, 
and url to complete results.html file, as suggested by David Kilzer in 
https://lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html 
). We 
should also add the tooltip support for iPad/iPhone https://webkit.org/b/201940 


3) Add a way to retry a patch in EWS. This would allow engineers to confirm 
that the failures indicated by EWS aren't flaky/incorrect. Maybe a good place 
to add the 'retry' button would be the status-bubble's tool-tip (visible only 
if the bubble is red) https://webkit.org/b/196599 

4) Parse the relevant build failure message from build logs (and display in 
summary page) https://webkit.org/b/201941 

5) Style failure should be displayed in-line on the review page along-with the 
code, just like the reviewer's comments https://webkit.org/b/202252 


6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests) 
https://webkit.org/b/189206 , 
https://webkit.org/b/201928 

7) Add commit-queue support for security bugs https://webkit.org/b/201939 


8) API tests should upload crashlogs https://webkit.org/b/201929 


Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev