Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

2019-12-03 Thread Ryosuke Niwa
Great. I've completed the survey. - R. Niwa On Mon, Dec 2, 2019 at 5:19 PM Aakash Jain wrote: > There were multiple ideas discussed in this thread. I would like to gather > more data about what do most people prefer. I have sent out a short survey > in

Re: [webkit-dev] Handling flaky layout-test failures in EWS

2019-12-03 Thread Ryosuke Niwa
On Tue, Dec 3, 2019 at 9:29 AM Alexey Proskuryakov wrote: > > Yes, I think that this makes more sense than retrying. > > What is the current behavior when a patch introduces substantial > flakiness? E.g. this scenario: > > - First test run produces 5 failures. > - Second test run produces 5

Re: [webkit-dev] Handling flaky layout-test failures in EWS

2019-12-03 Thread Alexey Proskuryakov
Yes, I think that this makes more sense than retrying. What is the current behavior when a patch introduces substantial flakiness? E.g. this scenario: - First test run produces 5 failures. - Second test run produces 5 different failures. - Clean re-run produces no failures. This looks like

[webkit-dev] Handling flaky layout-test failures in EWS

2019-12-03 Thread Aakash Jain
Hi Everyone, We have various layout-tests which are flaky (which sometimes pass and sometimes fail/crash/timeout). EWS needs to work despite these flaky tests, and need to be able to tell whether the patch being tested introduced any test failure or not. In EWS, we have logic (same logic in

Re: [webkit-dev] Safari Tech Preview 96 available on wpt.fyi!

2019-12-03 Thread Philip Jägenstedt
On Tue, Dec 3, 2019 at 3:05 AM Jiewen Tan wrote: > Hi Maciej, > > On Dec 2, 2019, at 4:10 PM, Maciej Stachowiak wrote: > > > There’s a number of mysterious timeouts with 96. Not sure if flakiness or > real? > > The new WebCrypto failures are surprising, but likely real and should be >