Re: [webkit-dev] Safari Tech Preview 96 available on wpt.fyi!

[Stephen Mcgruer]

>
> There’s a number of mysterious timeouts with 96. Not sure if flakiness or
> real?
> Many large new chunks of not-run tests are caused by a harness error or
> timeout. E.g. html/ and webauthn


We have unfortunately struggled historically with getting reliable +
consistent results for Safari, with spurious infrastructure + other
problems. That said, Igalia has been working on improving reliability and
have made good progress, and we have also invested in projects like the
WebkitGTK runs to help with this problem (e.g. if a test fails in both
Safari and WebKitGTK, there's a better chance it's a real failure[0])

We have had many Safari TP 96 runs since my announcement (
https://wpt.fyi/runs?label=master=100=safari); you can
click on a single run (the icon) to view the run itself, or click on two
runs to get a diff view - which can be useful for spotting flakes (as most
SHAs are only changing a few tests, so differences between consecutive runs
are usually flakes). There is also an explicit flake analyzer on our
insights page - https://wpt.fyi/insights (it's a little clunky but can
still be useful).

I believe these tests are flaky. I have made a PR to improve it a while
> ago. I should probably get those improvement landed sometime.


Improvements would be great! Looking at that PR, it is quite old, so will
need to be rebased and force pushed. Let me know if you need any help on
that :).

STP 96 has enabled user agent UI for WebAuthn, which could cause crashes.
> This could be the reason why webauthn is 0 all the time.


It does look like there are still some harness failures in further STP 96
runs (e.g. https://wpt.fyi/results/webauthn?run_id=361070008); I see a
mention in the failing tests of a "UnexpectedAlertOpenException: unexpected
alert open (500):" - is that what you mean? Is there some flag we could be
setting on STP, or something Safaridriver could be doing to avoid these?

[0]:
https://wpt.fyi/results/?label=master=safari%5Bexperimental%5D=webkitgtk=chrome%5Bexperimental%5D=firefox%5Bexperimental%5D=safari%3Afail%20webkitgtk%3Afail%20chrome%3Apass%20firefox%3Apass
is
an example search for tests that fail in both Safari + WebKitGTK, but pass
in both Chrome and Firefox.

On Mon, 2 Dec 2019 at 21:04, Jiewen Tan  wrote:

> Hi Maciej,
>
> On Dec 2, 2019, at 4:10 PM, Maciej Stachowiak  wrote:
>
>
> There’s a number of mysterious timeouts with 96. Not sure if flakiness or
> real?
>
> The new WebCrypto failures are surprising, but likely real and should be
> investigated:
> https://wpt.fyi/results/WebCryptoAPI/wrapKey_unwrapKey?diff=ADC=is%3Adifferent_id=347530011_id=381930013
> 
>
>
> I believe these tests are flaky. I have made a PR to improve it a while
> ago. I should probably get those improvement landed sometime.
> https://github.com/web-platform-tests/wpt/pull/6102
>
>
> Many large new chunks of not-run tests are caused by a harness error or
> timeout. E.g. html/ and webauthn/
>
>
> STP 96 has enabled user agent UI for WebAuthn, which could cause crashes.
> This could be the reason why webauthn is 0 all the time.
>
> Best,
> Jiewen
>
>
>
> On Nov 27, 2019, at 7:07 AM, Stephen Mcgruer 
> wrote:
>
> Excited to announce that Safari Tech Preview 96 is now available on
> wpt.fyi!
>
> Example run:
> https://wpt.fyi/results/?label=master=experimental=chrome=firefox=safari
>
> Diff against Safari Tech Preview 95 (not at exactly the same WPT sha, but
> very close):
> https://wpt.fyi/results/?diff=ADC=is%3Adifferent_id=347530011_id=381930013
>
> If you are surprised by the results in the diff view, I would love to hear
> about it. One thing that surprised me is that the release blog post[0]
> mentioned Web Animations being enabled by default, but we see no
> differences in the test diff for web-animations/. It's possible we enable
> some flag that turned on Web Animations already, but I can't see it
> obviously in our safari setup[1].
>
> [0]:
> https://webkit.org/blog/9658/release-notes-for-safari-technology-preview-96/
> [1]:
> https://github.com/web-platform-tests/wpt/blob/master/tools/ci/azure/install_safari.yml
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Safari Tech Preview 96 available on wpt.fyi!

[Jiewen Tan]

Hi Maciej,

> On Dec 2, 2019, at 4:10 PM, Maciej Stachowiak  wrote:
> 
> 
> There’s a number of mysterious timeouts with 96. Not sure if flakiness or 
> real?
> 
> The new WebCrypto failures are surprising, but likely real and should be 
> investigated: 
> https://wpt.fyi/results/WebCryptoAPI/wrapKey_unwrapKey?diff=ADC=is%3Adifferent_id=347530011_id=381930013
>  
> 
I believe these tests are flaky. I have made a PR to improve it a while ago. I 
should probably get those improvement landed sometime.
https://github.com/web-platform-tests/wpt/pull/6102 

> 
> Many large new chunks of not-run tests are caused by a harness error or 
> timeout. E.g. html/ and webauthn/

STP 96 has enabled user agent UI for WebAuthn, which could cause crashes. This 
could be the reason why webauthn is 0 all the time.

Best,
Jiewen

> 
> 
>> On Nov 27, 2019, at 7:07 AM, Stephen Mcgruer > > wrote:
>> 
>> Excited to announce that Safari Tech Preview 96 is now available on wpt.fyi 
>> !
>> 
>> Example run: 
>> https://wpt.fyi/results/?label=master=experimental=chrome=firefox=safari
>>  
>> 
>> 
>> Diff against Safari Tech Preview 95 (not at exactly the same WPT sha, but 
>> very close): 
>> https://wpt.fyi/results/?diff=ADC=is%3Adifferent_id=347530011_id=381930013
>>  
>> 
>> 
>> If you are surprised by the results in the diff view, I would love to hear 
>> about it. One thing that surprised me is that the release blog post[0] 
>> mentioned Web Animations being enabled by default, but we see no differences 
>> in the test diff for web-animations/. It's possible we enable some flag that 
>> turned on Web Animations already, but I can't see it obviously in our safari 
>> setup[1].
>> 
>> [0]: 
>> https://webkit.org/blog/9658/release-notes-for-safari-technology-preview-96/ 
>> 
>> [1]: 
>> https://github.com/web-platform-tests/wpt/blob/master/tools/ci/azure/install_safari.yml
>>  
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org 
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Aakash Jain]

There were multiple ideas discussed in this thread. I would like to gather more 
data about what do most people prefer. I have sent out a short survey in 
https://lists.webkit.org/pipermail/webkit-dev/2019-December/030980.html 


Thanks
Aakash

> On Nov 5, 2019, at 12:04 PM, Alexey Proskuryakov  wrote:
> 
> 
> 
>> 4 нояб. 2019 г., в 1:37 PM, Ryosuke Niwa > > написал(а):
>> 
>> 
>> On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov > > wrote:
>> 
>> Can you elaborate on that, how exactly is e-mailing on first failure useful 
>> to reviewers?
>> 
>> Getting rid of Bugzilla comments was one of the goals of EWS rewrite, based 
>> on engineering feedback about noise in bugs and in e-mail, and I 
>> wholeheartedly agree with this feedback. So I think that comments are 
>> generally undesirable.
>> 
>> Since I don't understand what your precise scenario is, I may be make straw 
>> man arguments below, but here are some things that I think make the proposed 
>> behavior unhelpful (add a comment on first failure, or when all EWSes pass).
>> 
>> 1. EWS comments in Bugzilla are so annoying that some people take the 
>> radical step of manually hiding them. EWS history is archived anyway, there 
>> is no need to look into comments for it.
>> 
>> 2. There are often many people CC'ed on the bug to whom EWS data is 
>> irrelevant or even mysterious (e.g. reporters, web developers or 
>> non-reviewers). The noise is a slight annoyance, discouraging further 
>> participation in the project.
>> 
>> 3. I believe that for most reviewers, the mode of operation is one of the 
>> two: (1) do it when pinged directly, or (2) go over the review queue when 
>> one has the time. Getting EWS comments helps neither.
>> 
>> 4. Commenting when all EWSes pass is not very practical - it's too often 
>> that we have some stragglers that take days (or forever). I don't think that 
>> we can make it reliable even if we start actively policing EWS 
>> responsiveness.
>> 
>> 5. The reviewer likely wants to know the state of multiple EWSes if they are 
>> going to wait for EWS at all. What exactly are they going to do after 
>> getting an e-mail that one EWS failed?
>> 
>> I often use a EWS failure as a signal to wait reviewing a patch. Otherwise, 
>> a bug mail will stay in my inbox as one of items to get to.
>> 
>> I can see the usefulness in the somewhat unusual case of a super urgent 
>> patch. We may want multiple people to watch it, so that members of CC list 
>> would go and ask the patch author to update it with more urgency than e-mail 
>> allows for. I think that opt-in is a better mechanism for that, so that 
>> people who opted in would receive information about each EWS data point.
>> 
>> I think there is a value in knowing that a patch isn't ready instead of 
>> having to open the bug to realize that.
> 
> So just to clarify, 
> 
> - a major part of how you get to review bugs is by being CC'ed, and you 
> review them when you have the time to read bugmail;
> - and you don't open the bug in Bugzilla if there is already an EWS failure 
> by the time you read the e-mail where review is requested?
> 
> That's clearly a valid benefit. In my mind, it probably doesn't outweigh the 
> downsides. On the other hand, yours is a voice of someone who reviews way 
> more patches than Maciej and me combined these days, so maybe more e-mail is 
> an overall benefit to many of the reviewers.
> 
> - Alexey
> 
> 
> 
>> - R. Niwa
>>> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak >> > написал(а):
>>> 
>>> 
>>> I think they are useful to actual and potential reviewers. Direct email to 
>>> the patch author is not something anyone can Cc themselves on, and is not 
>>> archived, so seems like a strictly worse form of communication.
>>> 
 On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov >>> > wrote:
 
 
 My preference is still e-mailing the patch author directly (possibly, also 
 having an option to opt in for anyone). Bugzilla comments will always be 
 irrelevant for most people CC'ed on the bug, and they are almost always 
 undesirable to keep within the discussion flow.
 
 - Alexey
 
> 1 нояб. 2019 г., в 18:28, Aakash Jain  > написал(а):
> 
> Sounds good. I prefer the single comment when the first failure occur. 
> That way notification would be sent as soon as the first failure happens.
> 
> I'll implement that (assuming it's acceptable to everyone).
> 
> Thanks
> Aakash
> 
>> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak > > wrote:
>> 
>> 
>> How about only a single comment when the first failure occurs? (Or else 
>> when all bots pass, if there is never a failure.)
>> 
>> This should help the author, the reviewer, and 

[webkit-dev] Survey regarding EWS

[Aakash Jain]

Hi Everyone,

I want to get everyone's opinion on EWS, especially regarding if (and how) EWS 
should send notifications when a patch fails on any EWS queue. Here is a link 
to a short survey: https://www.surveymonkey.com/r/XQ77LQD 


I would appreciate if you can fill it out.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Safari Tech Preview 96 available on wpt.fyi!

[Maciej Stachowiak]

There’s a number of mysterious timeouts with 96. Not sure if flakiness or real?

The new WebCrypto failures are surprising, but likely real and should be 
investigated: 
https://wpt.fyi/results/WebCryptoAPI/wrapKey_unwrapKey?diff=ADC=is%3Adifferent_id=347530011_id=381930013
 


Many large new chunks of not-run tests are caused by a harness error or 
timeout. E.g. html/ and webauthn/


> On Nov 27, 2019, at 7:07 AM, Stephen Mcgruer  wrote:
> 
> Excited to announce that Safari Tech Preview 96 is now available on wpt.fyi 
> !
> 
> Example run: 
> https://wpt.fyi/results/?label=master=experimental=chrome=firefox=safari
>  
> 
> 
> Diff against Safari Tech Preview 95 (not at exactly the same WPT sha, but 
> very close): 
> https://wpt.fyi/results/?diff=ADC=is%3Adifferent_id=347530011_id=381930013
>  
> 
> 
> If you are surprised by the results in the diff view, I would love to hear 
> about it. One thing that surprised me is that the release blog post[0] 
> mentioned Web Animations being enabled by default, but we see no differences 
> in the test diff for web-animations/. It's possible we enable some flag that 
> turned on Web Animations already, but I can't see it obviously in our safari 
> setup[1].
> 
> [0]: 
> https://webkit.org/blog/9658/release-notes-for-safari-technology-preview-96/ 
> 
> [1]: 
> https://github.com/web-platform-tests/wpt/blob/master/tools/ci/azure/install_safari.yml
>  
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Feedback on Blink's text fragment directive proposal

[David Bokan]

Hello webkit-dev,

I'd like to solicit feedback as well as an official position from Webkit on
our proposal for the text fragment directive:
https://github.com/WICG/ScrollToTextFragment.

In summary: this is a feature that allows authors and users to craft URLs
to pages and specify a snippet of text on the page as a subresource
(visually highlighting it and scrolling it into view). Analogous to
element-id based fragment anchors but for text.

You can try this out today in Chrome Beta by enabling
chrome://flags/#enable-text-fragment-anchor.
Here's an example link:

https://en.wikipedia.org/wiki/Uniform_Resource_Identifier#:~:text=An%20optional%20fragment,element%20into%20view

Relevant Links:


Explainer

Spec 
TAG Review 
(Currently Suspended) Blink Intent Thread

Issue on Mozilla standards-positions


We've been using the GitHub repo for issue tracking but happy to take
feedback (official or otherwise) in any form.

Thank you!
David
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev