Re: [webkit-dev] Another WPT bite

2017-05-12 Thread youenn fablet
I filed https://bugs.webkit.org/show_bug.cgi?id=172068 to track the need
for some extra tooling for HTTP/WPT served tests.
We already gathered information about related requirements & workflows here.
Let's add more there!

Le ven. 12 mai 2017 à 19:50,  a écrit :

>
> 12 мая 2017 г., в 19:38, Brian Burg  написал(а):
>
>
> I think that I explained it very clearly, but let me try again.
>
> When there is a test failure that I need to communicate to others, I say
> something "please open <
> https://trac.webkit.org/export/216812/webkit/trunk/LayoutTests/fast/images/destroyed-image-load-event.html
> >
> in Safari to reproduce". That's very easy to do, and makes it very easy for
> others to work on the issue.
> If your test requires complex setup, like WPT does, then I may not have
> the time to write up complicated steps to reproduce, or the person who gets
> the bug may not have the time to follow them. Those people don't have a
> WebKit checkout, so scripts won't help. This makes the test less effective,
> as problems that it finds are less likely to be addressed.
>
>
> If the person works on WebKit, then it seems unreasonable that they would
> do work without a checkout.
>
>
> It is correct that people who work on WebKit usually have a checkout. So I
> was taking about people who don't work on WebKit.
>
> If they don’t work on WebKit, then you could run wptserve on a machine
> somewhere and link to that copy. We have several servers that exist solely
> to host test content, it doesn’t seem like a big deal to make one of them
> update regularly and relaunch wptserve to pick up test harness changes.
>
>
> Yes, there is a number of things one could do. Those things would work in
> some cases but not in others - I mentioned linking to a stable version that
> won't change, which is something that trac gives us for free, and it would
> be non-trivial to implement otherwise.
>
> In practice, the best approach would be to reduce the test to a minimum
> that doesn't use complex harnesses before ending it over. Everyone likes
> minimal test cases.
>
> - Alexey
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Anne van Kesteren
On Fri, May 12, 2017 at 11:49 PM, Maciej Stachowiak  wrote:
> It seems like there's two unusual things about WPT:
> - At least according to Alexey, WPT tests are somewhat prone to flakiness in 
> Safari.

Although they haven't always been working perfectly, changes to
web-platform-tests run through some kind of stability check in both
Chrome and Firefox (run the new tests 10 times and fail if the results
are inconsistent). Ideally Safari is added to that mix, though I think
the last time folks looked into that it wasn't possible for some
reason. That might be something to put some resources on.


-- 
https://annevankesteren.nl/
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Brian Burg

> On May 12, 2017, at 7:31 PM, Alexey Proskuryakov  wrote:
> 
>> 
>> 12 мая 2017 г., в 16:12, Sam Weinig > > написал(а):
>> 
>> 
>> 
>>> On May 12, 2017, at 2:49 PM, Alexey Proskuryakov >> > wrote:
>>> 
>>> 
 12 мая 2017 г., в 14:38, Sam Weinig > написал(а):
 
 I regret piling on here, as I think this thread has diverged from it’s 
 original purpose, but…I understand this frustration. That said, perhaps 
 this is something we can solve with some tooling. For instance, a 
 run-test-in-safari (as a parallel to run-safari) script could be made 
 which starts the server, and then loads the test with the right URL in 
 your built Safari (or MiniBrowser, or whatever).  
>>> 
>>> 
>>> That's still not good enough, as this means that I can't point someone else 
>>> to an instance of the test on trac.webkit.org  to 
>>> reproduce an issue. There is of course be another place to point to when/if 
>>> the test gets upstreamed, but even that doesn't support stable links like 
>>> trac does.
>>> 
>>> That's not to mention that learning the name of this proposed script is no 
>>> easier than learning about run-webkit-httpd.
>>> 
>>> - Alexey
>>> 
>> 
>> 
>> I’m not sure what you mean by “good enough”.  Good enough for what?  What 
>> are we debating here?
> 
> I think that I explained it very clearly, but let me try again.
> 
> When there is a test failure that I need to communicate to others, I say 
> something "please open 
>   
> >
>  in Safari to reproduce". That's very easy to do, and makes it very easy for 
> others to work on the issue.
> If your test requires complex setup, like WPT does, then I may not have the 
> time to write up complicated steps to reproduce, or the person who gets the 
> bug may not have the time to follow them. Those people don't have a WebKit 
> checkout, so scripts won't help. This makes the test less effective, as 
> problems that it finds are less likely to be addressed.

If the person works on WebKit, then it seems unreasonable that they would do 
work without a checkout.

If they don’t work on WebKit, then you could run wptserve on a machine 
somewhere and link to that copy. We have several servers that exist solely to 
host test content, it doesn’t seem like a big deal to make one of them update 
regularly and relaunch wptserve to pick up test harness changes.

> 
> - Alexey
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org 
> https://lists.webkit.org/mailman/listinfo/webkit-dev 
> 
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread ap

> 12 мая 2017 г., в 19:38, Brian Burg  написал(а):
> 
>> I think that I explained it very clearly, but let me try again.
>> 
>> When there is a test failure that I need to communicate to others, I say 
>> something "please open 
>> >  
>> >
>>  in Safari to reproduce". That's very easy to do, and makes it very easy for 
>> others to work on the issue.
>> If your test requires complex setup, like WPT does, then I may not have the 
>> time to write up complicated steps to reproduce, or the person who gets the 
>> bug may not have the time to follow them. Those people don't have a WebKit 
>> checkout, so scripts won't help. This makes the test less effective, as 
>> problems that it finds are less likely to be addressed.
> 
> If the person works on WebKit, then it seems unreasonable that they would do 
> work without a checkout.

It is correct that people who work on WebKit usually have a checkout. So I was 
taking about people who don't work on WebKit.

> If they don’t work on WebKit, then you could run wptserve on a machine 
> somewhere and link to that copy. We have several servers that exist solely to 
> host test content, it doesn’t seem like a big deal to make one of them update 
> regularly and relaunch wptserve to pick up test harness changes.

Yes, there is a number of things one could do. Those things would work in some 
cases but not in others - I mentioned linking to a stable version that won't 
change, which is something that trac gives us for free, and it would be 
non-trivial to implement otherwise.

In practice, the best approach would be to reduce the test to a minimum that 
doesn't use complex harnesses before ending it over. Everyone likes minimal 
test cases.

- Alexey

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Ryosuke Niwa
On Fri, May 12, 2017 at 7:38 PM, Brian Burg  wrote:
>
> If the person works on WebKit, then it seems unreasonable that they would do 
> work without a checkout.

This is not entirely true. If I have an iOS device, it's a lot easier
to load up a web page on a public server somewhere rather than setting
up a server on my mac, and then making it visible to my iPhone / iPad.

Also, I find it extremely useful to be able to run a test without
having a WebKit checkout since I install many different OS'es on many
Macs in order to figure out when a regression got introduced in the
underlying system itself, etc...

> If they don’t work on WebKit, then you could run wptserve on a machine
> somewhere and link to that copy. We have several servers that exist solely
> to host test content, it doesn’t seem like a big deal to make one of them
> update regularly and relaunch wptserve to pick up test harness changes.

Indeed such a server already exists for web-platform-tests.:
http://w3c-test.org/tools/runner/index.html

We could setup a similar server for layout tests but I'm not certain
if layout tests are written to be secure to be hosted on a publicly
visible server.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Ryosuke Niwa
On Fri, May 12, 2017 at 7:31 PM, Alexey Proskuryakov  wrote:
>
> 12 мая 2017 г., в 16:12, Sam Weinig  написал(а):
>
>
>
> On May 12, 2017, at 2:49 PM, Alexey Proskuryakov  wrote:
>
>
> 12 мая 2017 г., в 14:38, Sam Weinig  написал(а):
>
> I regret piling on here, as I think this thread has diverged from it’s
> original purpose, but…I understand this frustration. That said, perhaps this
> is something we can solve with some tooling. For instance, a
> run-test-in-safari (as a parallel to run-safari) script could be made which
> starts the server, and then loads the test with the right URL in your built
> Safari (or MiniBrowser, or whatever).
>
>
> That's still not good enough, as this means that I can't point someone else
> to an instance of the test on trac.webkit.org to reproduce an issue. There
> is of course be another place to point to when/if the test gets upstreamed,
> but even that doesn't support stable links like trac does.
>
> That's not to mention that learning the name of this proposed script is no
> easier than learning about run-webkit-httpd.
>
> - Alexey
>
>
> I’m not sure what you mean by “good enough”.  Good enough for what?  What
> are we debating here?
>
>
> I think that I explained it very clearly, but let me try again.
>
> When there is a test failure that I need to communicate to others, I say
> something "please open
> 
> in Safari to reproduce". That's very easy to do, and makes it very easy for
> others to work on the issue.
>
> If your test requires complex setup, like WPT does, then I may not have the
> time to write up complicated steps to reproduce, or the person who gets the
> bug may not have the time to follow them. Those people don't have a WebKit
> checkout, so scripts won't help. This makes the test less effective, as
> problems that it finds are less likely to be addressed.

Note that W3C's web-plaform-tests are hosted on
http://w3c-test.org/tools/runner/index.html so you can could do the
same thing.

So this is about WPT server tests written in WebKit?  But if it had to
be a WPT server test, then the alternative would have been HTTP tests
so the situation wouldn't have been different.

If you're talking about tests that use testharness.js, then we would
be using a relative path, so that should continue to work.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Alexey Proskuryakov

> 12 мая 2017 г., в 16:12, Sam Weinig  написал(а):
> 
> 
> 
>> On May 12, 2017, at 2:49 PM, Alexey Proskuryakov > > wrote:
>> 
>> 
>>> 12 мая 2017 г., в 14:38, Sam Weinig >> > написал(а):
>>> 
>>> I regret piling on here, as I think this thread has diverged from it’s 
>>> original purpose, but…I understand this frustration. That said, perhaps 
>>> this is something we can solve with some tooling. For instance, a 
>>> run-test-in-safari (as a parallel to run-safari) script could be made which 
>>> starts the server, and then loads the test with the right URL in your built 
>>> Safari (or MiniBrowser, or whatever).  
>> 
>> 
>> That's still not good enough, as this means that I can't point someone else 
>> to an instance of the test on trac.webkit.org  to 
>> reproduce an issue. There is of course be another place to point to when/if 
>> the test gets upstreamed, but even that doesn't support stable links like 
>> trac does.
>> 
>> That's not to mention that learning the name of this proposed script is no 
>> easier than learning about run-webkit-httpd.
>> 
>> - Alexey
>> 
> 
> 
> I’m not sure what you mean by “good enough”.  Good enough for what?  What are 
> we debating here?

I think that I explained it very clearly, but let me try again.

When there is a test failure that I need to communicate to others, I say 
something "please open 
>
 in Safari to reproduce". That's very easy to do, and makes it very easy for 
others to work on the issue.

If your test requires complex setup, like WPT does, then I may not have the 
time to write up complicated steps to reproduce, or the person who gets the bug 
may not have the time to follow them. Those people don't have a WebKit 
checkout, so scripts won't help. This makes the test less effective, as 
problems that it finds are less likely to be addressed.

- Alexey

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread youenn fablet
> On the topic of LayoutTest/imported tests, can someone describe the
> current process of working with LayoutTest/imported?
>
> How do we handle a broken test in our tree?
>
> • Do we modify our expectations?
>
> - If so, how do we remember to change the expectations in a later import?
>
>
If a test breaks, you are supposed to rebase the expected file.
A later import will rebase the expected file.

When importing, some subtests may not be passing.
This is ok, keep the expectation as is and do not add a FAIL line in
TestExpectations.


> • Do we modify the test? More generally, are any modifications allowed
> inside of LayoutTest/imported?
> - If so, how do we track the fact that we have modifications so that a
> later import doesn't just wipe out our changes?
> - Without tooling, is the person who modifies the test responsible for
> making an upstream pull request?
>
>
It is ok to modify the test only if you are making sure the test change is
upstreamed at landing time.
The author of the patch is currently responsible for upstreaming the
changes.
We will make things better in a not-too-far future.



> How do we handle a new imported test that WebKit fails?
>
> To avoid bots being red this test will end up getting skipped or marked as
> flakey.
>
> If it is flaky, mark it as such or mark it as skip
If it is consistently failing, it is doing some level of regression testing.
Another possibility is to not import it at all.
Please use LayoutTests/imported/w3c/resources/import-expectation.json for
that purpose.

How do we ensure this actually gets addressed and not ignored / forgotten
> about?
>
>
This is currently the responsibility of the developer doing the feature
work.
When a feature is complete, we should look at how much tests are passing
vs. skipped/failing.
This is conformance testing, we can use our own copy of WPT or W3C one.


>
> To make upstream changes someone has to make a pull-request:
>
>
> I've had some upstream patches reviewed immediately and others are still
> in review after 3+ months.
>
> In an earlier webkit-dev email Youenn mentioned:
> https://lists.webkit.org/pipermail/webkit-dev/2017-April/028932.html
>
> As done by other browser teams, a test that is reviewed in WebKit bugzilla
> and
>
> landed in WebKit can be readily merged into WPT without additional review
>
> (although additional review is always great!).
>
>
> I don't have commit access to the WPT GitHub. So ultimately I still have
> to go through another review to get my pull requests accepted.
>
>
You might get commit access.
Otherwise, some WebKit members have it and will do the cq for you.
Please ping me for that purpose. I am sure others might help too.

>
> - Joe
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Joseph Pecoraro
> On May 12, 2017, at 2:39 PM, Ryosuke Niwa  wrote:
> 
> On Fri, May 12, 2017 at 12:04 PM, Alexey Proskuryakov  > wrote:
> 
>> We don't have a concept of "first class", but I hope that when choosing
>> between looking into a simple test that was added for a known important bug,
>> and looking into an imported test whose importance is unclear, any WebKit
>> engineer will pick the former. And since no one can fix all the things, such
>> prioritization makes imported tests less effective.
> 
> This is absolutely not how I operate at all. Since almost all custom
> elements and shadow DOM API tests I wrote are written using
> testharness.js and upstreamed to web-platform-tests, they have been
> deleted from LayoutTests/fast/shadow-dom &
> LayoutTests/fast/custom-elements.
> 
> As such, if any new shadow DOM or custom elements tests under
> LayoutTests/imported/ start to fail, then we must fix them since we
> idon'thave any other test coverage.


On the topic of LayoutTest/imported tests, can someone describe the current 
process of working with LayoutTest/imported?

How do we handle a broken test in our tree?

• Do we modify our expectations?
- If so, how do we remember to change the expectations in a later 
import?

• Do we modify the test? More generally, are any modifications allowed inside 
of LayoutTest/imported?
- If so, how do we track the fact that we have modifications so that a 
later import doesn't just wipe out our changes?
- Without tooling, is the person who modifies the test responsible for 
making an upstream pull request?

How do we handle a new imported test that WebKit fails?

To avoid bots being red this test will end up getting skipped or marked as 
flakey.

How do we ensure this actually gets addressed and not ignored / forgotten about?

To make upstream changes someone has to make a pull-request:

I've had some upstream patches reviewed immediately and others are still in 
review after 3+ months.

In an earlier webkit-dev email Youenn mentioned:
https://lists.webkit.org/pipermail/webkit-dev/2017-April/028932.html

> As done by other browser teams, a test that is reviewed in WebKit bugzilla and
> landed in WebKit can be readily merged into WPT without additional review
> (although additional review is always great!).


I don't have commit access to the WPT GitHub. So ultimately I still have to go 
through another review to get my pull requests accepted.

- Joe

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Sam Weinig


> On May 12, 2017, at 2:49 PM, Alexey Proskuryakov  wrote:
> 
> 
>> 12 мая 2017 г., в 14:38, Sam Weinig > > написал(а):
>> 
>> I regret piling on here, as I think this thread has diverged from it’s 
>> original purpose, but…I understand this frustration. That said, perhaps this 
>> is something we can solve with some tooling. For instance, a 
>> run-test-in-safari (as a parallel to run-safari) script could be made which 
>> starts the server, and then loads the test with the right URL in your built 
>> Safari (or MiniBrowser, or whatever).  
> 
> 
> That's still not good enough, as this means that I can't point someone else 
> to an instance of the test on trac.webkit.org  to 
> reproduce an issue. There is of course be another place to point to when/if 
> the test gets upstreamed, but even that doesn't support stable links like 
> trac does.
> 
> That's not to mention that learning the name of this proposed script is no 
> easier than learning about run-webkit-httpd.
> 
> - Alexey
> 


I’m not sure what you mean by “good enough”.  Good enough for what?  What are 
we debating here?  I was suggesting a mechanism to ease Simon’s specific 
problem, and I do think a script which both starts the server, and loads the 
correct URL would be helpful.  I often forget exactly what path to copy when 
writing http tests.

- Sam___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread youenn fablet
I would guess Chromium and Mozilla to have the same issues there.
Incidentally, some work is being done right now to ease the
run-a-server-then-launch-a-browser thing.
We should be able to piggy-back on that effort and also handle the same
thing for LayoutTetsts/http/tests.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Alexey Proskuryakov

> 12 мая 2017 г., в 14:39, Ryosuke Niwa  написал(а):
> 
> This is absolutely not how I operate at all. Since almost all custom
> elements and shadow DOM API tests I wrote are written using
> testharness.js and upstreamed to web-platform-tests, they have been
> deleted from LayoutTests/fast/shadow-dom &
> LayoutTests/fast/custom-elements.
> 
> As such, if any new shadow DOM or custom elements tests under
> LayoutTests/imported/ start to fail, then we must fix them since we
> idon'thave any other test coverage.


I'll try to remember that. I do think that this means that shadow DOM and 
custom elements now have less effective tests, which may result in overlooking 
important issues.

- Alexey

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Maciej Stachowiak


> On May 12, 2017, at 2:39 PM, Ryosuke Niwa  wrote:
> 
> On Fri, May 12, 2017 at 12:04 PM, Alexey Proskuryakov  wrote:
>> 
>> 12 мая 2017 г., в 11:52, Ben Kelly  написал(а):
>> 
>> On Fri, May 12, 2017 at 2:26 PM, Rick Byers  wrote:
>>> 
>>> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov 
>>> wrote:
 
 Since imported WPT tests are very flaky, and are not necessarily written
 to defend against important regressions, investigating issues with them is
 relatively lower priority than investigating issues observed with WebKit
 tests. So I would recommend not mixing tests for WebKit regressions with 
 WPT
 tests - if your test eventually ends up in LayoutTests/imported, it will
 become a lot less effective.
>>> 
>>> 
>>> FWIW this is absolutely NOT how we're treating this in chromium.  If this
>>> is how things end up in practice then we will have failed massively in this
>>> effort.
>>> 
>>> We figure if we want the web to behave consistently, we really have no
>>> choice but to treat web-platform-tests as first class with all the
>>> discipline we give to our own tests.  As such we are actively moving many of
>>> our LayoutTests to web-platform-tests and depending entirely on the
>>> regression prevention they provide us from there.  Obviously there will be
>>> hiccups, but because our product quality will depend on web-platform-tests
>>> being an effective and non-flaky testsuite (and because we're starting to
>>> require most new features have web-platform-tests before they ship), I'm
>>> confident that we've got the incentives in place to lead to constant
>>> ratcheting up the engineering discipline and quality of the test suite.
>> 
>> 
>> FWIW, mozilla also treats WPT as first class tests.  We're not actively
>> moving old tests to WPT like google, but all new tests (at least in DOM) are
>> being written in WPT.  Of course, we do have exceptions for some tests that
>> require gecko-specific features (forcing GC, etc).
>> 
>> 
>> We don't have a concept of "first class", but I hope that when choosing
>> between looking into a simple test that was added for a known important bug,
>> and looking into an imported test whose importance is unclear, any WebKit
>> engineer will pick the former. And since no one can fix all the things, such
>> prioritization makes imported tests less effective.
> 
> This is absolutely not how I operate at all. Since almost all custom
> elements and shadow DOM API tests I wrote are written using
> testharness.js and upstreamed to web-platform-tests, they have been
> deleted from LayoutTests/fast/shadow-dom &
> LayoutTests/fast/custom-elements.
> 
> As such, if any new shadow DOM or custom elements tests under
> LayoutTests/imported/ start to fail, then we must fix them since we
> idon'thave any other test coverage.

Our normal approach to imported conformance test suites is to treat them as 
seriously as any other test. Even when a test was originally written along with 
a specific bug report, we don't necessarily think about that when prioritizing 
its importance, since it often fails in a way that does not make clear whether 
the exact origins bug is back.

It seems like there's two unusual things about WPT:
- We pull from upstream more often, and upstream is evolving at a good pace. So 
it's more of a moving target than something like the old W3C DOM test suite.
- At least according to Alexey, WPT tests are somewhat prone to flakiness in 
Safari.

It seems like the first issue is something we need to adapt to, to get the best 
value from WPT. But the second issue is something that has to be resolved 
within WPT (perhaps with our help). We can't have a lot of our testing depend 
on a flaky process.


This is separate from the issues about ease of running those tests. On that, I 
agree with Sam:

> On May 12, 2017, at 2:38 PM, Sam Weinig  wrote:
> 
> I regret piling on here, as I think this thread has diverged from it’s 
> original purpose, but…I understand this frustration. That said, perhaps this 
> is something we can solve with some tooling. For instance, a 
> run-test-in-safari (as a parallel to run-safari) script could be made which 
> starts the server, and then loads the test with the right URL in your built 
> Safari (or MiniBrowser, or whatever).  


I think the pain can be reduced with tooling. The right tools might need to be 
a bit more subtle. You might want to reload the test repeatedly in the same 
Safari instance, or perhaps load it into already-running Safari. So maybe 
load-test-in-safari (that ensures the server is running or launches it, then 
loads the right URL for a test, and maybe even does the right thing for http, 
web sockets or plain-file tests) would be closer to the mark. It may still be a 
little inconvenient but it seems like we can make it significantly better.

Regards,
Maciej






Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Alexey Proskuryakov

> 12 мая 2017 г., в 14:38, Sam Weinig  написал(а):
> 
> I regret piling on here, as I think this thread has diverged from it’s 
> original purpose, but…I understand this frustration. That said, perhaps this 
> is something we can solve with some tooling. For instance, a 
> run-test-in-safari (as a parallel to run-safari) script could be made which 
> starts the server, and then loads the test with the right URL in your built 
> Safari (or MiniBrowser, or whatever). 


That's still not good enough, as this means that I can't point someone else to 
an instance of the test on trac.webkit.org to reproduce an issue. There is of 
course be another place to point to when/if the test gets upstreamed, but even 
that doesn't support stable links like trac does.

That's not to mention that learning the name of this proposed script is no 
easier than learning about run-webkit-httpd.

- Alexey

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Brian Burg

> On May 12, 2017, at 2:10 PM, Simon Fraser  wrote:
> 
>> 
>> On May 12, 2017, at 12:04 PM, Alexey Proskuryakov > > wrote:
>> 
>> 
>>> 12 мая 2017 г., в 11:52, Ben Kelly >> > написал(а):
>>> 
>>> On Fri, May 12, 2017 at 2:26 PM, Rick Byers >> > wrote:
>>> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov >> > wrote:
>>> Since imported WPT tests are very flaky, and are not necessarily written to 
>>> defend against important regressions, investigating issues with them is 
>>> relatively lower priority than investigating issues observed with WebKit 
>>> tests. So I would recommend not mixing tests for WebKit regressions with 
>>> WPT tests - if your test eventually ends up in LayoutTests/imported, it 
>>> will become a lot less effective.
>>> 
>>> FWIW this is absolutely NOT how we're treating this in chromium.  If this 
>>> is how things end up in practice then we will have failed massively in this 
>>> effort.
>>> 
>>> We figure if we want the web to behave consistently, we really have no 
>>> choice but to treat web-platform-tests as first class with all the 
>>> discipline we give to our own tests.  As such we are actively moving 
>>>  many of our LayoutTests to 
>>> web-platform-tests and depending entirely on the regression prevention they 
>>> provide us from there.  Obviously there will be hiccups, but because our 
>>> product quality will depend on web-platform-tests being an effective and 
>>> non-flaky testsuite (and because we're starting to require most new 
>>> features have web-platform-tests before they ship), I'm confident that 
>>> we've got the incentives in place to lead to constant ratcheting up the 
>>> engineering discipline and quality of the test suite.
>>> 
>>> FWIW, mozilla also treats WPT as first class tests.  We're not actively 
>>> moving old tests to WPT like google, but all new tests (at least in DOM) 
>>> are being written in WPT.  Of course, we do have exceptions for some tests 
>>> that require gecko-specific features (forcing GC, etc).
>> 
>> 
>> We don't have a concept of "first class", but I hope that when choosing 
>> between looking into a simple test that was added for a known important bug, 
>> and looking into an imported test whose importance is unclear, any WebKit 
>> engineer will pick the former. And since no one can fix all the things, such 
>> prioritization makes imported tests less effective.
> 
> I just ran into a classic example of how a WPT incurred more overhead. I made 
> a code change that broke 
> LayoutTests/imported/w3c/web-platform-tests/cssom-view/elementFromPoint.html. 
> I tried loading it in Safari and it doesn't run the test code because it 
> can't find /resources/testharness.js when loaded from a local file.
> 
> So then I have to figure out how to fire up the WPT server 
> (run-webkit-httpd), then figure out which host to use (localhost or 
> 128.0.0.1?) and which port, and finally to figure out the right path to the 
> test.

It sound like the pain point is just getting the http:// served test open in 
the browser. This workflow can and should be automated.
Maybe it could be an option passed to run-safari that does all the setup?

One thing I have considered authoring at some point is a catchall driver script 
for launching tests in a browser, attaching lldb to a layout test or API test, 
etc. I do all these things manually right now and it makes investigating test 
failures a drag. It would easiest to start a new script for these 
debug/triaging workflows than to than to try and tack it all onto 
./Tools/Scripts/run-safari.

> There's no reason this test should not work when loaded from file://.

If it’s easy to inspect a test served over http, then I am ambivalent. Loading 
over file:// protocol can have strange consequences, especially when CORS, 
caching, and resource loading are involved.

> 
> Simon
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org 
> https://lists.webkit.org/mailman/listinfo/webkit-dev 
> 
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Sam Weinig


> On May 12, 2017, at 2:10 PM, Simon Fraser  wrote:
> 
> 
>> On May 12, 2017, at 12:04 PM, Alexey Proskuryakov > > wrote:
>> 
>> 
>>> 12 мая 2017 г., в 11:52, Ben Kelly >> > написал(а):
>>> 
>>> On Fri, May 12, 2017 at 2:26 PM, Rick Byers >> > wrote:
>>> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov >> > wrote:
>>> Since imported WPT tests are very flaky, and are not necessarily written to 
>>> defend against important regressions, investigating issues with them is 
>>> relatively lower priority than investigating issues observed with WebKit 
>>> tests. So I would recommend not mixing tests for WebKit regressions with 
>>> WPT tests - if your test eventually ends up in LayoutTests/imported, it 
>>> will become a lot less effective.
>>> 
>>> FWIW this is absolutely NOT how we're treating this in chromium.  If this 
>>> is how things end up in practice then we will have failed massively in this 
>>> effort.
>>> 
>>> We figure if we want the web to behave consistently, we really have no 
>>> choice but to treat web-platform-tests as first class with all the 
>>> discipline we give to our own tests.  As such we are actively moving 
>>>  many of our LayoutTests to 
>>> web-platform-tests and depending entirely on the regression prevention they 
>>> provide us from there.  Obviously there will be hiccups, but because our 
>>> product quality will depend on web-platform-tests being an effective and 
>>> non-flaky testsuite (and because we're starting to require most new 
>>> features have web-platform-tests before they ship), I'm confident that 
>>> we've got the incentives in place to lead to constant ratcheting up the 
>>> engineering discipline and quality of the test suite.
>>> 
>>> FWIW, mozilla also treats WPT as first class tests.  We're not actively 
>>> moving old tests to WPT like google, but all new tests (at least in DOM) 
>>> are being written in WPT.  Of course, we do have exceptions for some tests 
>>> that require gecko-specific features (forcing GC, etc).
>> 
>> 
>> We don't have a concept of "first class", but I hope that when choosing 
>> between looking into a simple test that was added for a known important bug, 
>> and looking into an imported test whose importance is unclear, any WebKit 
>> engineer will pick the former. And since no one can fix all the things, such 
>> prioritization makes imported tests less effective.
> 
> I just ran into a classic example of how a WPT incurred more overhead. I made 
> a code change that broke 
> LayoutTests/imported/w3c/web-platform-tests/cssom-view/elementFromPoint.html. 
> I tried loading it in Safari and it doesn't run the test code because it 
> can't find /resources/testharness.js when loaded from a local file.
> 
> So then I have to figure out how to fire up the WPT server 
> (run-webkit-httpd), then figure out which host to use (localhost or 
> 128.0.0.1?) and which port, and finally to figure out the right path to the 
> test.
> 
> There's no reason this test should not work when loaded from file://.

I regret piling on here, as I think this thread has diverged from it’s original 
purpose, but…I understand this frustration. That said, perhaps this is 
something we can solve with some tooling. For instance, a run-test-in-safari 
(as a parallel to run-safari) script could be made which starts the server, and 
then loads the test with the right URL in your built Safari (or MiniBrowser, or 
whatever).  

- Sam



___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Ryosuke Niwa
On Fri, May 12, 2017 at 2:10 PM, Simon Fraser  wrote:
>
> On May 12, 2017, at 12:04 PM, Alexey Proskuryakov  wrote:
>
>
> 12 мая 2017 г., в 11:52, Ben Kelly  написал(а):
>
> On Fri, May 12, 2017 at 2:26 PM, Rick Byers  wrote:
>>
>> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov 
>> wrote:
>>>
>>> Since imported WPT tests are very flaky, and are not necessarily written
>>> to defend against important regressions, investigating issues with them is
>>> relatively lower priority than investigating issues observed with WebKit
>>> tests. So I would recommend not mixing tests for WebKit regressions with WPT
>>> tests - if your test eventually ends up in LayoutTests/imported, it will
>>> become a lot less effective.
>>
>>
>> FWIW this is absolutely NOT how we're treating this in chromium.  If this
>> is how things end up in practice then we will have failed massively in this
>> effort.
>>
>> We figure if we want the web to behave consistently, we really have no
>> choice but to treat web-platform-tests as first class with all the
>> discipline we give to our own tests.  As such we are actively moving many of
>> our LayoutTests to web-platform-tests and depending entirely on the
>> regression prevention they provide us from there.  Obviously there will be
>> hiccups, but because our product quality will depend on web-platform-tests
>> being an effective and non-flaky testsuite (and because we're starting to
>> require most new features have web-platform-tests before they ship), I'm
>> confident that we've got the incentives in place to lead to constant
>> ratcheting up the engineering discipline and quality of the test suite.
>
>
> FWIW, mozilla also treats WPT as first class tests.  We're not actively
> moving old tests to WPT like google, but all new tests (at least in DOM) are
> being written in WPT.  Of course, we do have exceptions for some tests that
> require gecko-specific features (forcing GC, etc).
>
>
> We don't have a concept of "first class", but I hope that when choosing
> between looking into a simple test that was added for a known important bug,
> and looking into an imported test whose importance is unclear, any WebKit
> engineer will pick the former. And since no one can fix all the things, such
> prioritization makes imported tests less effective.
>
>
> I just ran into a classic example of how a WPT incurred more overhead. I
> made a code change that broke
> LayoutTests/imported/w3c/web-platform-tests/cssom-view/elementFromPoint.html.
> I tried loading it in Safari and it doesn't run the test code because it
> can't find /resources/testharness.js when loaded from a local file.
>
> So then I have to figure out how to fire up the WPT server
> (run-webkit-httpd), then figure out which host to use (localhost or
> 128.0.0.1?) and which port, and finally to figure out the right path to the
> test.
>
> There's no reason this test should not work when loaded from file://.
>

This is an issue we can solve. We can rewrite link elements to use
relative path so that you can just open in the browser.
It's something we've wanted to do but haven't gotten around to yet.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Chris Dumez
Our test importer script is perfectly able to rewrite those paths to use 
relative paths. However, Youenn, who imports and re-syncs most tests does not 
like this option I believe.
I think, part of the issue is that *some* tests do not do the right thing when 
loading over file:// (e.g. security ones) and it is not necessarily obvious 
just by looking at the test.

--
 Chris Dumez




> On May 12, 2017, at 2:10 PM, Simon Fraser  wrote:
> 
>> 
>> On May 12, 2017, at 12:04 PM, Alexey Proskuryakov > > wrote:
>> 
>> 
>>> 12 мая 2017 г., в 11:52, Ben Kelly >> > написал(а):
>>> 
>>> On Fri, May 12, 2017 at 2:26 PM, Rick Byers >> > wrote:
>>> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov >> > wrote:
>>> Since imported WPT tests are very flaky, and are not necessarily written to 
>>> defend against important regressions, investigating issues with them is 
>>> relatively lower priority than investigating issues observed with WebKit 
>>> tests. So I would recommend not mixing tests for WebKit regressions with 
>>> WPT tests - if your test eventually ends up in LayoutTests/imported, it 
>>> will become a lot less effective.
>>> 
>>> FWIW this is absolutely NOT how we're treating this in chromium.  If this 
>>> is how things end up in practice then we will have failed massively in this 
>>> effort.
>>> 
>>> We figure if we want the web to behave consistently, we really have no 
>>> choice but to treat web-platform-tests as first class with all the 
>>> discipline we give to our own tests.  As such we are actively moving 
>>>  many of our LayoutTests to 
>>> web-platform-tests and depending entirely on the regression prevention they 
>>> provide us from there.  Obviously there will be hiccups, but because our 
>>> product quality will depend on web-platform-tests being an effective and 
>>> non-flaky testsuite (and because we're starting to require most new 
>>> features have web-platform-tests before they ship), I'm confident that 
>>> we've got the incentives in place to lead to constant ratcheting up the 
>>> engineering discipline and quality of the test suite.
>>> 
>>> FWIW, mozilla also treats WPT as first class tests.  We're not actively 
>>> moving old tests to WPT like google, but all new tests (at least in DOM) 
>>> are being written in WPT.  Of course, we do have exceptions for some tests 
>>> that require gecko-specific features (forcing GC, etc).
>> 
>> 
>> We don't have a concept of "first class", but I hope that when choosing 
>> between looking into a simple test that was added for a known important bug, 
>> and looking into an imported test whose importance is unclear, any WebKit 
>> engineer will pick the former. And since no one can fix all the things, such 
>> prioritization makes imported tests less effective.
> 
> I just ran into a classic example of how a WPT incurred more overhead. I made 
> a code change that broke 
> LayoutTests/imported/w3c/web-platform-tests/cssom-view/elementFromPoint.html. 
> I tried loading it in Safari and it doesn't run the test code because it 
> can't find /resources/testharness.js when loaded from a local file.
> 
> So then I have to figure out how to fire up the WPT server 
> (run-webkit-httpd), then figure out which host to use (localhost or 
> 128.0.0.1?) and which port, and finally to figure out the right path to the 
> test.
> 
> There's no reason this test should not work when loaded from file://.
> 
> Simon
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org 
> https://lists.webkit.org/mailman/listinfo/webkit-dev 
> 
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Christian Biesinger
For what it's worth, my personal solution to this was to put a symlink
in / for the resources directory:

lrwxrwxrwx 1 root root 55 Dec 11  2015 /resources ->
/home/cbiesinger/csswg-test/resources/

Hm, I guess I should update that to web-platform-tests now!

Christian

On Fri, May 12, 2017 at 5:10 PM, Simon Fraser  wrote:
>
> On May 12, 2017, at 12:04 PM, Alexey Proskuryakov  wrote:
>
>
> 12 мая 2017 г., в 11:52, Ben Kelly  написал(а):
>
> On Fri, May 12, 2017 at 2:26 PM, Rick Byers  wrote:
>>
>> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov 
>> wrote:
>>>
>>> Since imported WPT tests are very flaky, and are not necessarily written
>>> to defend against important regressions, investigating issues with them is
>>> relatively lower priority than investigating issues observed with WebKit
>>> tests. So I would recommend not mixing tests for WebKit regressions with WPT
>>> tests - if your test eventually ends up in LayoutTests/imported, it will
>>> become a lot less effective.
>>
>>
>> FWIW this is absolutely NOT how we're treating this in chromium.  If this
>> is how things end up in practice then we will have failed massively in this
>> effort.
>>
>> We figure if we want the web to behave consistently, we really have no
>> choice but to treat web-platform-tests as first class with all the
>> discipline we give to our own tests.  As such we are actively moving many of
>> our LayoutTests to web-platform-tests and depending entirely on the
>> regression prevention they provide us from there.  Obviously there will be
>> hiccups, but because our product quality will depend on web-platform-tests
>> being an effective and non-flaky testsuite (and because we're starting to
>> require most new features have web-platform-tests before they ship), I'm
>> confident that we've got the incentives in place to lead to constant
>> ratcheting up the engineering discipline and quality of the test suite.
>
>
> FWIW, mozilla also treats WPT as first class tests.  We're not actively
> moving old tests to WPT like google, but all new tests (at least in DOM) are
> being written in WPT.  Of course, we do have exceptions for some tests that
> require gecko-specific features (forcing GC, etc).
>
>
> We don't have a concept of "first class", but I hope that when choosing
> between looking into a simple test that was added for a known important bug,
> and looking into an imported test whose importance is unclear, any WebKit
> engineer will pick the former. And since no one can fix all the things, such
> prioritization makes imported tests less effective.
>
>
> I just ran into a classic example of how a WPT incurred more overhead. I
> made a code change that broke
> LayoutTests/imported/w3c/web-platform-tests/cssom-view/elementFromPoint.html.
> I tried loading it in Safari and it doesn't run the test code because it
> can't find /resources/testharness.js when loaded from a local file.
>
> So then I have to figure out how to fire up the WPT server
> (run-webkit-httpd), then figure out which host to use (localhost or
> 128.0.0.1?) and which port, and finally to figure out the right path to the
> test.
>
> There's no reason this test should not work when loaded from file://.
>
> Simon
>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Simon Fraser

> On May 12, 2017, at 12:04 PM, Alexey Proskuryakov  wrote:
> 
> 
>> 12 мая 2017 г., в 11:52, Ben Kelly > > написал(а):
>> 
>> On Fri, May 12, 2017 at 2:26 PM, Rick Byers > > wrote:
>> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov > > wrote:
>> Since imported WPT tests are very flaky, and are not necessarily written to 
>> defend against important regressions, investigating issues with them is 
>> relatively lower priority than investigating issues observed with WebKit 
>> tests. So I would recommend not mixing tests for WebKit regressions with WPT 
>> tests - if your test eventually ends up in LayoutTests/imported, it will 
>> become a lot less effective.
>> 
>> FWIW this is absolutely NOT how we're treating this in chromium.  If this is 
>> how things end up in practice then we will have failed massively in this 
>> effort.
>> 
>> We figure if we want the web to behave consistently, we really have no 
>> choice but to treat web-platform-tests as first class with all the 
>> discipline we give to our own tests.  As such we are actively moving 
>>  many of our LayoutTests to 
>> web-platform-tests and depending entirely on the regression prevention they 
>> provide us from there.  Obviously there will be hiccups, but because our 
>> product quality will depend on web-platform-tests being an effective and 
>> non-flaky testsuite (and because we're starting to require most new features 
>> have web-platform-tests before they ship), I'm confident that we've got the 
>> incentives in place to lead to constant ratcheting up the engineering 
>> discipline and quality of the test suite.
>> 
>> FWIW, mozilla also treats WPT as first class tests.  We're not actively 
>> moving old tests to WPT like google, but all new tests (at least in DOM) are 
>> being written in WPT.  Of course, we do have exceptions for some tests that 
>> require gecko-specific features (forcing GC, etc).
> 
> 
> We don't have a concept of "first class", but I hope that when choosing 
> between looking into a simple test that was added for a known important bug, 
> and looking into an imported test whose importance is unclear, any WebKit 
> engineer will pick the former. And since no one can fix all the things, such 
> prioritization makes imported tests less effective.

I just ran into a classic example of how a WPT incurred more overhead. I made a 
code change that broke 
LayoutTests/imported/w3c/web-platform-tests/cssom-view/elementFromPoint.html. I 
tried loading it in Safari and it doesn't run the test code because it can't 
find /resources/testharness.js when loaded from a local file.

So then I have to figure out how to fire up the WPT server (run-webkit-httpd), 
then figure out which host to use (localhost or 128.0.0.1?) and which port, and 
finally to figure out the right path to the test.

There's no reason this test should not work when loaded from file://.

Simon

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread youenn fablet
Filed https://github.com/w3c/web-platform-tests/issues/5909 and
https://github.com/w3c/web-platform-tests/issues/5910.


Le ven. 12 mai 2017 à 12:08, Rick Byers  a écrit :

> On Fri, May 12, 2017 at 2:43 PM, youenn fablet  wrote:
>
>>
>> Le ven. 12 mai 2017 à 11:07, Alexey Proskuryakov  a
>> écrit :
>>
>>>
>>> 9 мая 2017 г., в 11:27, Simon Fraser 
>>> написал(а):
>>>
>>>
>>> Another consideration here is "would my test be useful for other browser
>>> vendors". I don't think the answer is a unanimous "yes", so I think we
>>> should only use WPT for tests that will think are worth sharing.
>>>
>>>
>>> Since imported WPT tests are very flaky, and are not necessarily written
>>> to defend against important regressions, investigating issues with them is
>>> relatively lower priority than investigating issues observed with WebKit
>>> tests. So I would recommend not mixing tests for WebKit regressions with
>>> WPT tests - if your test eventually ends up in LayoutTests/imported, it
>>> will become a lot less effective.
>>>
>>
>> WPT tests are flaky in WebKit because WPT stability bots do not run yet
>> Safari, and most tests are written in non-WebKit environment.
>> Often, it is due to the fact that tests are not passing and flakiness is
>> only seen with failing assertions.
>>
>> From my experience with fetch API tests, I disagree that broken WPT tests
>> are lower priority.
>> I think it will change as more WebKit contributors will author WPT tests.
>>
>> I agree that tests doing subtle WebKit-specific regression checks are not
>> good candidates for WPT upstream.
>> When the test is all about conformance with a spec, it is a very good
>> candidate.
>>
>>
>>> Using the complicated harness has a similar consequence - if you use any
>>> WPT goodies like templates or server side scripts, the cost of
>>> investigating issues on the test increases, and makes the test less
>>> valuable.
>>>
>>
>> It is true that WPT put some emphasis on easing authoring of tests.
>> I guess there is a learning curve here in WPT test debugging.
>>
>> If you have a file with 20 tests, it is harder to debug.
>> It is also increasing the chances for flakiness/timeouts.
>> Maybe we could send that feedback.
>>
>> WPT infra could also be improved: more verbose debug-only output,
>> enabling running selected subtest only...
>> testharness.js is actively maintained and is continuously improving.
>> Since we have specific requirements as you are describing, we should push
>> them.
>>
>
> Concretely, please file issues with the "infra" label
> 
>  to
> track upstream WPT infrastructure bugs and feature requests.  Google has an
> ongoing contract with Bocoup (Bob and Mike cc'd) who are improving the
> infrastructure every day.  Of course there's an opportunity to make it even
> better fast with more funding from additional organizations who would
> benefit :-)
>
>>
>> I don't know if there is any way to adopt WPT that won't make testing
>>> less effective. WPT tests may be useful in very rare cases, where we
>>> actively want to communicate certain subtle behavior details to other
>>> vendors - but even so, I imagine that other vendors may not put high
>>> priority on those, for the same reasons.
>>>
>>
>> My own experience is that WPT tests are actually very valuable, at least
>> when we are talking about interoperability/spec conformance.
>> I also see WPT as an efficient way to author tests.
>>
>>
>>>
>>> - Alexey
>>>
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>>
>>
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Rick Byers
On Fri, May 12, 2017 at 2:43 PM, youenn fablet  wrote:

>
> Le ven. 12 mai 2017 à 11:07, Alexey Proskuryakov  a écrit :
>
>>
>> 9 мая 2017 г., в 11:27, Simon Fraser  написал(а):
>>
>>
>> Another consideration here is "would my test be useful for other browser
>> vendors". I don't think the answer is a unanimous "yes", so I think we
>> should only use WPT for tests that will think are worth sharing.
>>
>>
>> Since imported WPT tests are very flaky, and are not necessarily written
>> to defend against important regressions, investigating issues with them is
>> relatively lower priority than investigating issues observed with WebKit
>> tests. So I would recommend not mixing tests for WebKit regressions with
>> WPT tests - if your test eventually ends up in LayoutTests/imported, it
>> will become a lot less effective.
>>
>
> WPT tests are flaky in WebKit because WPT stability bots do not run yet
> Safari, and most tests are written in non-WebKit environment.
> Often, it is due to the fact that tests are not passing and flakiness is
> only seen with failing assertions.
>
> From my experience with fetch API tests, I disagree that broken WPT tests
> are lower priority.
> I think it will change as more WebKit contributors will author WPT tests.
>
> I agree that tests doing subtle WebKit-specific regression checks are not
> good candidates for WPT upstream.
> When the test is all about conformance with a spec, it is a very good
> candidate.
>
>
>> Using the complicated harness has a similar consequence - if you use any
>> WPT goodies like templates or server side scripts, the cost of
>> investigating issues on the test increases, and makes the test less
>> valuable.
>>
>
> It is true that WPT put some emphasis on easing authoring of tests.
> I guess there is a learning curve here in WPT test debugging.
>
> If you have a file with 20 tests, it is harder to debug.
> It is also increasing the chances for flakiness/timeouts.
> Maybe we could send that feedback.
>
> WPT infra could also be improved: more verbose debug-only output, enabling
> running selected subtest only...
> testharness.js is actively maintained and is continuously improving.
> Since we have specific requirements as you are describing, we should push
> them.
>

Concretely, please file issues with the "infra" label

to
track upstream WPT infrastructure bugs and feature requests.  Google has an
ongoing contract with Bocoup (Bob and Mike cc'd) who are improving the
infrastructure every day.  Of course there's an opportunity to make it even
better fast with more funding from additional organizations who would
benefit :-)

>
> I don't know if there is any way to adopt WPT that won't make testing less
>> effective. WPT tests may be useful in very rare cases, where we actively
>> want to communicate certain subtle behavior details to other vendors - but
>> even so, I imagine that other vendors may not put high priority on those,
>> for the same reasons.
>>
>
> My own experience is that WPT tests are actually very valuable, at least
> when we are talking about interoperability/spec conformance.
> I also see WPT as an efficient way to author tests.
>
>
>>
>> - Alexey
>>
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Alexey Proskuryakov

> 12 мая 2017 г., в 11:52, Ben Kelly  написал(а):
> 
> On Fri, May 12, 2017 at 2:26 PM, Rick Byers  > wrote:
> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov  > wrote:
> Since imported WPT tests are very flaky, and are not necessarily written to 
> defend against important regressions, investigating issues with them is 
> relatively lower priority than investigating issues observed with WebKit 
> tests. So I would recommend not mixing tests for WebKit regressions with WPT 
> tests - if your test eventually ends up in LayoutTests/imported, it will 
> become a lot less effective.
> 
> FWIW this is absolutely NOT how we're treating this in chromium.  If this is 
> how things end up in practice then we will have failed massively in this 
> effort.
> 
> We figure if we want the web to behave consistently, we really have no choice 
> but to treat web-platform-tests as first class with all the discipline we 
> give to our own tests.  As such we are actively moving 
>  many of our LayoutTests to 
> web-platform-tests and depending entirely on the regression prevention they 
> provide us from there.  Obviously there will be hiccups, but because our 
> product quality will depend on web-platform-tests being an effective and 
> non-flaky testsuite (and because we're starting to require most new features 
> have web-platform-tests before they ship), I'm confident that we've got the 
> incentives in place to lead to constant ratcheting up the engineering 
> discipline and quality of the test suite.
> 
> FWIW, mozilla also treats WPT as first class tests.  We're not actively 
> moving old tests to WPT like google, but all new tests (at least in DOM) are 
> being written in WPT.  Of course, we do have exceptions for some tests that 
> require gecko-specific features (forcing GC, etc).


We don't have a concept of "first class", but I hope that when choosing between 
looking into a simple test that was added for a known important bug, and 
looking into an imported test whose importance is unclear, any WebKit engineer 
will pick the former. And since no one can fix all the things, such 
prioritization makes imported tests less effective.

- Alexey

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Ben Kelly
On Fri, May 12, 2017 at 2:26 PM, Rick Byers  wrote:

> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov 
> wrote:
>
>> Since imported WPT tests are very flaky, and are not necessarily written
>> to defend against important regressions, investigating issues with them is
>> relatively lower priority than investigating issues observed with WebKit
>> tests. So I would recommend not mixing tests for WebKit regressions with
>> WPT tests - if your test eventually ends up in LayoutTests/imported, it
>> will become a lot less effective.
>>
>
> FWIW this is absolutely NOT how we're treating this in chromium.  If this
> is how things end up in practice then we will have failed massively in this
> effort.
>
> We figure if we want the web to behave consistently, we really have no
> choice but to treat web-platform-tests as first class with all the
> discipline we give to our own tests.  As such we are actively moving
>  many of our LayoutTests to
> web-platform-tests and depending entirely on the regression prevention they
> provide us from there.  Obviously there will be hiccups, but because our
> product quality will depend on web-platform-tests being an effective and
> non-flaky testsuite (and because we're starting to require most new
> features have web-platform-tests before they ship), I'm confident that
> we've got the incentives in place to lead to constant ratcheting up the
> engineering discipline and quality of the test suite.
>

FWIW, mozilla also treats WPT as first class tests.  We're not actively
moving old tests to WPT like google, but all new tests (at least in DOM)
are being written in WPT.  Of course, we do have exceptions for some tests
that require gecko-specific features (forcing GC, etc).
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread youenn fablet
Le ven. 12 mai 2017 à 11:07, Alexey Proskuryakov  a écrit :

>
> 9 мая 2017 г., в 11:27, Simon Fraser  написал(а):
>
>
> Another consideration here is "would my test be useful for other browser
> vendors". I don't think the answer is a unanimous "yes", so I think we
> should only use WPT for tests that will think are worth sharing.
>
>
> Since imported WPT tests are very flaky, and are not necessarily written
> to defend against important regressions, investigating issues with them is
> relatively lower priority than investigating issues observed with WebKit
> tests. So I would recommend not mixing tests for WebKit regressions with
> WPT tests - if your test eventually ends up in LayoutTests/imported, it
> will become a lot less effective.
>

WPT tests are flaky in WebKit because WPT stability bots do not run yet
Safari, and most tests are written in non-WebKit environment.
Often, it is due to the fact that tests are not passing and flakiness is
only seen with failing assertions.

>From my experience with fetch API tests, I disagree that broken WPT tests
are lower priority.
I think it will change as more WebKit contributors will author WPT tests.

I agree that tests doing subtle WebKit-specific regression checks are not
good candidates for WPT upstream.
When the test is all about conformance with a spec, it is a very good
candidate.


> Using the complicated harness has a similar consequence - if you use any
> WPT goodies like templates or server side scripts, the cost of
> investigating issues on the test increases, and makes the test less
> valuable.
>

It is true that WPT put some emphasis on easing authoring of tests.
I guess there is a learning curve here in WPT test debugging.

If you have a file with 20 tests, it is harder to debug.
It is also increasing the chances for flakiness/timeouts.
Maybe we could send that feedback.

WPT infra could also be improved: more verbose debug-only output, enabling
running selected subtest only...
testharness.js is actively maintained and is continuously improving.
Since we have specific requirements as you are describing, we should push
them.

I don't know if there is any way to adopt WPT that won't make testing less
> effective. WPT tests may be useful in very rare cases, where we actively
> want to communicate certain subtle behavior details to other vendors - but
> even so, I imagine that other vendors may not put high priority on those,
> for the same reasons.
>

My own experience is that WPT tests are actually very valuable, at least
when we are talking about interoperability/spec conformance.
I also see WPT as an efficient way to author tests.


>
> - Alexey
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Rick Byers
On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov  wrote:

>
> 9 мая 2017 г., в 11:27, Simon Fraser  написал(а):
>
> Another consideration here is "would my test be useful for other browser
> vendors". I don't think the answer is a unanimous "yes", so I think we
> should only use WPT for tests that will think are worth sharing.
>
>
> Since imported WPT tests are very flaky, and are not necessarily written
> to defend against important regressions, investigating issues with them is
> relatively lower priority than investigating issues observed with WebKit
> tests. So I would recommend not mixing tests for WebKit regressions with
> WPT tests - if your test eventually ends up in LayoutTests/imported, it
> will become a lot less effective.
>

FWIW this is absolutely NOT how we're treating this in chromium.  If this
is how things end up in practice then we will have failed massively in this
effort.

We figure if we want the web to behave consistently, we really have no
choice but to treat web-platform-tests as first class with all the
discipline we give to our own tests.  As such we are actively moving
 many of our LayoutTests to
web-platform-tests and depending entirely on the regression prevention they
provide us from there.  Obviously there will be hiccups, but because our
product quality will depend on web-platform-tests being an effective and
non-flaky testsuite (and because we're starting to require most new
features have web-platform-tests before they ship), I'm confident that
we've got the incentives in place to lead to constant ratcheting up the
engineering discipline and quality of the test suite.

Pragmatically to get there incrementally we're applying different policies
to different WPT directories.  Eg. for some directories if a WPT import
results in new failures we probably won't care enough to prioritize
investigating them.  But for many (which map to a feature team on the hook
for quality in that area) we'll treat it with the same priority as a newly
failing LayoutTest on our bots.

Of course the WebKit project might have a higher test quality bar than the
chromium project, so the right choice could be incredibly different for you
than for us.  But should you choose to treat WPT as a first-class test
suite for your project, we're committed to working together to make it work
well in practice for all of us.  We figure that in the long-run, if we
shift investments from our own LayoutTests to web-platform-tests, even
though there will be some additional costs and risks, it'll eventually lead
to both lower testing costs and higher product quality because we're
leveraging the work of spec editors like Anne and other implementors who
are also working to improve the test suite.

Using the complicated harness has a similar consequence - if you use any
> WPT goodies like templates or server side scripts, the cost of
> investigating issues on the test increases, and makes the test less
> valuable.
>
> I don't know if there is any way to adopt WPT that won't make testing less
> effective. WPT tests may be useful in very rare cases, where we actively
> want to communicate certain subtle behavior details to other vendors - but
> even so, I imagine that other vendors may not put high priority on those,
> for the same reasons.
>
> - Alexey
>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Alexey Proskuryakov

> 9 мая 2017 г., в 11:27, Simon Fraser  написал(а):
> 
> Another consideration here is "would my test be useful for other browser 
> vendors". I don't think the answer is a unanimous "yes", so I think we should 
> only use WPT for tests that will think are worth sharing.

Since imported WPT tests are very flaky, and are not necessarily written to 
defend against important regressions, investigating issues with them is 
relatively lower priority than investigating issues observed with WebKit tests. 
So I would recommend not mixing tests for WebKit regressions with WPT tests - 
if your test eventually ends up in LayoutTests/imported, it will become a lot 
less effective.

Using the complicated harness has a similar consequence - if you use any WPT 
goodies like templates or server side scripts, the cost of investigating issues 
on the test increases, and makes the test less valuable.

I don't know if there is any way to adopt WPT that won't make testing less 
effective. WPT tests may be useful in very rare cases, where we actively want 
to communicate certain subtle behavior details to other vendors - but even so, 
I imagine that other vendors may not put high priority on those, for the same 
reasons.

- Alexey

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Fwd: Port WebKit to GNUstep

2017-05-12 Thread Konstantin Tokarev


12.05.2017, 18:14, "Daniel Ferreira (theiostream)" :
> On Fri, May 12, 2017 at 11:17 AM, Konstantin Tokarev  
> wrote:
>>  Choice of the correct MemoryPressureHandler implementation can be done
>>  in your Platform*.cmake
>>
>>  Note that you don't need to duplicate PlatformMac files and make changes,
>>  it would be wiser to include(PlatformMac.cmake) and then make modifications
>>  to variables.
>
> Yeah, checking for a Linux host and choosing which file to pick seems
> reasonable.
>
> But then it also makes sense to wrap the entire
> MemoryPressureHandlerCocoa.mm around an #if OS(DARWIN), no? We have a
> check for OS(LINUX) on the Linux counterpart.
>
> Which would then bring us to another question: if that's a Darwin
> thing, then shouldn't it be renamed to *Darwin.mm? This is even more
> evident for MemoryFootprintCocoa.mm, which uses only Mach APIs and no
> Cocoa APIs.
>
>>  I think it's not appropriate to put OS(LINUX) stuff to *Mac files. However,
>>  it might be ok to rename/move some files to reflect contents better, e.g.
>>  mac/MainThreadMac.mm could become cocoa/MainThreadCocoa.mm
>>  (but people working on Mac port may disagree)
>
> Yeah, I'd really like to hear feedback about that. At a glance that
> seems reasonable.
>
> Also, one more adjustment that might need OS-specific checks: XPCSPI.h
> relies on audit_token_t to be defined in the host OS.

You say you don't have XPC, so you don't need to deal with XPCSPI.h either

 > I got around that with:

Bugzilla is the correct place for patches [1].

However, I guess #ifndef OS(DARWIN) inside darwin-specific header won't pass
review.

[1] https://webkit.org/contributing-code/

>
> Index: wtf/spi/darwin/XPCSPI.h
> ===
> --- wtf/spi/darwin/XPCSPI.h (revision 214620)
> +++ wtf/spi/darwin/XPCSPI.h (working copy)
> +#ifndef OS(DARWIN)
> +typedef struct {
> + unsigned int val[8];
> +} audit_token_t;
> +#endif
>
>  #if OS_OBJECT_USE_OBJC
>  OS_OBJECT_DECL(xpc_object);
>  typedef xpc_object_t xpc_connection_t;
> ---
>
> -- Daniel.

-- 
Regards,
Konstantin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Fwd: Port WebKit to GNUstep

2017-05-12 Thread Daniel Ferreira (theiostream)
On Fri, May 12, 2017 at 11:17 AM, Konstantin Tokarev  wrote:
> Choice of the correct MemoryPressureHandler implementation can be done
> in your Platform*.cmake
>
> Note that you don't need to duplicate PlatformMac files and make changes,
> it would be wiser to include(PlatformMac.cmake) and then make modifications
> to variables.

Yeah, checking for a Linux host and choosing which file to pick seems
reasonable.

But then it also makes sense to wrap the entire
MemoryPressureHandlerCocoa.mm around an #if OS(DARWIN), no? We have a
check for OS(LINUX) on the Linux counterpart.

Which would then bring us to another question: if that's a Darwin
thing, then shouldn't it be renamed to *Darwin.mm? This is even more
evident for MemoryFootprintCocoa.mm, which uses only Mach APIs and no
Cocoa APIs.

> I think it's not appropriate to put OS(LINUX) stuff to *Mac files. However,
> it might be ok to rename/move some files to reflect contents better, e.g.
> mac/MainThreadMac.mm could become cocoa/MainThreadCocoa.mm
> (but people working on Mac port may disagree)

Yeah, I'd really like to hear feedback about that. At a glance that
seems reasonable.

Also, one more adjustment that might need OS-specific checks: XPCSPI.h
relies on audit_token_t to be defined in the host OS. I got around
that with:

Index: wtf/spi/darwin/XPCSPI.h
===
--- wtf/spi/darwin/XPCSPI.h (revision 214620)
+++ wtf/spi/darwin/XPCSPI.h (working copy)
+#ifndef OS(DARWIN)
+typedef struct {
+ unsigned int val[8];
+} audit_token_t;
+#endif

 #if OS_OBJECT_USE_OBJC
 OS_OBJECT_DECL(xpc_object);
 typedef xpc_object_t xpc_connection_t;
---

-- Daniel.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Fwd: Port WebKit to GNUstep

2017-05-12 Thread Konstantin Tokarev


12.05.2017, 17:03, "Daniel Ferreira (theiostream)" :
> On Thu, May 11, 2017 at 11:58 PM, Ryosuke Niwa  wrote:
>>  Are you intending to maintain this port going forward? The bar for
>>  introducing a new port in the WebKit repository is pretty high due to
>>  the maintenance cost it incurs to all other contributors. Unless
>>  you're intending to maintain your own port, and then contribute to the
>>  rest of the WebKit project unrelated to your port work, we kindly ask
>>  you to maintain your port in some downstream repository such as a
>>  GitHub fork.
>>  - R. Niwa
>
> This is a very fair request – and thank you for specifying what "the
> bar is higher" actually means. This was very unclear from the previous
> response. 1) I do not intend to cause any liability to the project; 2)
> Becoming a WebKit contributor is not a scenario I discard depending on
> the outcome of this (since I naturally hope to learn about the
> project's internal APIs and other needs over this effort), but anyway,
> I don't believe this is the place to talk about personal involvement
> matters.
>
> That being said, I'd like to address a couple of issues that go beyond
> GNUstep and that are actual issues for building the Mac port on
> non-Darwin hosts, even if we hypothetically had Cocoa seamlessly
> running on Linux (which is an issue that transcends GNUstep, with a
> lot of good work being done on things like swift-corelibs-foundation).
>
> The first issue is with MemoryPressureHandlerCocoa.mm. We compile that
> automatically if we are building a Mac port, and we don't check for
> OS(DARWIN) before relying on Mach API headers (which naturally breaks
> everything). It seems like we should take that into account before
> just compiling the file, and if we discover we are on OS(LINUX), we
> should build the Linux pressure handlers instead. If that does in fact
> work is another question, and most likely would require adjustments.

Choice of the correct MemoryPressureHandler implementation can be done
in your Platform*.cmake

Note that you don't need to duplicate PlatformMac files and make changes,
it would be wiser to include(PlatformMac.cmake) and then make modifications
to variables.

>
> In my experiment, I built the Linux pressure handler instead of the
> Mac one and had to do this tweak:
>
> Index: wtf/linux/MemoryPressureHandlerLinux.cpp
> ===
> --- wtf/linux/MemoryPressureHandlerLinux.cpp (revision 214620)
> +++ wtf/linux/MemoryPressureHandlerLinux.cpp (working copy)
> @@ -117,7 +117,7 @@
>  }, this, nullptr);
>  g_source_attach(m_source.get(), nullptr);
>  #else
> - m_threadID = createThread("WTF: MemoryPressureHandler", [this] {
> readAndNotify(); }
> + m_threadID = createThread("WTF: MemoryPressureHandler", [this] {
> readAndNotify(); });
>
>  #endif
>  }
> ---
>
> We have a missing ')' here. Apparently no one ever had a build
> configuration that preprocessor-branched here.

You need to update your patches to current trunk, this code was changed
and now it has balanced parentheses.

>
> The other issue was related to the main thread code. On
> MainThread.cpp, we check for !OS(DARWIN) to define a isMainThread()
> symbol. If we build the Mac port with a non-Darwin-OS configuration,
> we get duplicated symbols from MainThreadMac.mm. This is another issue
> with assuming Mac port means OS(DARWIN), and we should do due
> adjustments to MainThreadMac.mm to handle this case (this time I'm not
> quite sure how that would be done).
>
> Another issue with MainThreadMac.mm is its assumption that
> pthread_main_np() will be available (it is not on Linux). I managed to
> get it working by applying this hack, borrowed from a Linux port of
> libdispatch:
>
> Index: wtf/mac/MainThreadMac.mm
> ===
> --- wtf/mac/MainThreadMac.mm (revision 214620)
> +++ wtf/mac/MainThreadMac.mm (working copy)
> @@ -37,6 +37,17 @@
>  #import 
>  #import 
>
> +#if __linux__
> +#include 
> +#include 
> +#include 
> +
> +static inline int pthread_main_np()
> +{
> + return syscall(SYS_gettid) == getpid() ? 1 : 0;
> +}
> +#endif
> +
>
>  #if USE(WEB_THREAD)
>  #include 
>  #endif
> ---
>
> Again, this is another issue that needs to be solved with some more
> elaborate checks for OS(DARWIN) and handling the non-Darwin cases.

I think it's not appropriate to put OS(LINUX) stuff to *Mac files. However,
it might be ok to rename/move some files to reflect contents better, e.g.
mac/MainThreadMac.mm could become cocoa/MainThreadCocoa.mm
(but people working on Mac port may disagree)

>
> I'd really appreciate comments on these issues, and thank you for the
> quick responses.
>
> -- Daniel.
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

-- 
Regards,
Konstantin

Re: [webkit-dev] Fwd: Port WebKit to GNUstep

2017-05-12 Thread Daniel Ferreira (theiostream)
On Thu, May 11, 2017 at 11:58 PM, Ryosuke Niwa  wrote:
> Are you intending to maintain this port going forward? The bar for
> introducing a new port in the WebKit repository is pretty high due to
> the maintenance cost it incurs to all other contributors. Unless
> you're intending to maintain your own port, and then contribute to the
> rest of the WebKit project unrelated to your port work, we kindly ask
> you to maintain your port in some downstream repository such as a
> GitHub fork.
> - R. Niwa

This is a very fair request – and thank you for specifying what "the
bar is higher" actually means. This was very unclear from the previous
response. 1) I do not intend to cause any liability to the project; 2)
Becoming a WebKit contributor is not a scenario I discard depending on
the outcome of this (since I naturally hope to learn about the
project's internal APIs and other needs over this effort), but anyway,
I don't believe this is the place to talk about personal involvement
matters.

That being said, I'd like to address a couple of issues that go beyond
GNUstep and that are actual issues for building the Mac port on
non-Darwin hosts, even if we hypothetically had Cocoa seamlessly
running on Linux (which is an issue that transcends GNUstep, with a
lot of good work being done on things like swift-corelibs-foundation).

The first issue is with MemoryPressureHandlerCocoa.mm. We compile that
automatically if we are building a Mac port, and we don't check for
OS(DARWIN) before relying on Mach API headers (which naturally breaks
everything). It seems like we should take that into account before
just compiling the file, and if we discover we are on OS(LINUX), we
should build the Linux pressure handlers instead. If that does in fact
work is another question, and most likely would require adjustments.

In my experiment, I built the Linux pressure handler instead of the
Mac one and had to do this tweak:

Index: wtf/linux/MemoryPressureHandlerLinux.cpp
===
--- wtf/linux/MemoryPressureHandlerLinux.cpp (revision 214620)
+++ wtf/linux/MemoryPressureHandlerLinux.cpp (working copy)
@@ -117,7 +117,7 @@
 }, this, nullptr);
 g_source_attach(m_source.get(), nullptr);
 #else
-m_threadID = createThread("WTF: MemoryPressureHandler", [this] {
readAndNotify(); }
+m_threadID = createThread("WTF: MemoryPressureHandler", [this] {
readAndNotify(); });

 #endif
 }
---

We have a missing ')' here. Apparently no one ever had a build
configuration that preprocessor-branched here.

The other issue was related to the main thread code. On
MainThread.cpp, we check for !OS(DARWIN) to define a isMainThread()
symbol. If we build the Mac port with a non-Darwin-OS configuration,
we get duplicated symbols from MainThreadMac.mm. This is another issue
with assuming Mac port means OS(DARWIN), and we should do due
adjustments to MainThreadMac.mm to handle this case (this time I'm not
quite sure how that would be done).

Another issue with MainThreadMac.mm is its assumption that
pthread_main_np() will be available (it is not on Linux). I managed to
get it working by applying this hack, borrowed from a Linux port of
libdispatch:

Index: wtf/mac/MainThreadMac.mm
===
--- wtf/mac/MainThreadMac.mm (revision 214620)
+++ wtf/mac/MainThreadMac.mm (working copy)
@@ -37,6 +37,17 @@
 #import 
 #import 

+#if __linux__
+#include 
+#include 
+#include 
+
+static inline int pthread_main_np()
+{
+return syscall(SYS_gettid) == getpid() ? 1 : 0;
+}
+#endif
+

 #if USE(WEB_THREAD)
 #include 
 #endif
---

Again, this is another issue that needs to be solved with some more
elaborate checks for OS(DARWIN) and handling the non-Darwin cases.

I'd really appreciate comments on these issues, and thank you for the
quick responses.

-- Daniel.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Ryosuke Niwa
On Thu, May 11, 2017 at 11:32 PM, Anne van Kesteren  wrote:
> On Fri, May 12, 2017 at 6:46 AM, Simon Fraser  wrote:
>> I was under the impression that tests upstreamed from vendor repositories 
>> would land in WPT tests
>> with minimal review, based on the fact that they had been reviewed when 
>> landed in the vendor repo.
>
> I think that's under the expectation that those reviewing WPT changes
> have some familiarity with the structure of WPT.
>
>
>> What's to stop 4 vendors from upstreaming similar but non-identical tests 
>> for a given feature?
>
> Usually when a new feature is in development developers from the
> various vendors use the relevant standards forum to coordinate on
> testing and avoid duplication.

Also, if someone is fixing a bug or implementing a feature with a new
test, and some other starts passing / failing, then the person would
be compelled to either modify the existing test or de-duplicating the
test.

We all have to be diligent in this regard but it's not much worse than
what we have to do in WebKit to avoid keep adding similar tests.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-12 Thread Anne van Kesteren
On Fri, May 12, 2017 at 6:46 AM, Simon Fraser  wrote:
> I was under the impression that tests upstreamed from vendor repositories 
> would land in WPT tests
> with minimal review, based on the fact that they had been reviewed when 
> landed in the vendor repo.

I think that's under the expectation that those reviewing WPT changes
have some familiarity with the structure of WPT.


> What's to stop 4 vendors from upstreaming similar but non-identical tests for 
> a given feature?

Usually when a new feature is in development developers from the
various vendors use the relevant standards forum to coordinate on
testing and avoid duplication.


-- 
https://annevankesteren.nl/
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev