> On May 12, 2017, at 2:10 PM, Simon Fraser <simon.fra...@apple.com> wrote:
> 
>> 
>> On May 12, 2017, at 12:04 PM, Alexey Proskuryakov <a...@webkit.org 
>> <mailto:a...@webkit.org>> wrote:
>> 
>> 
>>> 12 мая 2017 г., в 11:52, Ben Kelly <b...@wanderview.com 
>>> <mailto:b...@wanderview.com>> написал(а):
>>> 
>>> On Fri, May 12, 2017 at 2:26 PM, Rick Byers <rby...@chromium.org 
>>> <mailto:rby...@chromium.org>> wrote:
>>> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov <a...@webkit.org 
>>> <mailto:a...@webkit.org>> wrote:
>>> Since imported WPT tests are very flaky, and are not necessarily written to 
>>> defend against important regressions, investigating issues with them is 
>>> relatively lower priority than investigating issues observed with WebKit 
>>> tests. So I would recommend not mixing tests for WebKit regressions with 
>>> WPT tests - if your test eventually ends up in LayoutTests/imported, it 
>>> will become a lot less effective.
>>> 
>>> FWIW this is absolutely NOT how we're treating this in chromium.  If this 
>>> is how things end up in practice then we will have failed massively in this 
>>> effort.
>>> 
>>> We figure if we want the web to behave consistently, we really have no 
>>> choice but to treat web-platform-tests as first class with all the 
>>> discipline we give to our own tests.  As such we are actively moving 
>>> <https://codereview.chromium.org/2877673004> many of our LayoutTests to 
>>> web-platform-tests and depending entirely on the regression prevention they 
>>> provide us from there.  Obviously there will be hiccups, but because our 
>>> product quality will depend on web-platform-tests being an effective and 
>>> non-flaky testsuite (and because we're starting to require most new 
>>> features have web-platform-tests before they ship), I'm confident that 
>>> we've got the incentives in place to lead to constant ratcheting up the 
>>> engineering discipline and quality of the test suite.
>>> 
>>> FWIW, mozilla also treats WPT as first class tests.  We're not actively 
>>> moving old tests to WPT like google, but all new tests (at least in DOM) 
>>> are being written in WPT.  Of course, we do have exceptions for some tests 
>>> that require gecko-specific features (forcing GC, etc).
>> 
>> 
>> We don't have a concept of "first class", but I hope that when choosing 
>> between looking into a simple test that was added for a known important bug, 
>> and looking into an imported test whose importance is unclear, any WebKit 
>> engineer will pick the former. And since no one can fix all the things, such 
>> prioritization makes imported tests less effective.
> 
> I just ran into a classic example of how a WPT incurred more overhead. I made 
> a code change that broke 
> LayoutTests/imported/w3c/web-platform-tests/cssom-view/elementFromPoint.html. 
> I tried loading it in Safari and it doesn't run the test code because it 
> can't find /resources/testharness.js when loaded from a local file.
> 
> So then I have to figure out how to fire up the WPT server 
> (run-webkit-httpd), then figure out which host to use (localhost or 
> 128.0.0.1?) and which port, and finally to figure out the right path to the 
> test.

It sound like the pain point is just getting the http:// served test open in 
the browser. This workflow can and should be automated.
Maybe it could be an option passed to run-safari that does all the setup?

One thing I have considered authoring at some point is a catchall driver script 
for launching tests in a browser, attaching lldb to a layout test or API test, 
etc. I do all these things manually right now and it makes investigating test 
failures a drag. It would easiest to start a new script for these 
debug/triaging workflows than to than to try and tack it all onto 
./Tools/Scripts/run-safari.

> There's no reason this test should not work when loaded from file://.

If it’s easy to inspect a test served over http, then I am ambivalent. Loading 
over file:// protocol can have strange consequences, especially when CORS, 
caching, and resource loading are involved.

> 
> Simon
> 
> _______________________________________________
> webkit-dev mailing list
> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
> https://lists.webkit.org/mailman/listinfo/webkit-dev 
> <https://lists.webkit.org/mailman/listinfo/webkit-dev>
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Reply via email to