> On May 12, 2017, at 2:10 PM, Simon Fraser <simon.fra...@apple.com> wrote: > >> >> On May 12, 2017, at 12:04 PM, Alexey Proskuryakov <a...@webkit.org >> <mailto:a...@webkit.org>> wrote: >> >> >>> 12 мая 2017 г., в 11:52, Ben Kelly <b...@wanderview.com >>> <mailto:b...@wanderview.com>> написал(а): >>> >>> On Fri, May 12, 2017 at 2:26 PM, Rick Byers <rby...@chromium.org >>> <mailto:rby...@chromium.org>> wrote: >>> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov <a...@webkit.org >>> <mailto:a...@webkit.org>> wrote: >>> Since imported WPT tests are very flaky, and are not necessarily written to >>> defend against important regressions, investigating issues with them is >>> relatively lower priority than investigating issues observed with WebKit >>> tests. So I would recommend not mixing tests for WebKit regressions with >>> WPT tests - if your test eventually ends up in LayoutTests/imported, it >>> will become a lot less effective. >>> >>> FWIW this is absolutely NOT how we're treating this in chromium. If this >>> is how things end up in practice then we will have failed massively in this >>> effort. >>> >>> We figure if we want the web to behave consistently, we really have no >>> choice but to treat web-platform-tests as first class with all the >>> discipline we give to our own tests. As such we are actively moving >>> <https://codereview.chromium.org/2877673004> many of our LayoutTests to >>> web-platform-tests and depending entirely on the regression prevention they >>> provide us from there. Obviously there will be hiccups, but because our >>> product quality will depend on web-platform-tests being an effective and >>> non-flaky testsuite (and because we're starting to require most new >>> features have web-platform-tests before they ship), I'm confident that >>> we've got the incentives in place to lead to constant ratcheting up the >>> engineering discipline and quality of the test suite. >>> >>> FWIW, mozilla also treats WPT as first class tests. We're not actively >>> moving old tests to WPT like google, but all new tests (at least in DOM) >>> are being written in WPT. Of course, we do have exceptions for some tests >>> that require gecko-specific features (forcing GC, etc). >> >> >> We don't have a concept of "first class", but I hope that when choosing >> between looking into a simple test that was added for a known important bug, >> and looking into an imported test whose importance is unclear, any WebKit >> engineer will pick the former. And since no one can fix all the things, such >> prioritization makes imported tests less effective. > > I just ran into a classic example of how a WPT incurred more overhead. I made > a code change that broke > LayoutTests/imported/w3c/web-platform-tests/cssom-view/elementFromPoint.html. > I tried loading it in Safari and it doesn't run the test code because it > can't find /resources/testharness.js when loaded from a local file. > > So then I have to figure out how to fire up the WPT server > (run-webkit-httpd), then figure out which host to use (localhost or > 18.104.22.168?) and which port, and finally to figure out the right path to the > test.
It sound like the pain point is just getting the http:// served test open in the browser. This workflow can and should be automated. Maybe it could be an option passed to run-safari that does all the setup? One thing I have considered authoring at some point is a catchall driver script for launching tests in a browser, attaching lldb to a layout test or API test, etc. I do all these things manually right now and it makes investigating test failures a drag. It would easiest to start a new script for these debug/triaging workflows than to than to try and tack it all onto ./Tools/Scripts/run-safari. > There's no reason this test should not work when loaded from file://. If it’s easy to inspect a test served over http, then I am ambivalent. Loading over file:// protocol can have strange consequences, especially when CORS, caching, and resource loading are involved. > > Simon > > _______________________________________________ > webkit-dev mailing list > firstname.lastname@example.org <mailto:email@example.com> > https://lists.webkit.org/mailman/listinfo/webkit-dev > <https://lists.webkit.org/mailman/listinfo/webkit-dev>
_______________________________________________ webkit-dev mailing list firstname.lastname@example.org https://lists.webkit.org/mailman/listinfo/webkit-dev