Filed https://github.com/w3c/web-platform-tests/issues/5909 and https://github.com/w3c/web-platform-tests/issues/5910.
Le ven. 12 mai 2017 à 12:08, Rick Byers <rby...@chromium.org> a écrit : > On Fri, May 12, 2017 at 2:43 PM, youenn fablet <youe...@gmail.com> wrote: > >> >> Le ven. 12 mai 2017 à 11:07, Alexey Proskuryakov <a...@webkit.org> a >> écrit : >> >>> >>> 9 мая 2017 г., в 11:27, Simon Fraser <simon.fra...@apple.com> >>> написал(а): >>> >>> >>> Another consideration here is "would my test be useful for other browser >>> vendors". I don't think the answer is a unanimous "yes", so I think we >>> should only use WPT for tests that will think are worth sharing. >>> >>> >>> Since imported WPT tests are very flaky, and are not necessarily written >>> to defend against important regressions, investigating issues with them is >>> relatively lower priority than investigating issues observed with WebKit >>> tests. So I would recommend not mixing tests for WebKit regressions with >>> WPT tests - if your test eventually ends up in LayoutTests/imported, it >>> will become a lot less effective. >>> >> >> WPT tests are flaky in WebKit because WPT stability bots do not run yet >> Safari, and most tests are written in non-WebKit environment. >> Often, it is due to the fact that tests are not passing and flakiness is >> only seen with failing assertions. >> >> From my experience with fetch API tests, I disagree that broken WPT tests >> are lower priority. >> I think it will change as more WebKit contributors will author WPT tests. >> >> I agree that tests doing subtle WebKit-specific regression checks are not >> good candidates for WPT upstream. >> When the test is all about conformance with a spec, it is a very good >> candidate. >> >> >>> Using the complicated harness has a similar consequence - if you use any >>> WPT goodies like templates or server side scripts, the cost of >>> investigating issues on the test increases, and makes the test less >>> valuable. >>> >> >> It is true that WPT put some emphasis on easing authoring of tests. >> I guess there is a learning curve here in WPT test debugging. >> >> If you have a file with 20 tests, it is harder to debug. >> It is also increasing the chances for flakiness/timeouts. >> Maybe we could send that feedback. >> >> WPT infra could also be improved: more verbose debug-only output, >> enabling running selected subtest only... >> testharness.js is actively maintained and is continuously improving. >> Since we have specific requirements as you are describing, we should push >> them. >> > > Concretely, please file issues with the "infra" label > <https://github.com/w3c/web-platform-tests/issues?q=is%3Aissue+is%3Aopen+label%3Ainfra> > to > track upstream WPT infrastructure bugs and feature requests. Google has an > ongoing contract with Bocoup (Bob and Mike cc'd) who are improving the > infrastructure every day. Of course there's an opportunity to make it even > better fast with more funding from additional organizations who would > benefit :-) > >> >> I don't know if there is any way to adopt WPT that won't make testing >>> less effective. WPT tests may be useful in very rare cases, where we >>> actively want to communicate certain subtle behavior details to other >>> vendors - but even so, I imagine that other vendors may not put high >>> priority on those, for the same reasons. >>> >> >> My own experience is that WPT tests are actually very valuable, at least >> when we are talking about interoperability/spec conformance. >> I also see WPT as an efficient way to author tests. >> >> >>> >>> - Alexey >>> >>> _______________________________________________ >>> webkit-dev mailing list >>> webkit-dev@lists.webkit.org >>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>> >> >> _______________________________________________ >> webkit-dev mailing list >> webkit-dev@lists.webkit.org >> https://lists.webkit.org/mailman/listinfo/webkit-dev >> >>
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
