On 17/11/2017 17:23, Maciej Stachowiak wrote: > I should also note that pulling in the test suite won't automatically > get the bugs fixed or prioritized, or even filed in the bug tracker > necessarily.eratwerATWeRTWAerATWertwAerer Sure. However, I believe in general having more people importing the WPT tests and trying to match the behavior of the spec and of other browsers would help, if only to be aware of issues (as Philip mentioned we could also improve notifications etc). In the particular cases I mentioned https://bugs.webkit.org/show_bug.cgi?id=162668 was reported one year ago after Khaled Hosny (invited expert for the WebFonts WG) fixed the bug in the GTK port while https://bugs.webkit.org/show_bug.cgi?id=179237 just requires to backport the changes in Chromium to match the latest WebVTT spec. These are of course only two examples, but in general more tests shared with the rest of the Web Platform community (developers and spec authors) are very useful for improving WebKit and interoperability. > I find it super convenient that EWS runs all the tests even on WIP > patches. It often catches test failures in tests I didn't think to run > myself. I think it would be great if EWS could run tests on more > platforms. Just running release regression tests on more platforms > would be a big win, if debug builds are the gating factor. I guess it > would be kind of neat to have a feature of "run the tests across all > configurations, but only some of them". But I don't think I would > prioritize it over having more kinds of tests, or finding ways to make > the full test suite run faster. Right, running all the tests is nice. But sometimes I personally feel guilty to have to execute everything on EWS for example to retrieve results for macOS/iOS (when I did not have access to a mac) or to fix compilation failures on Windows ports. So better granularity would be helpful in my opinion. In any cases, this was just one idea and people working on CI and infrastructure are in better position to know how to improve running time of WPT tests. I'm surprised that I don't hear such complaints from Mozilla and Chromium (but maybe I'm not aware of them).
-- Frédéric Wang - frederic-wang.fr
signature.asc
Description: OpenPGP digital signature
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev