Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-07 Thread Maciej Stachowiak
> On Feb 6, 2017, at 5:15 PM, Alexey Proskuryakov wrote: > > >> 6 февр. 2017 г., в 12:28, Ryosuke Niwa > > написал(а): >> >> The concern I've heard about is not how run-webkit-tests run the tests. It's >> about how a test is

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-07 Thread youenn fablet
I filed https://bugs.webkit.org/show_bug.cgi?id=167938 for the submodules issue. Le lun. 6 févr. 2017 à 17:15, Alexey Proskuryakov a écrit : > > 6 февр. 2017 г., в 12:28, Ryosuke Niwa написал(а): > > > The concern I've heard about is not how run-webkit-tests

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-07 Thread youenn fablet
Yes, let's continue this discussion on bugzilla. Le lun. 6 févr. 2017 à 16:32, Ryosuke Niwa a écrit : > On Mon, Feb 6, 2017 at 4:22 PM, youenn fablet wrote: > > It seems we agree in moving this forward. Any objection? > If so, let's start with small

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-06 Thread Philip Jägenstedt
On Mon, Feb 6, 2017 at 7:32 PM Ryosuke Niwa wrote: > On Mon, Feb 6, 2017 at 4:22 PM, youenn fablet wrote: > > It seems we agree in moving this forward. Any objection? > If so, let's start with small practical steps, something like a script to > automatically

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-06 Thread Alexey Proskuryakov
> 6 февр. 2017 г., в 12:28, Ryosuke Niwa написал(а): > > The concern I've heard about is not how run-webkit-tests run the tests. It's > about how a test is opened inside a browser, DRT, WTR while debugging. +1 I think that making web platform tests more practical for

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-06 Thread Ryosuke Niwa
On Mon, Feb 6, 2017 at 4:22 PM, youenn fablet wrote: > It seems we agree in moving this forward. Any objection? > If so, let's start with small practical steps, something like a script to > automatically generate WPT PR requests from a WebKit repo. > I think we need to first

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-06 Thread youenn fablet
I filed https://bugs.webkit.org/show_bug.cgi?id=167911. Le lun. 6 févr. 2017 à 16:22, youenn fablet a écrit : > It seems we agree in moving this forward. Any objection? > If so, let's start with small practical steps, something like a script to > automatically generate WPT PR

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-06 Thread youenn fablet
It seems we agree in moving this forward. Any objection? If so, let's start with small practical steps, something like a script to automatically generate WPT PR requests from a WebKit repo. Le lun. 6 févr. 2017 à 12:28, Ryosuke Niwa a écrit : > > On Monday, February 6, 2017,

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-06 Thread Ryosuke Niwa
On Monday, February 6, 2017, youenn fablet wrote: > The two complaints I heard against testharness.js are: > >> - They are less easy to debug as test harness.js does not print out >>> messages for each assert. There might be room for updating testharness to >>> support that

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-06 Thread youenn fablet
The two complaints I heard against testharness.js are: > - They are less easy to debug as test harness.js does not print out > messages for each assert. There might be room for updating testharness to > support that > - Async tests are less easy to write. While this is probably true, >

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-06 Thread Ryosuke Niwa
On Sun, Feb 5, 2017 at 6:54 PM, Philip Jägenstedt wrote: > FWIW, in Blink we stopped rewriting the testharness.js paths before > switching to wptserve, by instead rewriting those URLs only when running > LayoutTests: > https://cs.chromium.org/chromium/src/content/shell/ >

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-06 Thread Ryosuke Niwa
On Sun, Feb 5, 2017 at 5:04 PM, youenn fablet wrote: > I too am a big proponent of us moving more and more towards WPT. > As part of the streams and fetch API implementation, most of the related > functional tests have been made as WPT. > The benefits of having tests being

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-05 Thread Philip Jägenstedt
FWIW, in Blink we stopped rewriting the testharness.js paths before switching to wptserve, by instead rewriting those URLs only when running LayoutTests:

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-05 Thread Chris Dumez
I also think we should do 2-way sync-ing of WPT tests, similarly to other browser engines. These tests are extremely useful for interoperability. Every major browser engine is involved. Google also has a nice dashboard showing the pass rates for each WPT test in each major browser. It makes it

Re: [webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-05 Thread youenn fablet
I too am a big proponent of us moving more and more towards WPT. As part of the streams and fetch API implementation, most of the related functional tests have been made as WPT. The benefits of having tests being improved and updated largely outweighs the testharness.js initial cost. Somehow,

[webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

2017-02-05 Thread Ryosuke Niwa
Hi all, *Background* Web Platform Tests is a suite of tests written for W3C specifications =, and Blink, Edge, Gecko, and WebKit all run them as a part of their continuous build system. Historically, each working group had their own repository for