Forgot to CC webkit-dev. - R. Niwa
On Tue, May 9, 2017 at 12:36 PM, Ryosuke Niwa <rn...@webkit.org> wrote: > On Tue, May 9, 2017 at 1:12 AM, Maciej Stachowiak <m...@apple.com> wrote: >> >> >> On May 8, 2017, at 11:15 PM, Ryosuke Niwa <rn...@webkit.org> wrote: >> >> On Mon, May 8, 2017 at 11:01 PM, Brady Eidson <beid...@apple.com> wrote: >> >> On May 8, 2017, at 10:44 PM, Ryosuke Niwa <rn...@webkit.org> wrote: >> On Mon, May 8, 2017 at 10:17 PM, Brady Eidson <beid...@apple.com> wrote: >> >> But now talking about testharness.js directly, I object on the grounds of "a >> file:// regression test is dirt easy to hack on and work with, whereas >> anything that requires me to have an httpd running is a PITA" >> >> I think whether we use file:// or http:// is orthogonal point to using >> testharness.js. Many of the tests Chris and I have written using >> testharness.js are checked into regular LayoutTests/ directories, and >> they work just fine. >> >> Yes, I misunderstood this in Youenn's original message. Good to know! >> >> I just object to making it the "recommended way" of writing tests. >> >> Would you equally object to making js-test.js / js-test-pre.js the >> recommended way of writing tests? >> >> Yes. >> >> If not, why? >> >> N/A >> >> What we're suggesting is to give preferential treatments to >> testharness.js over js-test.js / js-test-pre.js when you were already >> planning to write a test with the latter two scripts. >> >> "It's okay to write your test however you'd like. If you were considering >> using js-test, maybe you should consider using testharness instead." >> Is that's what's being proposed? >> >> >> On May 8, 2017, at 10:44 PM, Ryosuke Niwa <rn...@webkit.org> wrote: >> >> On Mon, May 8, 2017 at 10:17 PM, Brady Eidson <beid...@apple.com> wrote: >> >> But now talking about testharness.js directly, I object on the grounds of "a >> file:// regression test is dirt easy to hack on and work with, whereas >> anything that requires me to have an httpd running is a PITA" >> >> >> I think whether we use file:// or http:// is orthogonal point to using >> testharness.js. Many of the tests Chris and I have written using >> testharness.js are checked into regular LayoutTests/ directories, and >> they work just fine. >> >> >> Yes, I misunderstood this in Youenn's original message. Good to know! >> >> >> I just object to making it the "recommended way" of writing tests. >> >> >> Would you equally object to making js-test.js / js-test-pre.js the >> recommended way of writing tests? >> >> >> Yes. >> >> If not, why? >> >> >> N/A >> >> What we're suggesting is to give preferential treatments to >> testharness.js over js-test.js / js-test-pre.js when you were already >> planning to write a test with the latter two scripts. >> >> >> "It's okay to write your test however you'd like. If you were considering >> using js-test, maybe you should consider using testharness instead." >> >> Is that's what's being proposed? >> >> >> Besides other issues mentioned, testharness tends to result in more verbose >> tests compared to js-test, at least for simple cases. >> >> >> The thing I specifically asked Youenn to ask is, whether we should >> place a test inside LayoutTests/wpt like LayoutTests/http/tests when >> we want to write a test using testharness.js which requires some sort >> of network code. >> >> Since people have had some opinions about directory structures in the past. >> >> >> It seems like we need a few different directories, here are my opinions on >> them: >> >> (1) Imported web platform tests that don't need a server >> Currently LayoutTests/imported/w3c/web-platform-tests, which seems fine. >> (2) Imported web platform tests that do need a server >> Probably should be under LayoutTests/imported/w3c/ somewhere, or maybe > > I don't think it's a good idea to split web-platform-tests into two > separate directories like that because I'd imagine there are resource > dependencies between tests. Note that this whole process of detecting > & separating tests that can be ran locally is the work we'd have to > do, and upstream won't maintain it. Given that, we should be making > the minimal amount of differences to the upstream directory structure > as well. > > Granted, this would make it harder to know which tests require a > server and which one doesn't. Perhaps this is bad enough that we need > two directories after all but perhaps there is some other way to solve > this problem like adding suffix to test names. (Although some ref > tests in web-platform-tests use other tests as expectation but ref > tests don't tend to require servers anyway; I know this is a terrible > idea but we don't really have a control over what upstream does). > >> under http/tests/ per point (4) >> (3) Custom testharness.js tests that don't need a server >> Probably these should just go in their normal topic-specific directories >> and should not need a special directory >> (4) Custom testharness.js tests that do need a server >> Can these just be a subdirectory of http/tests/? We have websocket and >> ssl/tls tests in there too. Would be nice to not need a separate directory >> for networking tests that to use a particular test framework. > > I guess we could put them under http/tests/wpt/ then? One weird thing > is that by default, http/tests/wpt/ would map to the top-level > directory during testing: i.e. /http/tests/wpt/ would be localhost:X/ > where X is WPT server's port. > > - R. Niwa _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
