Maciej Stachowiak <m...@apple.com>, 2017-05-13 14:58 -0700: > ... From what I gather, there are a lot of tests where only the paths to > the test harness end up requiring the server.
Yeah that’s the case for the vast majority of tests. Relatively few — less than 5% altogether, I’d estimate — actually rely on any special server behavior. Of the ones needing special server behavior, I think even there by far most are just cases of an accompanying foo.html.headers file in the tree along with the foo.html test, to specify that the server send particular response headers. Out of ~50,000 test files in WPT, there are less than 1000 with a .headers file. I think the next-most-common case that require special server behavior are cases where the server is doing some parsing and substitution of special parameters. In those cases the test file itself will be named in the pattern foo.sub.html, or one of its JS assets in the pattern foo.sub.js. Out of ~50,000 test files in WPT, less than 300 are *.sub.* files. So those two cases take together amount to only 3% of the total. So even with whatever else I’m missing added to that I’d estimate the number of tests that don’t rely on special server behavior is on the order of 95%. So those ~95% all only need the “/resources/testharness.js” path to the test harness to resolve and then they’ll just work. > Doing the fixup on import seems bad to me, since it seems safer and > cleaner for our WPT checkout to match WPT. But we could follow the > practice of using relative URLs for self-created tests, and perhaps not > even run them under the server when they don't need it. > > For upstream, perhaps we could advocate with WPT to use relative paths to > load the harness Given that a specific problem case Alexey mentioned was linking to tests within the WebKit Trac and having them run as expected, I wonder if at least y’all could find a way to just make https://trac.webkit.org/resources/testharness.js work — I guess by making it redirect to some place where you have a current WPT checkout. That’d at least solve things for the main specific case that’s been brought up so far being a real problem. > (and perhaps make sure that tests that absolutely require the server fail in a > way that clearly indicates this for the tests that truly do need networking or > some other facility of the WPT server). Yeah that would be good to make happen in general regardless. But I think right now that maybe can mostly be (programatically) determined by (1) checking if the name of the test file itself is in the *.sub.* pattern, or (2) checking to see if there’s an accompanying *.headers file for the test. —Mike -- Michael[tm] Smith https://people.w3.org/mike
Description: PGP signature
_______________________________________________ webkit-dev mailing list email@example.com https://lists.webkit.org/mailman/listinfo/webkit-dev