Hi all, You may be aware that some people are working on getting w3c-style reftests to work in our infrastructure (using new-run-webkit-tests).
The few existing reftests we have follow a naming convention of <testname>-expected.html or <testname>-expected-mismatch.html. This makes it easy to determine by looking at a directory which files are tests (vs. expected output or references only), and also which tests are reftests as opposed to tests that have baselines or reference output. The W3C is recommending (at least in the CSS WG) that reference files that are not themselves tests should be named as <testname>-ref.html (or the appropriate extension); test files can also live in a "reftest" subdirectory. (*) One can debate the various naming conventions; I don't particularly care what they are as long as they are something consistent, obvious, and easily automated. However, the naming conventions are currently not normative; they are a "should" rather than a "must". I think the "should" should be changed to a "must", and I'd like to ask this of the testing WG with the WebKit community's endorsement. Any one object to this or have other thoughts? -- Dirk (*) Note that it is acceptable for tests to use other tests as references, though, so not all reference files will end in -ref.html, so at least at the moment you can't tell that a file that doesn't end in -ref.html isn't both a test and a reference. See http://wiki.csswg.org/test/reftest#the-reftest-reference for more. It would also be good to allow for a "-notref.html" for expected mismatches; I'm not sure that that is explicitly standardized, but it should be. Once we establish a standard, it would also make sense to rename our existing reftests. _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
