On Wed, Apr 11, 2012 at 4:00 PM, Ryosuke Niwa <[email protected]> wrote: > On Wed, Apr 11, 2012 at 3:52 PM, Jacob Goldstein <[email protected]> wrote: >> >> +1 on not introducing new pixel tests and allowing someone other than the >> test author to create the -expected file. >> >> We may also be able to streamline some of this process by implementing >> some helper scripts. Ultimately, someone will still have to review new >> files manually, but scripts should be able to speed up the process. > > > -1 on that. As I said on other threads about this topic, determining whether > a reference file adequately detect all bugs a test is intended to test is > hard, and losing the test coverage at the cost of lowering maintenance cost > is not necessary a good thing. > > Also, adding a reference file would mean that either we're adding -ref.html > / -noref.html files or modifying reftest.list. If doing the former, then we > can't use this approach in any directory where we use reftest.list at the > moment because we explicitly prohibit mixing naming convention and > reftest.list. > > Modifying reftest.list is essentially modifying the test suite, and it seems > like there is a consensus that we don't want to do it. >
This is a quibble compared to the first paragraph, but the last two paragraphs are merely implementation details (especially if we think we're generating reftest.list from <link> tags embedded in the tests). I think if we agree that it's okay to add -ref.html / -noref.html files for tests we can revisit what the best way to manage such a process is. I think the initial guideline was established when we thought that an imported test suite would come with all of the needed reference files, and in such a case I agree we should leave it as "stock" as possible. -- Dirk _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
