On Wed, Apr 11, 2012 at 4:15 PM, Ryosuke Niwa <[email protected]> wrote: > On Wed, Apr 11, 2012 at 4:05 PM, Jacob Goldstein <[email protected]> wrote: >> >> Don't people currently fix tests that are failing and that they didn't >> author? > > > We don't fix imported tests if you meant modifying html files, etc... to fix > a broken test file. > >> If so, isn't that similar as generating a new ref file in that you first >> have to understand what is being tested? There would be the old pixel file >> to go from, so in many cases this may not be too hard. > > > Yes, but determining whether a pixel result is correct or not is easier than > determining whether a reference file is correct AND adequately addresses all > intended test purposes.
It may sometimes be easier to tell if a pixel result is correct or not, but not always: I've stared at plenty of failing pixel tests where there were no visible changes at all. Moreover, I think that requiring a reference file to address "all intended test purposes" is too high of a bar to allow it to replace a PNG; I'm not even sure if it's a practical goal at all, especially given the limited amount of experience we have with reference tests. I think there is some lower bar of goodness that we can reach in practice through experience. -- Dirk _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
