On Wed, Mar 7, 2012 at 4:50 PM, Ryosuke Niwa <rn...@webkit.org> wrote:
> On Wed, Mar 7, 2012 at 4:44 PM, Darin Fisher <da...@chromium.org> wrote: > >> Hrm, if the test expectations are customized already for different ports >> of WebKit, then why not support replacing a PNG file with a HTML file that >> is intended to generate exactly the same result? How does this impair our >> ability to update the tests? >> >> (I realize that our current reftest system may not work like this. I'm >> not familiar with the details of how it works in fact, but it seems like it >> could be as simple as having an expected result that is a HTML file instead >> of a PNG file.) >> > > How do we know that we are testing what the test is intending to test > after the conversion? e.g. it's possible to create a reference file that > fails to catch certain bugs. > This sums up my worry as well. I can imagine a bug causing a CSS test and its reference to fail in the same way, masking the failure. > > It's not obvious to me how one would figure out how many reference files > are needed for a given test to make sure we're not making the test more > permissible than the author intended it to be. > > - Ryosuke > > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > >
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
