On Dec 22, 2009, at 4:27 PM, Dirk Pranke wrote:
> In the completely generic case, I hope we are not checking in incorrect
> results.
We do intentionally check in incorrect results, fairly often. For example,
we’ve checked in whole test suites and then generated expected results without
studying the tests to see which ones are successful and which are failures.
> An alternative would be to move to the more general syntax (and hopefully,
> just move to the tool) that Chromium uses.
I’m surprised that Chromium developed a separate tool. If instead the Chromium
team had enhanced the WebKit project’s shared run-webkit-tests we’d be better
off. How did we end up with two separate tools?!
> Second, there's the question of whether or not you want to track what the
> "expected incorrect" results are, separate from what the "expected correct"
> results are. That way, you can detect when a test fails *differently* than it
> has been in the past.
I do think we want to track this. It’s part of why the original system worked
they way it did when I created it back in 2005.
Also, in some cases it may be difficult to generate “correct” results if the
engine doesn’t yet have correct behavior at the time the test is being created.
-- Darin
_______________________________________________
webkit-dev mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
