+pam, tc, darin in case they disagree with what I'm saying here. Also a bunch of current expectations would need to be modified. All the cases where there is currently FAIL would need to be changed to either FAIL or IMAGE or both if it's a text and image failure. You should be able to get most of the data for this by looking at the layout test dashboard. The only exception is you won't be able to distinguish tests that fail both image and text from tests that only fail image.
A short-term solution could be to leave FAIL meaning IMAGE and/or TEXT and adding IMAGE and TEXT for image-only and text-only failures. Then we can gradually excise the FAIL lines from text_expectations. I think this would be a good permanent change, but I can see arguments to the contrary. Ojan On Wed, Sep 23, 2009 at 12:25 PM, Ojan Vafai <[email protected]> wrote: > There is not. But adding it would be easy. There's been mention of > doing this for a while, but noone has made the effort to make it work. > All you'd have to do is: > -modify a few lines in TestExpectationsFile in > src/webkit/tools/layout_tests/layout_package/test_expectations.py to > add support for IMAGE in test_expectations. > -treat IMAGE and other failures separately in > src/webkit/tools/layout_tests/layout_package/compare_failures.py. > Specifically, take test_failures.FailureImageHashMismatch out of > FAILURE_TYPES and add an IMAGE_FAILURE_TYPE and use it below. > > Ojan > > On Wed, Sep 23, 2009 at 12:16 PM, Avi Drissman <[email protected]> wrote: >> I've been looking into the pixel test situation on the Mac, and it isn't bad >> at all. Of ~5300 tests that have png results, we're failing ~800, most of >> which fall into huge buckets of easily-separable fail. >> >> Is there a way to specify that we're expecting an image compare to fail but >> still want the layout to succeed? We don't want to turn off the tests >> entirely while we fix them and run the chance of breaking something that >> layout would have caught. >> >> Avi >> >> >> >> > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
