On Wed, Sep 23, 2009 at 9:52 PM, Ojan Vafai <[email protected]> wrote:
> I prefer IMAGE and TEXT since people maintaining these lists need to type > these all the time. Also, longer names make for more bloat in the file and > in the dashboard. Anyone who works with these lists even a small amount will > know that IMAGE and TEXT refer to failures. > > We should really get rid of FAIL as a valid expectation (forces us to be > more strict about what kind of failure it is), so the overlap of FAIL with > IMAGE and TEXT is just temporary. > > Really, the FAIL part is redundant, no? Why not rename TIMEOUT/CRASH to > TIMEOUT-FAIL/CRASH-FAIL? > Well, it's pretty obvious that foo/bar.html = CRASH is a failure. foo/bar.html = IMAGE doesn't say failure to me, and I'd probably forget constantly whether it's the part that's failing or succeeding. However, I'll say no more about this particular bikeshed. :) Stephen > > Ojan > > On Wed, Sep 23, 2009 at 6:32 PM, Dirk Pranke <[email protected]> wrote: > >> I think this plan sounds good, too. >> >> I'm mucking with those scripts a bit at the moment for the LTTF >> reporting, so I can make this change tomorrow, unless someone else >> would rather do it. >> >> I might actually prefer FAIL-TEXT and FAIL-IMAGE, but that's just me. >> I agree that TEXTFAIL is better than TEXT. Anyone else care to express >> a preference? >> >> -- Dirk >> >> On Wed, Sep 23, 2009 at 1:50 PM, Stephen White <[email protected]> >> wrote: >> > Could we make them TEXTFAIL and IMAGEFAIL, just to be clear? >> > Stephen >> > >> > (And then post them to failblog if they're really embarassing.. J/K ;) >> > On Wed, Sep 23, 2009 at 3:33 PM, Ojan Vafai <[email protected]> wrote: >> >> >> >> +pam, tc, darin in case they disagree with what I'm saying here. >> >> >> >> Also a bunch of current expectations would need to be modified. All >> >> the cases where there is currently FAIL would need to be changed to >> >> either FAIL or IMAGE or both if it's a text and image failure. You >> >> should be able to get most of the data for this by looking at the >> >> layout test dashboard. The only exception is you won't be able to >> >> distinguish tests that fail both image and text from tests that only >> >> fail image. >> >> >> >> A short-term solution could be to leave FAIL meaning IMAGE and/or TEXT >> >> and adding IMAGE and TEXT for image-only and text-only failures. Then >> >> we can gradually excise the FAIL lines from text_expectations. >> >> >> >> I think this would be a good permanent change, but I can see arguments >> >> to the contrary. >> >> >> >> Ojan >> >> >> >> On Wed, Sep 23, 2009 at 12:25 PM, Ojan Vafai <[email protected]> >> wrote: >> >> > There is not. But adding it would be easy. There's been mention of >> >> > doing this for a while, but noone has made the effort to make it >> work. >> >> > All you'd have to do is: >> >> > -modify a few lines in TestExpectationsFile in >> >> > src/webkit/tools/layout_tests/layout_package/test_expectations.py to >> >> > add support for IMAGE in test_expectations. >> >> > -treat IMAGE and other failures separately in >> >> > src/webkit/tools/layout_tests/layout_package/compare_failures.py. >> >> > Specifically, take test_failures.FailureImageHashMismatch out of >> >> > FAILURE_TYPES and add an IMAGE_FAILURE_TYPE and use it below. >> >> > >> >> > Ojan >> >> > >> >> > On Wed, Sep 23, 2009 at 12:16 PM, Avi Drissman <[email protected]> >> wrote: >> >> >> I've been looking into the pixel test situation on the Mac, and it >> >> >> isn't bad >> >> >> at all. Of ~5300 tests that have png results, we're failing ~800, >> most >> >> >> of >> >> >> which fall into huge buckets of easily-separable fail. >> >> >> >> >> >> Is there a way to specify that we're expecting an image compare to >> fail >> >> >> but >> >> >> still want the layout to succeed? We don't want to turn off the >> tests >> >> >> entirely while we fix them and run the chance of breaking something >> >> >> that >> >> >> layout would have caught. >> >> >> >> >> >> Avi >> >> >> >> >> >> >> >> >> >> >> >> > >> >> >> >> >> > >> > >> > >> > -- >> > All truth passes through three stages. First, it is ridiculed. Second, >> it is >> > violently opposed. Third, it is accepted as being self-evident. -- >> > Schopenhauer >> > >> > >> > >> > >> > > -- All truth passes through three stages. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as being self-evident. -- Schopenhauer --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
