In https://bugzilla.mozilla.org/show_bug.cgi?id=1369941 (which just landed on autoland) I removed support for fuzzy() and fuzzy-if() annotations *without* numeric ranges in reftest manifests.
This means that what used to be written as: fuzzy(1,8) == ... fuzzy-if(Android,3,50) == ... now needs to be written (but see next paragraph!) as: fuzzy(0-1,0-8) == ... fuzzy-if(Android,0-3,0-50) == ... to keep the same behavior and indicate that the lower bound of the range is a 0 color component difference, and 0 pixels differing. However, the reason for this change is that I actually prefer that you *not* use ranges with 0s in them. In particular, when tests are failing in a minor but non-intermittent way (e.g., due to an antializing bug), it is generally better to annotate them in a stricter way, such as: fuzzy-if(Android,3-3,50-50) to indicate that the expectation is that on Android, there will be exactly 50 pixels differing, with a max color component difference of exactly 3. This means that if the difference goes away (or gets smaller), we'll get an unexpected pass message that will tell us that we need to change the fuzzy() annotation, and that change in annotation will in turn mean that we'll get a test failure if it regresses later. For more details, see the reftest documentation, the updated verson of which is currently only on autoland, i.e.: https://hg.mozilla.org/integration/autoland/file/tip/layout/tools/reftest/README.txt#l117 (At some point in the future it may make sense to add back the syntax without ranges, but with a different meaning, where 8 means the range 8-8 rather than the range 0-8.) -David -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla https://www.mozilla.org/ 𝄂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914)
signature.asc
Description: PGP signature
_______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
