On Thu, Oct 20, 2011 at 8:16 AM, Simon Fraser <[email protected]> wrote: > > On Oct 20, 2011, at 1:04 AM, Ryosuke Niwa wrote: > > On Wed, Oct 19, 2011 at 2:04 PM, Elliot Poger <[email protected]> wrote: >> >> Here are the various approaches I can think of... what's the >> Hive-Mind-Approved approach? >> >> Commit 4500 new baseline images for SnowLeopard >> >> pro: known to work, will catch any regressions that come later >> con: takes a long time to commit, chews up disk space and bandwidth for >> all developers, future minor changes may require yet another set of new >> baselines >> >> Leave all SnowLeopard tests marked as "PASS FAIL" (or maybe mark them >> "SKIP") in test_expectations >> >> pro: known to work, quick and easy, doesn't clog repo space and developer >> update bandwidth, future minor changes won't break any bots >> con: will not catch any regressions that come later on SnowLeopard >> >> Remove descriptive text from all these tests, so that text rendering is >> only evaluated in tests specifically for that purpose >> >> pro: prevents this problem for future OS versions, should allow for lots >> more baseline images to be shared across platforms >> con: a lot of work to replace all existing baseline images, must >> coordinate across community of Chromium/WebKit developers, tests will be >> more difficult to interpret without text >> >> Figure out how our test pages can be rendered with a completely >> cross-platform pixel-equivalent font >> >> pro: similar to above but tests keep their descriptive text >> con: similar to above but more technically challenging >> >> Augment our pixel-diff tools to allow for comparison masks (only pay >> attention to pixel diffs within this rectangle) >> >> pro: existing baseline images can stay in place, and perhaps be shared >> with new OS versions and platforms >> con: requires modification of pixel-diff tools, need to add comparison >> mask to each test definition > > I'd add another option to increase the tolerance level so that we ignore all > these tiny gradient/font rendering differences. I don't think the > added maintenance cost is not worth the benefit of being able to catch all > regressions. > But I'd argue that we should keep baselines for Snow Leopard with > tolerance=0 and increase the tolerance level of Leopard since Snow Leopard > is a newer platform and will probably be supported for a longer period of > time than Leopard. > > Why not use Lion as the tolerance=0 baseline? > Something else to bear in mind before lots of rebasing. I implemented mock > scrollbars with the intention that they'd be enabled by all platforms, to > reduce platform diffs in image results. Sadly, we can't use them in WK1 on > Mac (since AppKit draws the scrollbars there), so perhaps we should have a > policy that all pixel results on Mac are generated with WK2. I'm not sure > how that fits into your Chromium plans. > Simon
Chromium's DRT on Windows actually already has a custom theme that we use to eliminate diffs between windows versions. It probably wouldn't be too hard to match your scrollbars. -- Dirk _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
