Hey all,
A reoccurring problem when trying to maintain layout-test results is
differences in font and theme metrics for tests that dump the render
tree. Often a test does not actually test font loading/rendering or
theming, but has a piece of text or an input element somewhere in the
test which causes the metrics in the render tree to be different, and
hence the test failing.
We're seeing this in the Qt port when results are checked in as
platform-independent and causing failures, or when we have to generate
platform-dependent results for new tests when the test looks cross-platform.
I also had this problem when trying to get a Windows build up and
running with 0 failing layout-tests, so that I could verify that I
didn't break anything before landing. Even after using various tricks to
get the same font setup as the buildbots I still had failures that
looked like they were due to font metrics [1].
Lately we've been playing with the idea of using SVG fonts for the Qt
port to get the same set of expected results for qt-mac, qt-linux and
qt-win, by injecting new @font-face rules using a user-stylesheet and
preventing platform-fonts from being loaded, but this approach/hack has
proven to be quite fragile, and we will also miss out on those tests
that actually test font loading/rendering.
Another approach would be to use a fixed set of metrics for text and
themes when running the DRT, unless the test specifically asks for real
font and theme metrics. We could for example add two additional
functions to the layout-test controller:
layoutTestController.dumpRealFontMetrics()
and
layoutTestController.dumpRealThemeMetrics()
This would allow us to share render-tree results between all ports for
those tests that don't explicitly test font or theme rendering (unless I
missed more platform-differences).
Thoughts? Will this work? :)
Or, are there other ways we could achieve the same thing?
My worry is basically that the number of platform-specific-result will
not scale as the number of ports and platforms increases if we continue
to have all these "false positive" platform-specific-results that are
not really platform-specific.
Tor Arne
[1]
http://trac.webkit.org/wiki/BuildingOnWindows#RunningtheLayoutTestsonWindows
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
