Hi,
Balazs Kelemen írta:
[snip]
Ossy, another issue: currently we have some outdated *-expected.png
files hanging around the platform/qt. I say outdated because I don't
think they pass anymore based on my tests, the history, and the fact
that we don't do pixel tests. What's the approach here: do we generate
new versions of the ones that exist again? We get rid of them until we
have a bot running pixel tests? In the IRC people mentioned they are
good for reference purposes, but I think the other platforms updated
results could serve better as a reference than this.
I agree on that. It doesn't make sense to store outdated png's in the
tree. That just confuses people.
I have an objection against removing "outdated" png files from trunk.
They aren't so outdated as you think it. :) If a pixel test fails,
it doesn't mean that the png result is absolutely invalid.
When we update an *-expected.txt, we update its *-expected.png too. (almost
always)
Of course/unfortunately not everybody updates png files, but we can live with
this small noise. (And we can catch this cases easily by scripts.) Until the
test
isn't disabled, *-expected.txt guarantees that the results doesn't change
significantly.
Of course minimal pixel differences can occure which can't be catched by text
rendertree
dump. But the png is still a good reference if we have to update the test
result once.
It's much harder to compare actual pngs to Mac/GTK/Chromium pngs (which have
different
fonts, different scrollbars, input boxes, linebreaks, ...) than 1-2 month old
Qt pngs.
Just try to run ORWT --platform <mac/qt> -p <css2.1/fast> and you will see the
non-trivial
differences. So outdated png files shouldn't confuse anybody, but they are very
helpful for
gardeners (Szeged guys). Otherwise now only Chromium bots run pixel tests, but
GTK/Apple
and us regularly check in pngs. Of course it would be great if we can run pixel
tests on
the bots ... but pixel results aren't so stable (can be different on 32/64 bit,
release/
debug, Qt4.8/Qt5, ..., can depends on DRT sideefects, ...) and keep them
up-to-date/
keep the tree green would need more and more full time gardener.
But we should be optimistic. ;-) With updating ~5000 test results (and their
pngs too)
we will have more or less up-to-date pixel results. So we can try to start run
pixel
tests only on one bot to experiment a little bit. You'll see that maintaining
pixel
results aren't so easy.
br,
Ossy
_______________________________________________
webkit-qt mailing list
webkit-qt@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-qt
