Re: [webkit-dev] sharing more test references
On Nov 13, 2012, at 4:01 PM, Dirk Pranke dpra...@chromium.org wrote: It turns out that we have a reasonably large number of tests that produce the exact same pixel results. On chromium-mac on 10.8, for example, there are 2048 tests that share a result with some other test. 50 of them, for example, draw a green square in the upper left corner of the page (e.g., for svg/custom/root-element.html). It seems to me that when we find a pattern like this, we should create a hand-coded reference test result. The fact that so many tests produce the same pixels means that each reference can be used to move a lot of tests from the pixel test to the reference test category. I don’t think we need to do that iframe thing you said, as long as there are large sets of tests with the same result, since if the payoff for each reference is big enough, it should be affordable to hand-write the reference file. -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] sharing more test references
On Tue, Nov 13, 2012 at 4:33 PM, Darin Adler da...@apple.com wrote: On Nov 13, 2012, at 4:01 PM, Dirk Pranke dpra...@chromium.org wrote: It turns out that we have a reasonably large number of tests that produce the exact same pixel results. On chromium-mac on 10.8, for example, there are 2048 tests that share a result with some other test. 50 of them, for example, draw a green square in the upper left corner of the page (e.g., for svg/custom/root-element.html). It seems to me that when we find a pattern like this, we should create a hand-coded reference test result. The fact that so many tests produce the same pixels means that each reference can be used to move a lot of tests from the pixel test to the reference test category. I don’t think we need to do that iframe thing you said, as long as there are large sets of tests with the same result, since if the payoff for each reference is big enough, it should be affordable to hand-write the reference file. I don't think I'm understanding you. Wouldn't the fact that there are a large set of tests with the same result be an argument *for* doing the iframe thing? What is the advantage to having 50 copies of a hand-coded green square in upper left corner reference test? -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] sharing more test references
On Nov 13, 2012, at 4:56 PM, Dirk Pranke dpra...@chromium.org wrote: Wouldn't the fact that there are a large set of tests with the same result be an argument *for* doing the iframe thing? The simple hand-coded green square in upper left corner should be simple, perhaps even simpler than the iframe thing. What is the advantage to having 50 copies of a hand-coded green square in upper left corner reference test? Tests standing alone and being independent, easy to move around, revise, and understand individually rather than as part of a suite. I don’t have a strong objection to your iframe technique, but I’d start simpler and do it only if it’s really needed. -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] sharing more test references
!DOCTYPE html body style=margin: 0px div style=height: 100px; width: 100px; background-color: green Does seem pretty simple. !DOCTYPE html body style=margin: 0px svgrect width=100px height=100px fill=greensvg is even shorter. :) I support getting rid of pixel tests. I suspect that some very dumb scripts could turn large chunks of these existing pixel-tests into ref-tests. I doubt that those would be the interesting ones though (where platforms have divergent results). On Tue, Nov 13, 2012 at 4:59 PM, Darin Adler da...@apple.com wrote: On Nov 13, 2012, at 4:56 PM, Dirk Pranke dpra...@chromium.org wrote: Wouldn't the fact that there are a large set of tests with the same result be an argument *for* doing the iframe thing? The simple hand-coded green square in upper left corner should be simple, perhaps even simpler than the iframe thing. What is the advantage to having 50 copies of a hand-coded green square in upper left corner reference test? Tests standing alone and being independent, easy to move around, revise, and understand individually rather than as part of a suite. I don’t have a strong objection to your iframe technique, but I’d start simpler and do it only if it’s really needed. -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] sharing more test references
On Tue, Nov 13, 2012 at 4:59 PM, Darin Adler da...@apple.com wrote: On Nov 13, 2012, at 4:56 PM, Dirk Pranke dpra...@chromium.org wrote: Wouldn't the fact that there are a large set of tests with the same result be an argument *for* doing the iframe thing? The simple hand-coded green square in upper left corner should be simple, perhaps even simpler than the iframe thing. What is the advantage to having 50 copies of a hand-coded green square in upper left corner reference test? Tests standing alone and being independent, easy to move around, revise, and understand individually rather than as part of a suite. Got it. It seems like referencing a well-known result makes things easier to understand, not harder, once you see it at least once, but I imagine it certainly depends on the complexity of the result. E.g., PASSED is better than iframe src='path-to-test-containing-the-word-passed'. Past about four lines it seems like the iframe would win. I don’t have a strong objection to your iframe technique, but I’d start simpler and do it only if it’s really needed. I will also note that there are a large number of tests where we seem to have duplicate results, e.g., dom/html/level1 and dom/xhtml/level1 where the results are basically the same between the two suites, and having the xhtml results just be iframe'd versions of the html one seems like it would make sense. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] sharing more test references
On Tue, Nov 13, 2012 at 5:06 PM, Eric Seidel e...@webkit.org wrote: !DOCTYPE html body style=margin: 0px div style=height: 100px; width: 100px; background-color: green Does seem pretty simple. !DOCTYPE html body style=margin: 0px svgrect width=100px height=100px fill=greensvg is even shorter. :) I support getting rid of pixel tests. I suspect that some very dumb scripts could turn large chunks of these existing pixel-tests into ref-tests. I doubt that those would be the interesting ones though (where platforms have divergent results). I've been spending a fair amount of time working on this, actually. I think it's harder than you might think. I'm happy to talk further about it. From what I can tell, we get most of divergence between platforms from the fact that we render text differently everywhere and we tend to render controls differently everywhere. Most of the time the differences are unrelated to what's actually being tested, unfortunately :(. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
