Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
>
>
>> I'm curious, what would you imagine the ref test contains?
>>
>
> If I am not mistaken, the composition operations are parallel with the
> ones of SVG and Canvas (aren't they?).
> I would have attempted comparing the 3 implementations as it seems to me
> the pixels values should be the same.
>
> That was a genuine question about that case :)
>
Thank you for feedback! :)
I presented gaussian blur case to express my opinion effectively.
As I know, Adobe want to draw CSS Filters and SVG Filters using the same
code. And Canvas does not have something like Filters.
What I want to say is that there are some cases in which snapshot
functionality is more useful.
>> >> Case 2: Fixed Position Element
>> >> [...]
>> >> function repeatedlyCalledDuringScrolling() {
>> >> ASSERT(getPixel(15, 9) == white);
>> >> ASSERT(getPixel(15, 10) == green);
>> >> ASSERT(getPixel(9, 15) == white);
>> >> ASSERT(getPixel(10, 15) == green);
>> >>
>> >> }
>> >
>> >
>> > I think this shows what I said about correctness and readability:
>> > -Asserting the correctness of the test and the result becomes close to
>> > impossible for the reader. One has to review the full code to have a
>> chance
>> > of understanding an error.
>> > -You cannot cover non trivial cases (images, text, form elements, etc).
>> > -And it is inefficient. You have to render each frame on the UIProcess,
>> move
>> > it to the WebProcess, and box it for JavaScript to process (with pixel
>> > format conversions depending on your graphics system)
>> >
>> > Of the ideas raised, I think this is one of my least favorite for
>> testing
>> > fixed positioning.
>> >
>>
>> Isn't his suggestion the equivalent of what we do today in text-only
>> tests? i.e., printing "pass" or "fail" and making you have to look at
>> the test itself to see what's being tested?
>>
>> If the correctness of the rendering depends on those 4 specific pixels
>> having those four specific values, how exactly are you going to verify
>> that by looking at it?
>>
>> Again, I think I'm just not understanding you here?
>>
>
> When looking at a test test, you follow the flow to know what it is
> supposed to do and where it breaks.
>
> How are you supposed to know, _by reading the code_, that the color at
> position 15, 9 should be white?
>
> Benjamin
>
I agree on Dirk's opinion. I supports 3 points as follows:
1. we can help to understand what's goal of test by comments
2. when we want to see the graphical result, we can use MiniBrowser like
other text based test.
3. philip's canvas tests (http://philip.html5.org/tests/canvas/suite/tests/)
already use well getImageData to test.
I agree that snapshot test can make hardly maintainable tests, but we have
review process and we can make succinct tests like philip canvas test.
And when pixel test is more suitable, we can use pixel test also.
Best regards,
Dongsung Huang
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Wed, Feb 13, 2013 at 11:33 PM, Benjamin Poulain wrote:
> On Wed, Feb 13, 2013 at 11:16 PM, Dirk Pranke wrote:
>>
>> > Those changes are not harmless. There are people monitoring tests
>> > results
>> > full time in order to keep WebKit in good shape. No other part of WebKit
>> > require continuous attention.
>>
>> I'm sorry, but either I don't understand Dongsung's suggestion, or I
>> don't understand your criticism. What does this sentence paragraph
>> mean? Are you suggesting someone needs to look at this type of test
>> full time in a way that we don't have to look at the other tests?
>
>
> My lack of sleep is likely to blame here :)
>
> All I am saying is tests have a hight cost when they start failing, we
> should be careful how we design them so we can easily triage bad tests,
> flaky tests, and real errors.
>
No argument here :).
>
>> >> Case 1: CSS Filters & Shaders
>> >> I wanted this test functionality when I commented
>> >> http://webkit.org/b/97859#c19
>> >> If I want to make gaussian blur test, I prefer using 'getPixel' test as
>> >> follows,
>> >
>> >
>> > Why wasn't a ref-test a better solution in this particular case?
>> >
>>
>> I'm curious, what would you imagine the ref test contains?
>
>
> If I am not mistaken, the composition operations are parallel with the ones
> of SVG and Canvas (aren't they?).
> I would have attempted comparing the 3 implementations as it seems to me the
> pixels values should be the same.
>
> That was a genuine question about that case :)
>
>>
>>
>> >> Case 2: Fixed Position Element
>> >> [...]
>> >> function repeatedlyCalledDuringScrolling() {
>> >> ASSERT(getPixel(15, 9) == white);
>> >> ASSERT(getPixel(15, 10) == green);
>> >> ASSERT(getPixel(9, 15) == white);
>> >> ASSERT(getPixel(10, 15) == green);
>> >>
>> >> }
>> >
>> >
>> > I think this shows what I said about correctness and readability:
>> > -Asserting the correctness of the test and the result becomes close to
>> > impossible for the reader. One has to review the full code to have a
>> > chance
>> > of understanding an error.
>> > -You cannot cover non trivial cases (images, text, form elements, etc).
>> > -And it is inefficient. You have to render each frame on the UIProcess,
>> > move
>> > it to the WebProcess, and box it for JavaScript to process (with pixel
>> > format conversions depending on your graphics system)
>> >
>> > Of the ideas raised, I think this is one of my least favorite for
>> > testing
>> > fixed positioning.
>> >
>>
>> Isn't his suggestion the equivalent of what we do today in text-only
>> tests? i.e., printing "pass" or "fail" and making you have to look at
>> the test itself to see what's being tested?
>>
>> If the correctness of the rendering depends on those 4 specific pixels
>> having those four specific values, how exactly are you going to verify
>> that by looking at it?
>>
>> Again, I think I'm just not understanding you here?
>
>
> When looking at a test test, you follow the flow to know what it is supposed
> to do and where it breaks.
>
> How are you supposed to know, _by reading the code_, that the color at
> position 15, 9 should be white?
>
is "the code" the test code, or the source code we're testing? Of
course, without any larger context, those assertions are impossible to
interpret. I figured he was just indicating what he meant by a
programmatic check, and a real test case would be more informative.
And of course it's easy to write bad tests that are neither readable
nor maintainable. But we also have lots of pixel tests that are
difficult for people not intimately familiar with the code being
tested to evaluate for correctness, too.
Lastly, I don't think he was suggesting that this would replace all
pixel tests; as you say, it is more well suited for some times of
tests than others.
As far as efficiency goes, even moving the picture back and forth
between two processes is still going to be far faster than running a
second reftest and comparing the output of the two pages.
-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
From: ext Benjamin Poulain mailto:benja...@webkit.org>> Date: Thursday, February 14, 2013 8:36 AM To: Noam Rosenthal mailto:noam.rosent...@nokia.com>> Cc: "luxte...@company100.net<mailto:luxte...@company100.net>" mailto:luxte...@company100.net>>, "webkit-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>" mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Wed, Feb 13, 2013 at 11:27 PM, mailto:noam.rosent...@nokia.com>> wrote: Why wasn't a ref-test a better solution in this particular case? Because gaussian blurs on the GPU are not accurate, and look slightly different on different GPUs, but usually "close enough". We need a way to measure "close enough" for features where all you can get is "close enough"! Ref-tests and pixel-tests are way to rigid for that, and require constant rebaselines and headaches. There is _NO_ gaussian blur in that patch. DongSung mentioned gaussian blur in the comment, which is what I was referring to. But other bugs as well, like delegated scrolling, also have that problem of results not being 100% the same every time. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Wed, Feb 13, 2013 at 11:27 PM, wrote: > Why wasn't a ref-test a better solution in this particular case? > > Because gaussian blurs on the GPU are not accurate, and look slightly > different on different GPUs, but usually "close enough". > We need a way to measure "close enough" for features where all you can get > is "close enough"! Ref-tests and pixel-tests are way to rigid for that, and > require constant rebaselines and headaches. > There is _NO_ gaussian blur in that patch. Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Wed, Feb 13, 2013 at 11:16 PM, Dirk Pranke wrote:
> > Those changes are not harmless. There are people monitoring tests results
> > full time in order to keep WebKit in good shape. No other part of WebKit
> > require continuous attention.
>
> I'm sorry, but either I don't understand Dongsung's suggestion, or I
> don't understand your criticism. What does this sentence paragraph
> mean? Are you suggesting someone needs to look at this type of test
> full time in a way that we don't have to look at the other tests?
>
My lack of sleep is likely to blame here :)
All I am saying is tests have a hight cost when they start failing, we
should be careful how we design them so we can easily triage bad tests,
flaky tests, and real errors.
>> Case 1: CSS Filters & Shaders
> >> I wanted this test functionality when I commented
> >> http://webkit.org/b/97859#c19
> >> If I want to make gaussian blur test, I prefer using 'getPixel' test as
> >> follows,
> >
> >
> > Why wasn't a ref-test a better solution in this particular case?
> >
>
> I'm curious, what would you imagine the ref test contains?
>
If I am not mistaken, the composition operations are parallel with the ones
of SVG and Canvas (aren't they?).
I would have attempted comparing the 3 implementations as it seems to me
the pixels values should be the same.
That was a genuine question about that case :)
>
> >> Case 2: Fixed Position Element
> >> [...]
> >> function repeatedlyCalledDuringScrolling() {
> >> ASSERT(getPixel(15, 9) == white);
> >> ASSERT(getPixel(15, 10) == green);
> >> ASSERT(getPixel(9, 15) == white);
> >> ASSERT(getPixel(10, 15) == green);
> >>
> >> }
> >
> >
> > I think this shows what I said about correctness and readability:
> > -Asserting the correctness of the test and the result becomes close to
> > impossible for the reader. One has to review the full code to have a
> chance
> > of understanding an error.
> > -You cannot cover non trivial cases (images, text, form elements, etc).
> > -And it is inefficient. You have to render each frame on the UIProcess,
> move
> > it to the WebProcess, and box it for JavaScript to process (with pixel
> > format conversions depending on your graphics system)
> >
> > Of the ideas raised, I think this is one of my least favorite for testing
> > fixed positioning.
> >
>
> Isn't his suggestion the equivalent of what we do today in text-only
> tests? i.e., printing "pass" or "fail" and making you have to look at
> the test itself to see what's being tested?
>
> If the correctness of the rendering depends on those 4 specific pixels
> having those four specific values, how exactly are you going to verify
> that by looking at it?
>
> Again, I think I'm just not understanding you here?
>
When looking at a test test, you follow the flow to know what it is
supposed to do and where it breaks.
How are you supposed to know, _by reading the code_, that the color at
position 15, 9 should be white?
Benjamin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
From: ext Benjamin Poulain mailto:benja...@webkit.org>>
Date: Thursday, February 14, 2013 8:07 AM
To: Dongsung Huang mailto:luxte...@company100.net>>
Cc: "webkit-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>"
mailto:webkit-dev@lists.webkit.org>>
Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction
pixel-results "on the fly"
On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang
mailto:luxte...@company100.net>> wrote:
I like this idea. I cannot find any harm if we have this functionality.
Those changes are not harmless. There are people monitoring tests results full
time in order to keep WebKit in good shape. No other part of WebKit require
continuous attention.
Case 1: CSS Filters & Shaders
I wanted this test functionality when I commented http://webkit.org/b/97859#c19
If I want to make gaussian blur test, I prefer using 'getPixel' test as follows,
Why wasn't a ref-test a better solution in this particular case?
Because gaussian blurs on the GPU are not accurate, and look slightly different
on different GPUs, but usually "close enough".
We need a way to measure "close enough" for features where all you can get is
"close enough"! Ref-tests and pixel-tests are way to rigid for that, and
require constant rebaselines and headaches.
Case 2: Fixed Position Element
[...]
function repeatedlyCalledDuringScrolling() {
ASSERT(getPixel(15, 9) == white);
ASSERT(getPixel(15, 10) == green);
ASSERT(getPixel(9, 15) == white);
ASSERT(getPixel(10, 15) == green);
}
I think this shows what I said about correctness and readability:
-Asserting the correctness of the test and the result becomes close to
impossible for the reader. One has to review the full code to have a chance of
understanding an error.
We can generate PNGs when a snapshot is requested so that the tester can
eyeball the results.
-You cannot cover non trivial cases (images, text, form elements, etc).
-And it is inefficient. You have to render each frame on the UIProcess, move it
to the WebProcess, and box it for JavaScript to process (with pixel format
conversions depending on your graphics system)
Another option is to do expose a partial comparison mechanism that runs in the
UI process.
E.g.
var snapshotHandle = testRunner.createSnapshot();
testRunner.comparePixel(snapshotHandle, 15, 20, 'white', { tolerance: 0.2 });
This way we can generate a visual representation of failures.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Wed, Feb 13, 2013 at 11:07 PM, Benjamin Poulain wrote:
> On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang
> wrote:
>>
>> I like this idea. I cannot find any harm if we have this functionality.
>
>
> Those changes are not harmless. There are people monitoring tests results
> full time in order to keep WebKit in good shape. No other part of WebKit
> require continuous attention.
>
I'm sorry, but either I don't understand Dongsung's suggestion, or I
don't understand your criticism. What does this sentence paragraph
mean? Are you suggesting someone needs to look at this type of test
full time in a way that we don't have to look at the other tests?
>>
>> Case 1: CSS Filters & Shaders
>> I wanted this test functionality when I commented
>> http://webkit.org/b/97859#c19
>> If I want to make gaussian blur test, I prefer using 'getPixel' test as
>> follows,
>
>
> Why wasn't a ref-test a better solution in this particular case?
>
I'm curious, what would you imagine the ref test contains?
>> Case 2: Fixed Position Element
>> [...]
>> function repeatedlyCalledDuringScrolling() {
>> ASSERT(getPixel(15, 9) == white);
>> ASSERT(getPixel(15, 10) == green);
>> ASSERT(getPixel(9, 15) == white);
>> ASSERT(getPixel(10, 15) == green);
>>
>> }
>
>
> I think this shows what I said about correctness and readability:
> -Asserting the correctness of the test and the result becomes close to
> impossible for the reader. One has to review the full code to have a chance
> of understanding an error.
> -You cannot cover non trivial cases (images, text, form elements, etc).
> -And it is inefficient. You have to render each frame on the UIProcess, move
> it to the WebProcess, and box it for JavaScript to process (with pixel
> format conversions depending on your graphics system)
>
> Of the ideas raised, I think this is one of my least favorite for testing
> fixed positioning.
>
Isn't his suggestion the equivalent of what we do today in text-only
tests? i.e., printing "pass" or "fail" and making you have to look at
the test itself to see what's being tested?
If the correctness of the rendering depends on those 4 specific pixels
having those four specific values, how exactly are you going to verify
that by looking at it?
Again, I think I'm just not understanding you here?
-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang wrote:
> I like this idea. I cannot find any harm if we have this functionality.
>
Those changes are not harmless. There are people monitoring tests results
full time in order to keep WebKit in good shape. No other part of WebKit
require continuous attention.
> Case 1: CSS Filters & Shaders
> I wanted this test functionality when I commented
> http://webkit.org/b/97859#c19
> If I want to make gaussian blur test, I prefer using 'getPixel' test as
> follows,
>
Why wasn't a ref-test a better solution in this particular case?
Case 2: Fixed Position Element
> [...]
> function repeatedlyCalledDuringScrolling() {
> ASSERT(getPixel(15, 9) == white);
> ASSERT(getPixel(15, 10) == green);
> ASSERT(getPixel(9, 15) == white);
> ASSERT(getPixel(10, 15) == green);
>
> }
>
I think this shows what I said about correctness and readability:
-Asserting the correctness of the test and the result becomes close to
impossible for the reader. One has to review the full code to have a chance
of understanding an error.
-You cannot cover non trivial cases (images, text, form elements, etc).
-And it is inefficient. You have to render each frame on the UIProcess,
move it to the WebProcess, and box it for JavaScript to process (with pixel
format conversions depending on your graphics system)
Of the ideas raised, I think this is one of my least favorite for testing
fixed positioning.
Benjamin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
I like this idea. I cannot find any harm if we have this functionality.
When a pixel test is more succinct, we can make a pixel test. When a
'getPixel on Javascript' test is more succinct, we can make a 'getPixel'
test.
As
http://trac.webkit.org/wiki/Writing%20Layout%20Tests%20for%20DumpRenderTree
said
'Pixel tests are a burden on every ports as any small change in the engine
could lead to your test needing a new pixel result.', new test
functionality that can be an alternative of pixel test is good!
philip canvas tests (LayoutTests/canvas/philip) has no pixel test because
of the power of getImageData API in canvas spec.
I can put two examples for the power of functionality that Noam suggested,
Case 1: CSS Filters & Shaders
I wanted this test functionality when I commented
http://webkit.org/b/97859#c19
If I want to make gaussian blur test, I prefer using 'getPixel' test as
follows,
ASSERT(getPixel(10, 10) = rgba(255, 0, 0, 255));
ASSERT(getPixel(10, 15) > rgba(255, 0, 0, 98) && getPixel(10, 15) <
rgba(255, 0, 0, 102));
If we don't have 'getPixel', we need to capture all port's image results
because all port's painting engine would make a bit different result. As
you know, it is painful.
Case 2: Fixed Position Element
I guess it is one of reasons why Noam suggested.
Currently, Qt and EFL AC (a.k.a Coordinated Graphics) has a problem related
to fixed position element.
During panning animation, Fixed position element is slightly flickering.
If we have 'getPixel', we can make a succinct test case.
PRECONDITION : a green fixed position div is on the white body. The rect of
the div is rect(10, 10, 30, 30).
function repeatedlyCalledDuringScrolling() {
ASSERT(getPixel(15, 9) == white);
ASSERT(getPixel(15, 10) == green);
ASSERT(getPixel(9, 15) == white);
ASSERT(getPixel(10, 15) == green);
}
WDYT?
Best regards,
Dongsung Huang
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Sun, Feb 10, 2013 at 1:30 AM, Ryosuke Niwa wrote: > On Sat, Feb 9, 2013 at 5:35 PM, Dirk Pranke wrote: >> >> Perhaps -- and this is a tangent to this thread -- we could also >> investigate ways of writing tests that did render pngs but told NRWT >> to ignore them (e.g., dumpAsTextAndIgnoredImage() or something), or >> conveyed which parts of the image were important ... > > > What's the point of that? > Oops. I think I deleted the part of my reply explaining that :). What I meant to say is that the downside of text-only/programmatic tests is that you don't actually get a rendered page to look at. The downside of pixel tests is that you can get a rendered page that shows you what's wrong, but the rendered page often contains a bunch of "extraneous" things like explanatory text that can cause unnecessary diffs. It might be nice if we had better ways to filter out the parts that caused the unnecessary diffs. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Sat, Feb 9, 2013 at 5:35 PM, Dirk Pranke wrote: > Perhaps -- and this is a tangent to this thread -- we could also > investigate ways of writing tests that did render pngs but told NRWT > to ignore them (e.g., dumpAsTextAndIgnoredImage() or something), or > conveyed which parts of the image were important ... > What's the point of that? - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Sat, Feb 9, 2013 at 1:57 AM, Benjamin Poulain wrote: > On Sat, Feb 9, 2013 at 1:24 AM, wrote: >> >> I did not say they cannot be tested with the two methods suggested :) It's >> more about a preference to move some of those decisions from the test >> infrastructure to the test itself, but potentially those bugs could be >> tested in either way. >> Examples for bugs I've encountered/reviewed and can use better in-motion >> testing (note that those are specific to Qt/EFL, but I'm sure there are tons >> of bugs like this that come up for Apple/Google as well) >> http://trac.webkit.org/changeset/140825 >> http://trac.webkit.org/changeset/142112 >> http://trac.webkit.org/changeset/134953 >> https://bugs.webkit.org/show_bug.cgi?id=109179 >> >> Controlling the clock and programmatically sampling the end result would >> definitely make those more testable, but of course any progress in this area >> would be beneficial and my preference to a canvas-based API is more of an >> opinion. > > > To explain my concerns: > Sometime, I look at a failing test, and think: "what the f**k is this > supposed to test?". Then I have to dig a ton of logs, and finally read the > change to understand a the single JS statement in the whole script that make > the test useful. > > This is the situation I am afraid with a solution where pixels would be > evaluated from JavaScript. You can test, but you lack visibility why > something is correct or incorrect. Text tests, ref-tests and pixel tests > have a great readability, you can quickly evaluate their correctness. This > is important in my opinion, I don't think we want more opaque output like > the RenderTree dump. > > I am not against your plan if the readability of the tests can be good. I'd > be happy to review patches toward testing dynamic changes in webpages. > A counter-argument, of course, that there are a lot of pixel tests that would be a lot better as text-only tests that were verifying certain aspects of how the page rendered programmatically. Perhaps -- and this is a tangent to this thread -- we could also investigate ways of writing tests that did render pngs but told NRWT to ignore them (e.g., dumpAsTextAndIgnoredImage() or something), or conveyed which parts of the image were important ... -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
Oops, forgot to reply to the list. From: Noam Rosenthal mailto:noam.rosent...@nokia.com>> From: ext Simon Fraser mailto:simon.fra...@apple.com>> The existing pauseAnimation DRT API can stop an animation mid-flight, to allow for reliable snapshotting. Why isn't this enough? I would say that the main things that are missing are: 1. Sample and compare several images in the same test 2. Sample images "heuristically", e.g. sample that a green box is "close enough" to a particular location – A full ImageDiff is very rigid in that way. 3. Sample images mid-flight without pausing. Pausing can mess with animation timing and thus skew the results. I don't have a full list of what's missing – but I know for sure that when trying to hunt mid-flight bugs in the past I was not able to create sufficient tests with the current DRT tools. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
The existing pauseAnimation DRT API can stop an animation mid-flight, to allow for reliable snapshotting. Why isn't this enough? Simon On Feb 9, 2013, at 8:44 AM, noam.rosent...@nokia.com wrote: > Since people seem to agree that this is a good problem to solve, I've created > a bug for it: https://bugs.webkit.org/show_bug.cgi?id=109356 > I can't promise to fix it myself right this moment as I'm spread a bit too > thin, but if someone wants to help or pick it up before I get to it please > feel free :) > Noam > > From: ext Benjamin Poulain > Date: Saturday, February 9, 2013 10:57 AM > To: Noam Rosenthal > Cc: "rn...@webkit.org" , "webkit-dev@lists.webkit.org" > > Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction > pixel-results "on the fly" > >> On Sat, Feb 9, 2013 at 1:24 AM, wrote: >>> >>> I did not say they cannot be tested with the two methods suggested :) It's >>> more about a preference to move some of those decisions from the test >>> infrastructure to the test itself, but potentially those bugs could be >>> tested in either way. >>> Examples for bugs I've encountered/reviewed and can use better in-motion >>> testing (note that those are specific to Qt/EFL, but I'm sure there are >>> tons of bugs like this that come up for Apple/Google as well) >>> http://trac.webkit.org/changeset/140825 >>> http://trac.webkit.org/changeset/142112 >>> http://trac.webkit.org/changeset/134953 >>> https://bugs.webkit.org/show_bug.cgi?id=109179 >>> >>> Controlling the clock and programmatically sampling the end result would >>> definitely make those more testable, but of course any progress in this >>> area would be beneficial and my preference to a canvas-based API is more of >>> an opinion. >> >> To explain my concerns: >> Sometime, I look at a failing test, and think: "what the f**k is this >> supposed to test?". Then I have to dig a ton of logs, and finally read the >> change to understand a the single JS statement in the whole script that make >> the test useful. >> >> This is the situation I am afraid with a solution where pixels would be >> evaluated from JavaScript. You can test, but you lack visibility why >> something is correct or incorrect. Text tests, ref-tests and pixel tests >> have a great readability, you can quickly evaluate their correctness. This >> is important in my opinion, I don't think we want more opaque output like >> the RenderTree dump. >> >> I am not against your plan if the readability of the tests can be good. I'd >> be happy to review patches toward testing dynamic changes in webpages. >> >> Benjamin > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
Since people seem to agree that this is a good problem to solve, I've created a bug for it: https://bugs.webkit.org/show_bug.cgi?id=109356 I can't promise to fix it myself right this moment as I'm spread a bit too thin, but if someone wants to help or pick it up before I get to it please feel free :) Noam From: ext Benjamin Poulain mailto:benja...@webkit.org>> Date: Saturday, February 9, 2013 10:57 AM To: Noam Rosenthal mailto:noam.rosent...@nokia.com>> Cc: "rn...@webkit.org<mailto:rn...@webkit.org>" mailto:rn...@webkit.org>>, "webkit-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>" mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Sat, Feb 9, 2013 at 1:24 AM, mailto:noam.rosent...@nokia.com>> wrote: I did not say they cannot be tested with the two methods suggested :) It's more about a preference to move some of those decisions from the test infrastructure to the test itself, but potentially those bugs could be tested in either way. Examples for bugs I've encountered/reviewed and can use better in-motion testing (note that those are specific to Qt/EFL, but I'm sure there are tons of bugs like this that come up for Apple/Google as well) http://trac.webkit.org/changeset/140825 http://trac.webkit.org/changeset/142112 http://trac.webkit.org/changeset/134953 https://bugs.webkit.org/show_bug.cgi?id=109179 Controlling the clock and programmatically sampling the end result would definitely make those more testable, but of course any progress in this area would be beneficial and my preference to a canvas-based API is more of an opinion. To explain my concerns: Sometime, I look at a failing test, and think: "what the f**k is this supposed to test?". Then I have to dig a ton of logs, and finally read the change to understand a the single JS statement in the whole script that make the test useful. This is the situation I am afraid with a solution where pixels would be evaluated from JavaScript. You can test, but you lack visibility why something is correct or incorrect. Text tests, ref-tests and pixel tests have a great readability, you can quickly evaluate their correctness. This is important in my opinion, I don't think we want more opaque output like the RenderTree dump. I am not against your plan if the readability of the tests can be good. I'd be happy to review patches toward testing dynamic changes in webpages. Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
From: ext Benjamin Poulain mailto:benja...@webkit.org>> This is the situation I am afraid with a solution where pixels would be evaluated from JavaScript. You can test, but you lack visibility why something is correct or incorrect. Text tests, ref-tests and pixel tests have a great readability, you can quickly evaluate their correctness. This is important in my opinion, I don't think we want more opaque output like the RenderTree dump. I am not against your plan if the readability of the tests can be good. I'd be happy to review patches toward testing dynamic changes in webpages. We're on the same page. I'm afraid of the same thing… Let's talk specifics when a patch is available to review. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Sat, Feb 9, 2013 at 1:24 AM, wrote: >I did not say they cannot be tested with the two methods suggested :) > It's more about a preference to move some of those decisions from the test > infrastructure to the test itself, but potentially those bugs could be > tested in either way. > Examples for bugs I've encountered/reviewed and can use better in-motion > testing (note that those are specific to Qt/EFL, but I'm sure there are > tons of bugs like this that come up for Apple/Google as well) > http://trac.webkit.org/changeset/140825 > http://trac.webkit.org/changeset/142112 > http://trac.webkit.org/changeset/134953 > https://bugs.webkit.org/show_bug.cgi?id=109179 > > Controlling the clock and programmatically sampling the end result would > definitely make those more testable, but of course any progress in this > area would be beneficial and my preference to a canvas-based API is more of > an opinion. > To explain my concerns: Sometime, I look at a failing test, and think: "what the f**k is this supposed to test?". Then I have to dig a ton of logs, and finally read the change to understand a the single JS statement in the whole script that make the test useful. This is the situation I am afraid with a solution where pixels would be evaluated from JavaScript. You can test, but you lack visibility why something is correct or incorrect. Text tests, ref-tests and pixel tests have a great readability, you can quickly evaluate their correctness. This is important in my opinion, I don't think we want more opaque output like the RenderTree dump. I am not against your plan if the readability of the tests can be good. I'd be happy to review patches toward testing dynamic changes in webpages. Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
From: ext Benjamin Poulain mailto:benja...@webkit.org>> Date: Saturday, February 9, 2013 12:52 AM To: Noam Rosenthal mailto:noam.rosent...@nokia.com>> Cc: "rn...@webkit.org<mailto:rn...@webkit.org>" mailto:rn...@webkit.org>>, "webkit-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>" mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Fri, Feb 8, 2013 at 3:16 PM, mailto:noam.rosent...@nokia.com>> wrote: The problem with dynamic features of the web like animations/interactions is that they're non-deterministic, or at least a lot less deterministic than static features of the web like layouts. Ref tests, pixel tests etc. are tools built for deterministic testing: load a file, take a snapshot, compare against a result. Testing an animation (or a filter) needs to feel a lot more dynamic and expressive: Animate green boxes, make sure that they're within a particular range at particular points in time". The tests also have to be deterministic and comprehensive. I am afraid of loosing both with the Render-to-Canvas approach. Can you give concrete examples of the kind of bugs you are hunting, and why testing cannot use the two methods suggested? I did not say they cannot be tested with the two methods suggested :) It's more about a preference to move some of those decisions from the test infrastructure to the test itself, but potentially those bugs could be tested in either way. Examples for bugs I've encountered/reviewed and can use better in-motion testing (note that those are specific to Qt/EFL, but I'm sure there are tons of bugs like this that come up for Apple/Google as well) http://trac.webkit.org/changeset/140825 http://trac.webkit.org/changeset/142112 http://trac.webkit.org/changeset/134953 https://bugs.webkit.org/show_bug.cgi?id=109179 Controlling the clock and programmatically sampling the end result would definitely make those more testable, but of course any progress in this area would be beneficial and my preference to a canvas-based API is more of an opinion. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
I'm not sure if this is generally known: we use [svg document].setCurrentTime(time) in the svg/animations tests for both script and pixel tests. The SVG animation test harness leaves a lot to be desired, but it may be a good place to start for a more general DRT approach: http://trac.webkit.org/browser/trunk/LayoutTests/svg/animations/script-tests/animate-css-xml-attributeType.js#L62 Philip On Fri, Feb 8, 2013 at 4:26 PM, Gregg Tavares wrote: > > > > On Fri, Feb 8, 2013 at 4:16 PM, Benjamin Poulain wrote: > >> On Fri, Feb 8, 2013 at 4:12 PM, Gregg Tavares wrote: >> >>> Can you expose a time or setTime function in DRT and some option that >>> says "let JS control the clock"? >>> >> >> Unfortunately, that only works in the simple cases. >> We could cheat the WebCore clock, but that would ultimately be wrong for >> animations running in threads or driven outside WebCore. >> > > The point was you need to be able to control all the clocks if you want > repeatable results. If you're saying some clocks can't be controlled that > sounds like something needs to be refactored so it's testable. > > >> >> Benjamin >> > > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev > > ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Fri, Feb 8, 2013 at 4:16 PM, Benjamin Poulain wrote: > On Fri, Feb 8, 2013 at 4:12 PM, Gregg Tavares wrote: > >> Can you expose a time or setTime function in DRT and some option that >> says "let JS control the clock"? >> > > Unfortunately, that only works in the simple cases. > We could cheat the WebCore clock, but that would ultimately be wrong for > animations running in threads or driven outside WebCore. > The point was you need to be able to control all the clocks if you want repeatable results. If you're saying some clocks can't be controlled that sounds like something needs to be refactored so it's testable. > > Benjamin > ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Fri, Feb 8, 2013 at 4:12 PM, Gregg Tavares wrote: > Can you expose a time or setTime function in DRT and some option that says > "let JS control the clock"? > Unfortunately, that only works in the simple cases. We could cheat the WebCore clock, but that would ultimately be wrong for animations running in threads or driven outside WebCore. Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
Can you expose a time or setTime function in DRT and some option that says
"let JS control the clock"?
Then a test could do something like
layoutTestController.overridePreference("JsControlledClock", "1");
// Render 5 frames over 1 second
var frame =0;
function renderFrame() {
layoutTestController.getWindowSnapshot(...);
if (frame == 5) {
finishTest();
return;
}
++frame;
layoutTestController.setAnimationClock(frame * 1000 / 5); // assumes
clock is in ms.
requestAnimationFrame(renderFrame);
}
On Fri, Feb 8, 2013 at 3:52 PM, Benjamin Poulain wrote:
> On Fri, Feb 8, 2013 at 3:16 PM, wrote:
>
>> The problem with dynamic features of the web like
>> animations/interactions is that they're non-deterministic, or at least a
>> lot less deterministic than static features of the web like layouts.
>> Ref tests, pixel tests etc. are tools built for deterministic testing:
>> load a file, take a snapshot, compare against a result. Testing an
>> animation (or a filter) needs to feel a lot more dynamic and expressive:
>> Animate green boxes, make sure that they're within a particular range at
>> particular points in time".
>>
>
> The tests also have to be deterministic and comprehensive. I am afraid of
> loosing both with the Render-to-Canvas approach.
>
> Can you give concrete examples of the kind of bugs you are hunting, and
> why testing cannot use the two methods suggested?
>
>
>> I think that when testing some of the dynamic features it's best to
>> create a minimal mechanism such as retrieving snapshot pixels, giving more
>> power to the tests themselves in a dynamic language like JavaScript, rather
>> than try to force the tests to fit into a predefined set of features
>> supported by the DRT/WTR infrastructure. Also this would allow some
>> differences between the ports, which implement animations a bit
>> differently, without constant rebaselines of reference results – e.g. A
>> heuristic animation timing test may stay valid across platform even if the
>> actual animation implementation is different.
>>
>
> Again, can you give concrete examples of this?
>
> Benjamin
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Fri, Feb 8, 2013 at 3:16 PM, wrote: > The problem with dynamic features of the web like > animations/interactions is that they're non-deterministic, or at least a > lot less deterministic than static features of the web like layouts. > Ref tests, pixel tests etc. are tools built for deterministic testing: > load a file, take a snapshot, compare against a result. Testing an > animation (or a filter) needs to feel a lot more dynamic and expressive: > Animate green boxes, make sure that they're within a particular range at > particular points in time". > The tests also have to be deterministic and comprehensive. I am afraid of loosing both with the Render-to-Canvas approach. Can you give concrete examples of the kind of bugs you are hunting, and why testing cannot use the two methods suggested? > I think that when testing some of the dynamic features it's best to create > a minimal mechanism such as retrieving snapshot pixels, giving more power > to the tests themselves in a dynamic language like JavaScript, rather than > try to force the tests to fit into a predefined set of features supported > by the DRT/WTR infrastructure. Also this would allow some differences > between the ports, which implement animations a bit differently, without > constant rebaselines of reference results – e.g. A heuristic animation > timing test may stay valid across platform even if the actual animation > implementation is different. > Again, can you give concrete examples of this? Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
From: ext Ryosuke Niwa mailto:rn...@webkit.org>> I'd like to propose a solution, and would welcome some feedback on whether it's a good one... The idea is that you would be able to programatically retrieve the current snapshot into a canvas ImageData, and then compare the pixel results with JavaScript in the LayoutTest. Something like: I had similar thoughts but my counter "proposal" is to let DRT/WTR generate multiple actual results either in the form of multiple layers in PNG or multiple PNG images. The advantage of this latter approach is that we can have multiple reference files as well. e.g. if a green box is to be moved from (0,0) to (50,0) and to (100, 0), we could create three reference files that correspond to each state. But perhaps there is a good reason you didn't choose this approach. Could you elaborate on the reason you picked this particular API? The problem with dynamic features of the web like animations/interactions is that they're non-deterministic, or at least a lot less deterministic than static features of the web like layouts. Ref tests, pixel tests etc. are tools built for deterministic testing: load a file, take a snapshot, compare against a result. Testing an animation (or a filter) needs to feel a lot more dynamic and expressive: Animate green boxes, make sure that they're within a particular range at particular points in time". I think that when testing some of the dynamic features it's best to create a minimal mechanism such as retrieving snapshot pixels, giving more power to the tests themselves in a dynamic language like JavaScript, rather than try to force the tests to fit into a predefined set of features supported by the DRT/WTR infrastructure. Also this would allow some differences between the ports, which implement animations a bit differently, without constant rebaselines of reference results – e.g. A heuristic animation timing test may stay valid across platform even if the actual animation implementation is different. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Fri, Feb 8, 2013 at 11:12 AM, Ryosuke Niwa wrote: > On Fri, Feb 8, 2013 at 5:35 AM, wrote: >> >> we don't currently have a solution in webkit's test infrastructure for >> testing animations "on the fly", or in general for testing multiple image >> results in the same test. >> (If this was discussed before and I'm unaware of that discussion, please >> stop me here...) >> >> This is because ImageDiff works on the single-test level, and you can't >> explicitly make image comparisons in the same test at different points in >> time. >> This has before caused us several regressions that were not caught by >> layout tests. >> >> I'd like to propose a solution, and would welcome some feedback on whether >> it's a good one... >> The idea is that you would be able to programatically retrieve the current >> snapshot into a canvas ImageData, and then compare the pixel results with >> JavaScript in the LayoutTest. Something like: > > > I had similar thoughts but my counter "proposal" is to let DRT/WTR generate > multiple actual results either in the form of multiple layers in PNG or > multiple PNG images. The advantage of this latter approach is that we can > have multiple reference files as well. e.g. if a green box is to be moved > from (0,0) to (50,0) and to (100, 0), we could create three reference files > that correspond to each state. > > But perhaps there is a good reason you didn't choose this approach. Could > you elaborate on the reason you picked this particular API? > I also appreciate the problem -- and would be interesting in solving it -- but I'm a bit concerned that either proposal is at least somewhat nondeterministic (or flaky), and I'm not sure how you'd be able to control which frames of the animation you were getting well enough. But I'm not much of expert here so I'm not sure how often it matters. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Fri, Feb 8, 2013 at 11:12 AM, Ryosuke Niwa wrote: > I had similar thoughts but my counter "proposal" is to let DRT/WTR > generate multiple actual results either in the form of multiple layers in > PNG or multiple PNG images. The advantage of this latter approach is that > we can have multiple reference files as well. e.g. if a green box is to be > moved from (0,0) to (50,0) and to (100, 0), we could create three reference > files that correspond to each state. > This sounds like a good idea for programmatic updates. I would just argue that instead of multiple images, it would be more maintainable to have multiple html references; a kind of multiple-frame-ref-test. A problem with this approach is, it is still fragile for animations. The position of a given frame is depends on timing. For animations, we could use mask-based test. The reference file of a test is a mask which gives an area where pixels can be. For example: -Take a block in a fixed position, scroll the page. The mask is the block in its original position. If for any frame, the block moved outside the mask, the test fails. -A block animating from left to right, while at the same time the page is scrolled. There is a deterministic path where the object can be. If for any frame, the block moved outside the path, the test fails. Making the mask would not always be trivial. The advantage is, if any single frame is wrong, we have a chance to catch it, regardless of the timing. Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
On Fri, Feb 8, 2013 at 5:35 AM, wrote: > we don't currently have a solution in webkit's test infrastructure for > testing animations "on the fly", or in general for testing multiple image > results in the same test. > (If this was discussed before and I'm unaware of that discussion, please > stop me here...) > > This is because ImageDiff works on the single-test level, and you can't > explicitly make image comparisons in the same test at different points in > time. > This has before caused us several regressions that were not caught by > layout tests. > > I'd like to propose a solution, and would welcome some feedback on whether > it's a good one... > The idea is that you would be able to programatically retrieve the current > snapshot into a canvas ImageData, and then compare the pixel results with > JavaScript in the LayoutTest. Something like: > I had similar thoughts but my counter "proposal" is to let DRT/WTR generate multiple actual results either in the form of multiple layers in PNG or multiple PNG images. The advantage of this latter approach is that we can have multiple reference files as well. e.g. if a green box is to be moved from (0,0) to (50,0) and to (100, 0), we could create three reference files that correspond to each state. But perhaps there is a good reason you didn't choose this approach. Could you elaborate on the reason you picked this particular API? - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
> I'd like to propose a solution, and would welcome some feedback on whether > it's a good one... > The idea is that you would be able to programatically retrieve the current > snapshot into a canvas ImageData, and then compare the pixel results with > JavaScript in the LayoutTest. Something like: +1 Lovely idea. For instance, we've been having some issue [1] testing CustomFilters, so having the ability to /sample/ the final result from JS would be great. [1] https://bugs.webkit.org/show_bug.cgi?id=104012 -- Bye, Michelangelo ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
From: webkit-dev-boun...@lists.webkit.org [webkit-dev-boun...@lists.webkit.org] on behalf of ext Robert Kroeger [rjkro...@chromium.org] > How would the test wait for the animation to happen? Tests that use > timeouts seem more likely to be flaky from what I've seen. Could > window.testRunner.getWindowSnapshot take a time delta of some kind? Most of the bugs I've encountered in this area can be caught by a test that would have "obvious" results that can buffer the flakiness, e.g. by moving a big green rectangle and testing only its center. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"
Questions inline.
On Fri, Feb 8, 2013 at 8:35 AM, wrote:
>
> Hellos
>
> we don't currently have a solution in webkit's test infrastructure for
> testing animations "on the fly", or in general for testing multiple image
> results in the same test.
I have been wishing for such a thing around how to test scroll animations.
> (If this was discussed before and I'm unaware of that discussion, please
> stop me here...)
>
> This is because ImageDiff works on the single-test level, and you can't
> explicitly make image comparisons in the same test at different points in
> time.
> This has before caused us several regressions that were not caught by
> layout tests.
>
> I'd like to propose a solution, and would welcome some feedback on whether
> it's a good one...
> The idea is that you would be able to programatically retrieve the current
> snapshot into a canvas ImageData, and then compare the pixel results with
> JavaScript in the LayoutTest. Something like:
>
> window.testRunner.getWindowSnapshot(function(snapshot) {
> if (snapshot.data[snapshot.width * 100 + 10] != 127)
> output.innerHTML = "FAIL";
> });
>
> WDYT?
How would the test wait for the animation to happen? Tests that use
timeouts seem more likely to be flaky from what I've seen. Could
window.testRunner.getWindowSnapshot take a time delta of some kind?
Rob.
> Noam
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
