Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-14 Thread Dongsung Huang
> > >> I'm curious, what would you imagine the ref test contains? >> > > If I am not mistaken, the composition operations are parallel with the > ones of SVG and Canvas (aren't they?). > I would have attempted comparing the 3 implementations as it seems to me > the pixels values should be the same.

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-14 Thread Dirk Pranke
On Wed, Feb 13, 2013 at 11:33 PM, Benjamin Poulain wrote: > On Wed, Feb 13, 2013 at 11:16 PM, Dirk Pranke wrote: >> >> > Those changes are not harmless. There are people monitoring tests >> > results >> > full time in order to keep WebKit in good shape. No other part of WebKit >> > require contin

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread noam . rosenthal
-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>" mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Wed, Feb 13, 2013 at 11:27 PM, mailto:noam.rosent...@nokia.com&g

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Benjamin Poulain
On Wed, Feb 13, 2013 at 11:27 PM, wrote: > Why wasn't a ref-test a better solution in this particular case? > > Because gaussian blurs on the GPU are not accurate, and look slightly > different on different GPUs, but usually "close enough". > We need a way to measure "close enough" for features

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Benjamin Poulain
On Wed, Feb 13, 2013 at 11:16 PM, Dirk Pranke wrote: > > Those changes are not harmless. There are people monitoring tests results > > full time in order to keep WebKit in good shape. No other part of WebKit > > require continuous attention. > > I'm sorry, but either I don't understand Dongsung's

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread noam . rosenthal
Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang mailto:luxte...@company100.net>> wrote: I like this idea. I cannot find any harm if we have this functionality. Those changes a

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Dirk Pranke
On Wed, Feb 13, 2013 at 11:07 PM, Benjamin Poulain wrote: > On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang > wrote: >> >> I like this idea. I cannot find any harm if we have this functionality. > > > Those changes are not harmless. There are people monitoring tests results > full time in order t

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Benjamin Poulain
On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang wrote: > I like this idea. I cannot find any harm if we have this functionality. > Those changes are not harmless. There are people monitoring tests results full time in order to keep WebKit in good shape. No other part of WebKit require continuous

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Dongsung Huang
I like this idea. I cannot find any harm if we have this functionality. When a pixel test is more succinct, we can make a pixel test. When a 'getPixel on Javascript' test is more succinct, we can make a 'getPixel' test. As http://trac.webkit.org/wiki/Writing%20Layout%20Tests%20for%20DumpRenderTree

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-11 Thread Dirk Pranke
On Sun, Feb 10, 2013 at 1:30 AM, Ryosuke Niwa wrote: > On Sat, Feb 9, 2013 at 5:35 PM, Dirk Pranke wrote: >> >> Perhaps -- and this is a tangent to this thread -- we could also >> investigate ways of writing tests that did render pngs but told NRWT >> to ignore them (e.g., dumpAsTextAndIgnoredIma

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-10 Thread Ryosuke Niwa
On Sat, Feb 9, 2013 at 5:35 PM, Dirk Pranke wrote: > Perhaps -- and this is a tangent to this thread -- we could also > investigate ways of writing tests that did render pngs but told NRWT > to ignore them (e.g., dumpAsTextAndIgnoredImage() or something), or > conveyed which parts of the image we

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread Dirk Pranke
On Sat, Feb 9, 2013 at 1:57 AM, Benjamin Poulain wrote: > On Sat, Feb 9, 2013 at 1:24 AM, wrote: >> >> I did not say they cannot be tested with the two methods suggested :) It's >> more about a preference to move some of those decisions from the test >> infrastructure to the test itself, but pot

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread noam . rosenthal
Oops, forgot to reply to the list. From: Noam Rosenthal mailto:noam.rosent...@nokia.com>> From: ext Simon Fraser mailto:simon.fra...@apple.com>> The existing pauseAnimation DRT API can stop an animation mid-flight, to allow for reliable snapshotting. Why isn't this enough? I would say that the ma

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread Simon Fraser
> Date: Saturday, February 9, 2013 10:57 AM > To: Noam Rosenthal > Cc: "rn...@webkit.org" , "webkit-dev@lists.webkit.org" > > Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction > pixel-results "on the fly" > >&

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread noam . rosenthal
, "webkit-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>" mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Sat, Feb 9, 2013 at 1:24 AM, mailto:noam.rosent...@nokia.com>

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread noam . rosenthal
From: ext Benjamin Poulain mailto:benja...@webkit.org>> This is the situation I am afraid with a solution where pixels would be evaluated from JavaScript. You can test, but you lack visibility why something is correct or incorrect. Text tests, ref-tests and pixel tests have a great readability

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread Benjamin Poulain
On Sat, Feb 9, 2013 at 1:24 AM, wrote: >I did not say they cannot be tested with the two methods suggested :) > It's more about a preference to move some of those decisions from the test > infrastructure to the test itself, but potentially those bugs could be > tested in either way. > Exampl

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread noam . rosenthal
.org<mailto:webkit-dev@lists.webkit.org>" mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Fri, Feb 8, 2013 at 3:16 PM, mailto:noam.rosent...@nokia.com>> wrote: The proble

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Philip Rogers
I'm not sure if this is generally known: we use [svg document].setCurrentTime(time) in the svg/animations tests for both script and pixel tests. The SVG animation test harness leaves a lot to be desired, but it may be a good place to start for a more general DRT approach: http://trac.webkit.org/bro

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Gregg Tavares
On Fri, Feb 8, 2013 at 4:16 PM, Benjamin Poulain wrote: > On Fri, Feb 8, 2013 at 4:12 PM, Gregg Tavares wrote: > >> Can you expose a time or setTime function in DRT and some option that >> says "let JS control the clock"? >> > > Unfortunately, that only works in the simple cases. > We could cheat

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Benjamin Poulain
On Fri, Feb 8, 2013 at 4:12 PM, Gregg Tavares wrote: > Can you expose a time or setTime function in DRT and some option that says > "let JS control the clock"? > Unfortunately, that only works in the simple cases. We could cheat the WebCore clock, but that would ultimately be wrong for animation

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Gregg Tavares
Can you expose a time or setTime function in DRT and some option that says "let JS control the clock"? Then a test could do something like layoutTestController.overridePreference("JsControlledClock", "1"); // Render 5 frames over 1 second var frame =0; function renderFrame() {

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Benjamin Poulain
On Fri, Feb 8, 2013 at 3:16 PM, wrote: > The problem with dynamic features of the web like > animations/interactions is that they're non-deterministic, or at least a > lot less deterministic than static features of the web like layouts. > Ref tests, pixel tests etc. are tools built for determini

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread noam . rosenthal
From: ext Ryosuke Niwa mailto:rn...@webkit.org>> I'd like to propose a solution, and would welcome some feedback on whether it's a good one... The idea is that you would be able to programatically retrieve the current snapshot into a canvas ImageData, and then compare the pixel results with Ja

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Dirk Pranke
On Fri, Feb 8, 2013 at 11:12 AM, Ryosuke Niwa wrote: > On Fri, Feb 8, 2013 at 5:35 AM, wrote: >> >> we don't currently have a solution in webkit's test infrastructure for >> testing animations "on the fly", or in general for testing multiple image >> results in the same test. >> (If this was disc

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Benjamin Poulain
On Fri, Feb 8, 2013 at 11:12 AM, Ryosuke Niwa wrote: > I had similar thoughts but my counter "proposal" is to let DRT/WTR > generate multiple actual results either in the form of multiple layers in > PNG or multiple PNG images. The advantage of this latter approach is that > we can have multiple

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Ryosuke Niwa
On Fri, Feb 8, 2013 at 5:35 AM, wrote: > we don't currently have a solution in webkit's test infrastructure for > testing animations "on the fly", or in general for testing multiple image > results in the same test. > (If this was discussed before and I'm unaware of that discussion, please > sto

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Michelangelo De Simone
> I'd like to propose a solution, and would welcome some feedback on whether > it's a good one... > The idea is that you would be able to programatically retrieve the current > snapshot into a canvas ImageData, and then compare the pixel results with > JavaScript in the LayoutTest. Something lik

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread noam . rosenthal
From: webkit-dev-boun...@lists.webkit.org [webkit-dev-boun...@lists.webkit.org] on behalf of ext Robert Kroeger [rjkro...@chromium.org] > How would the test wait for the animation to happen? Tests that use > timeouts seem more likely to be flaky from what

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Robert Kroeger
Questions inline. On Fri, Feb 8, 2013 at 8:35 AM, wrote: > > Hellos > > we don't currently have a solution in webkit's test infrastructure for > testing animations "on the fly", or in general for testing multiple image > results in the same test. I have been wishing for such a thing around how t

[webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread noam . rosenthal
Hellos we don't currently have a solution in webkit's test infrastructure for testing animations "on the fly", or in general for testing multiple image results in the same test. (If this was discussed before and I'm unaware of that discussion, please stop me here...) This is because ImageDiff