Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-14 Thread Dongsung Huang
> > >> I'm curious, what would you imagine the ref test contains? >> > > If I am not mistaken, the composition operations are parallel with the > ones of SVG and Canvas (aren't they?). > I would have attempted comparing the 3 implementations as it seems to me > the pixels values should be the same.

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-14 Thread Dirk Pranke
On Wed, Feb 13, 2013 at 11:33 PM, Benjamin Poulain wrote: > On Wed, Feb 13, 2013 at 11:16 PM, Dirk Pranke wrote: >> >> > Those changes are not harmless. There are people monitoring tests >> > results >> > full time in order to keep WebKit in good shape. No other part of WebKit >> > require contin

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread noam . rosenthal
-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>" mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Wed, Feb 13, 2013 at 11:27 PM, mailto:noam.rosent...@nokia.com&g

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Benjamin Poulain
On Wed, Feb 13, 2013 at 11:27 PM, wrote: > Why wasn't a ref-test a better solution in this particular case? > > Because gaussian blurs on the GPU are not accurate, and look slightly > different on different GPUs, but usually "close enough". > We need a way to measure "close enough" for features

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Benjamin Poulain
On Wed, Feb 13, 2013 at 11:16 PM, Dirk Pranke wrote: > > Those changes are not harmless. There are people monitoring tests results > > full time in order to keep WebKit in good shape. No other part of WebKit > > require continuous attention. > > I'm sorry, but either I don't understand Dongsung's

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread noam . rosenthal
Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang mailto:luxte...@company100.net>> wrote: I like this idea. I cannot find any harm if we have this functionality. Those changes a

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Dirk Pranke
On Wed, Feb 13, 2013 at 11:07 PM, Benjamin Poulain wrote: > On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang > wrote: >> >> I like this idea. I cannot find any harm if we have this functionality. > > > Those changes are not harmless. There are people monitoring tests results > full time in order t

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Benjamin Poulain
On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang wrote: > I like this idea. I cannot find any harm if we have this functionality. > Those changes are not harmless. There are people monitoring tests results full time in order to keep WebKit in good shape. No other part of WebKit require continuous

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-13 Thread Dongsung Huang
I like this idea. I cannot find any harm if we have this functionality. When a pixel test is more succinct, we can make a pixel test. When a 'getPixel on Javascript' test is more succinct, we can make a 'getPixel' test. As http://trac.webkit.org/wiki/Writing%20Layout%20Tests%20for%20DumpRenderTree

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-11 Thread Dirk Pranke
On Sun, Feb 10, 2013 at 1:30 AM, Ryosuke Niwa wrote: > On Sat, Feb 9, 2013 at 5:35 PM, Dirk Pranke wrote: >> >> Perhaps -- and this is a tangent to this thread -- we could also >> investigate ways of writing tests that did render pngs but told NRWT >> to ignore them (e.g., dumpAsTextAndIgnoredIma

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-10 Thread Ryosuke Niwa
On Sat, Feb 9, 2013 at 5:35 PM, Dirk Pranke wrote: > Perhaps -- and this is a tangent to this thread -- we could also > investigate ways of writing tests that did render pngs but told NRWT > to ignore them (e.g., dumpAsTextAndIgnoredImage() or something), or > conveyed which parts of the image we

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread Dirk Pranke
On Sat, Feb 9, 2013 at 1:57 AM, Benjamin Poulain wrote: > On Sat, Feb 9, 2013 at 1:24 AM, wrote: >> >> I did not say they cannot be tested with the two methods suggested :) It's >> more about a preference to move some of those decisions from the test >> infrastructure to the test itself, but pot

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread noam . rosenthal
Oops, forgot to reply to the list. From: Noam Rosenthal mailto:noam.rosent...@nokia.com>> From: ext Simon Fraser mailto:simon.fra...@apple.com>> The existing pauseAnimation DRT API can stop an animation mid-flight, to allow for reliable snapshotting. Why isn't this enough? I would say that the ma

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread Simon Fraser
> Date: Saturday, February 9, 2013 10:57 AM > To: Noam Rosenthal > Cc: "rn...@webkit.org" , "webkit-dev@lists.webkit.org" > > Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction > pixel-results "on the fly" > >&

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread noam . rosenthal
, "webkit-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>" mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Sat, Feb 9, 2013 at 1:24 AM, mailto:noam.rosent...@nokia.com>

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread noam . rosenthal
From: ext Benjamin Poulain mailto:benja...@webkit.org>> This is the situation I am afraid with a solution where pixels would be evaluated from JavaScript. You can test, but you lack visibility why something is correct or incorrect. Text tests, ref-tests and pixel tests have a great readability

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread Benjamin Poulain
On Sat, Feb 9, 2013 at 1:24 AM, wrote: >I did not say they cannot be tested with the two methods suggested :) > It's more about a preference to move some of those decisions from the test > infrastructure to the test itself, but potentially those bugs could be > tested in either way. > Exampl

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-09 Thread noam . rosenthal
.org<mailto:webkit-dev@lists.webkit.org>" mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly" On Fri, Feb 8, 2013 at 3:16 PM, mailto:noam.rosent...@nokia.com>> wrote: The proble

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Philip Rogers
I'm not sure if this is generally known: we use [svg document].setCurrentTime(time) in the svg/animations tests for both script and pixel tests. The SVG animation test harness leaves a lot to be desired, but it may be a good place to start for a more general DRT approach: http://trac.webkit.org/bro

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Gregg Tavares
On Fri, Feb 8, 2013 at 4:16 PM, Benjamin Poulain wrote: > On Fri, Feb 8, 2013 at 4:12 PM, Gregg Tavares wrote: > >> Can you expose a time or setTime function in DRT and some option that >> says "let JS control the clock"? >> > > Unfortunately, that only works in the simple cases. > We could cheat

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Benjamin Poulain
On Fri, Feb 8, 2013 at 4:12 PM, Gregg Tavares wrote: > Can you expose a time or setTime function in DRT and some option that says > "let JS control the clock"? > Unfortunately, that only works in the simple cases. We could cheat the WebCore clock, but that would ultimately be wrong for animation

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Gregg Tavares
Can you expose a time or setTime function in DRT and some option that says "let JS control the clock"? Then a test could do something like layoutTestController.overridePreference("JsControlledClock", "1"); // Render 5 frames over 1 second var frame =0; function renderFrame() {

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Benjamin Poulain
On Fri, Feb 8, 2013 at 3:16 PM, wrote: > The problem with dynamic features of the web like > animations/interactions is that they're non-deterministic, or at least a > lot less deterministic than static features of the web like layouts. > Ref tests, pixel tests etc. are tools built for determini

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread noam . rosenthal
From: ext Ryosuke Niwa mailto:rn...@webkit.org>> I'd like to propose a solution, and would welcome some feedback on whether it's a good one... The idea is that you would be able to programatically retrieve the current snapshot into a canvas ImageData, and then compare the pixel results with Ja

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Dirk Pranke
On Fri, Feb 8, 2013 at 11:12 AM, Ryosuke Niwa wrote: > On Fri, Feb 8, 2013 at 5:35 AM, wrote: >> >> we don't currently have a solution in webkit's test infrastructure for >> testing animations "on the fly", or in general for testing multiple image >> results in the same test. >> (If this was disc

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Benjamin Poulain
On Fri, Feb 8, 2013 at 11:12 AM, Ryosuke Niwa wrote: > I had similar thoughts but my counter "proposal" is to let DRT/WTR > generate multiple actual results either in the form of multiple layers in > PNG or multiple PNG images. The advantage of this latter approach is that > we can have multiple

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Ryosuke Niwa
On Fri, Feb 8, 2013 at 5:35 AM, wrote: > we don't currently have a solution in webkit's test infrastructure for > testing animations "on the fly", or in general for testing multiple image > results in the same test. > (If this was discussed before and I'm unaware of that discussion, please > sto

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Michelangelo De Simone
> I'd like to propose a solution, and would welcome some feedback on whether > it's a good one... > The idea is that you would be able to programatically retrieve the current > snapshot into a canvas ImageData, and then compare the pixel results with > JavaScript in the LayoutTest. Something lik

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread noam . rosenthal
From: webkit-dev-boun...@lists.webkit.org [webkit-dev-boun...@lists.webkit.org] on behalf of ext Robert Kroeger [rjkro...@chromium.org] > How would the test wait for the animation to happen? Tests that use > timeouts seem more likely to be flaky from what

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

2013-02-08 Thread Robert Kroeger
Questions inline. On Fri, Feb 8, 2013 at 8:35 AM, wrote: > > Hellos > > we don't currently have a solution in webkit's test infrastructure for > testing animations "on the fly", or in general for testing multiple image > results in the same test. I have been wishing for such a thing around how t