From: ext Benjamin Poulain <benja...@webkit.org<mailto:benja...@webkit.org>>
Date: Thursday, February 14, 2013 8:07 AM
To: Dongsung Huang <luxte...@company100.net<mailto:luxte...@company100.net>>
Cc: "webkit-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>" 
<webkit-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org>>
Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction 
pixel-results "on the fly"

On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang 
<luxte...@company100.net<mailto:luxte...@company100.net>> wrote:
I like this idea. I cannot find any harm if we have this functionality.

Those changes are not harmless. There are people monitoring tests results full 
time in order to keep WebKit in good shape. No other part of WebKit require 
continuous attention.

Case 1: CSS Filters & Shaders
I wanted this test functionality when I commented http://webkit.org/b/97859#c19
If I want to make gaussian blur test, I prefer using 'getPixel' test as follows,

Why wasn't a ref-test a better solution in this particular case?
Because gaussian blurs on the GPU are not accurate, and look slightly different 
on different GPUs, but usually "close enough".
We need a way to measure "close enough" for features where all you can get is 
"close enough"! Ref-tests and pixel-tests are way to rigid for that, and 
require constant rebaselines and headaches.


Case 2: Fixed Position Element
[...]
function repeatedlyCalledDuringScrolling() {
    ASSERT(getPixel(15, 9) == white);
    ASSERT(getPixel(15, 10) == green);
    ASSERT(getPixel(9, 15) == white);
    ASSERT(getPixel(10, 15) == green);
    ....
}

I think this shows what I said about correctness and readability:
-Asserting the correctness of the test and the result becomes close to 
impossible for the reader. One has to review the full code to have a chance of 
understanding an error.
We can generate PNGs when a snapshot is requested so that the tester can 
eyeball the results.

-You cannot cover non trivial cases (images, text, form elements, etc).
-And it is inefficient. You have to render each frame on the UIProcess, move it 
to the WebProcess, and box it for JavaScript to process (with pixel format 
conversions depending on your graphics system)
Another option is to do expose a partial comparison mechanism that runs in  the 
UI process.
E.g.
var snapshotHandle = testRunner.createSnapshot();
testRunner.comparePixel(snapshotHandle, 15, 20, 'white', { tolerance: 0.2 });

This way we can generate a visual representation of failures.
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Reply via email to