Re: [webkit-dev] How to deal with 1-pixel differences in reftests ?

[Darin Adler]

> On Nov 18, 2015, at 4:36 AM, Carlos Alberto Lopez Perez  
> wrote:
> 
> Some reference tests give a 1-pixel or very few pixel differences [1].

Why? We need to understand why.

> Should we tolerate a few pixel differences for reftests ?

I don’t think we should. I think we should look more deeply into why these 
differences exist and consider writing the reference tests differently. We 
should not add tolerance when we don’t yet know why the results are different.

— Darin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] How to deal with 1-pixel differences in reftests ?

[Alexey Proskuryakov]

Hi,

I do not think that there is a way to algorithmically define what an acceptable 
difference is. Here are a few cases where it's critical to detect small 
differences:

- color management, e.g. testing different code paths that should match 
precisely;
- finding uninitialized memory use bugs in rendering pipeline, which do cause 
minor pixel noise;
- testing antialiasing and scaling behavior (e.g. that we should revert to high 
quality scaling after an animation is done);
- testing text rendering, where the difference can easily be small, e.g one 
accent over one letter on a mostly blank test result.

Talking from past experience that we had with pixel test tolerance and with 
retrying failing tests, I believe that leeway in reporting failures quickly 
causes significant deterioration in infrastructure quality, up to a point where 
we can't tell what's going on with many tests.

- Alexey



> 18 нояб. 2015 г., в 4:36, Carlos Alberto Lopez Perez  
> написал(а):
> 
> Hi,
> 
> Some reference tests give a 1-pixel or very few pixel differences [1].
> 
> I'm not sure if this really indicates a problem in the WebKit code, or
> it indicates that we are just too strict not allowing even a very small
> percentage in pixel differences for this kind of tests.
> 
> Should we tolerate a few pixel differences for reftests ?
> 
> I have done some tests, and the test in [1] passes for any value of
> tolerance >= 0.1% (with the GTK port).
> 
> I'm inclined to allow a very small value, for example a 0.001% (that
> would be 100 times stricter than the tolerance value we use for the
> other tests)
> 
> 
> Regards
> ---
> 
> [1]
> 
> For example, this is happening in the GTK port:
> https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20%28Tests%29/r192415%20%2812243%29/imported/blink/fast/canvas/canvas-clip-stack-persistence-diffs.html
> The diff image normalized (so you can see where is the diff):
> http://people.igalia.com/clopez/wkbug/151261/canvas-clip-stack-persistence-diff_normalized.png
> 
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] How to deal with 1-pixel differences in reftests ?

[Simon Fraser]

> On Nov 18, 2015, at 9:04 AM, Darin Adler  wrote:
> 
>> On Nov 18, 2015, at 4:36 AM, Carlos Alberto Lopez Perez  
>> wrote:
>> 
>> Some reference tests give a 1-pixel or very few pixel differences [1].
> 
> Why? We need to understand why.
> 
>> Should we tolerate a few pixel differences for reftests ?
> 
> I don’t think we should. I think we should look more deeply into why these 
> differences exist and consider writing the reference tests differently. We 
> should not add tolerance when we don’t yet know why the results are different.

There are some well-understood reasons why a test might not exactly match its 
reference. One is that the test uses compositing layers to do clipping, but the 
reference just clips with drawing, and these are not expected to give a 
pixel-perfect match.

I proposed a way to allow a test to specify a custom tolerance in 
https://bugs.webkit.org/show_bug.cgi?id=149828. If we had this, it would allow 
me to fix 142258  and 146523 
.

Simon

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] How to deal with 1-pixel differences in reftests ?

[Carlos Alberto Lopez Perez]

Hi,

Some reference tests give a 1-pixel or very few pixel differences [1].

I'm not sure if this really indicates a problem in the WebKit code, or
it indicates that we are just too strict not allowing even a very small
percentage in pixel differences for this kind of tests.

Should we tolerate a few pixel differences for reftests ?

I have done some tests, and the test in [1] passes for any value of
tolerance >= 0.1% (with the GTK port).

I'm inclined to allow a very small value, for example a 0.001% (that
would be 100 times stricter than the tolerance value we use for the
other tests)


Regards
---

[1]

For example, this is happening in the GTK port:
https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20%28Tests%29/r192415%20%2812243%29/imported/blink/fast/canvas/canvas-clip-stack-persistence-diffs.html
The diff image normalized (so you can see where is the diff):
http://people.igalia.com/clopez/wkbug/151261/canvas-clip-stack-persistence-diff_normalized.png


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev]

[Yoav Weiss]

Hello Alexey,

On Wed, Nov 18, 2015 at 1:27 AM, Alexey Proskuryakov  wrote:

> Hello Yoav,
>
> Is there any technology on the horizon that would simplify doing this kind
> of optimization? If done manually, this seems:
> - complicated, so only a few sites will do this;
> - very likely to go stale, as the content changes, but preload
> instructions do not get updated;
> - related to the above, the failure mode is opaque, as the website will
> only get a little slower to load, and not break functionally.
>

I believe that the goal for many of the use cases is to automate that
optimization in order to mitigate the issues you raised:
* For static sites, build scripts could easily add these hints as part of
the build process.
* For dynamic site frameworks (e.g. Django), middleware code can be used to
add the hints dynamically.
* CDNs and optimization engines can use site analysis and heuristics in
order to add the hints automatically for their customers.

The framework and CDN use cases can mean that few implementations may
result in many deployments of that optimization.

Regarding the third point (opaque failure mode), we could emit console
errors in case where the preloaded resource is not used.


> Sending the preload requests in HTTP response headers seems like it would
> provide the most benefit, but is also more prone to the above issues.
>

For the CDN/optimization-engine use-case, HTTP headers would be
significantly easier to add automatically.


> Preloading resources as untyped data doesn't seem like a good match to the
> loader implementation mostly dealing with typed resources.
>

The `as` attribute makes it so that the resources will not be downloaded
untyped, but will (most likely) merge with the current HTMLPreloadScanner
loading logic.


> Additionally, fetching depends on the referring document's properties
> (notably the charset is inherited for same origin subresources). This is
> not necessarily a blocker, but the proposal adds a different way to think
> about subresource loading.
>

Good point to consider while implementing! :)


>
> There appears to be some feature duplication with HTTP/2 server push
> functionality, could you please characterize the differences that would
> make it worth having both?
>

Preload has several advantages over HTTP/2 push:
* Preload is not limited to first-party resources.
* Preload will take into consideration the browser's cache.
* Preload doesn't require server side smarts (so can be used to speed up
static sites hosted on e.g. GitHub pages or S3).
* Preload is not limited to secure origins.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Prototyping Custom Elements API

[Ryosuke Niwa]

Hi all,

I would like to start prototyping custom elements API in WebKit.
Custom elements API allows authors to define a author-defined tag name
associated with its own JS interface.  There is a draft spec proposed
by Google (http://w3c.github.io/webcomponents/spec/custom/) but there
has been many open questions regarding the construction timing and
mechanism:
https://github.com/w3c/webcomponents/tree/gh-pages/proposals

Unfortunately, many of those open questions are hard to answer without
further implementation experience and feedback as well as Web
developer feedback.  As such, I've started prototyping what we've been
discussing at W3C and posted work-in-progress patches on
webkit.org/b/150225.

Since it's troublesome for developers to build trunk WebKit with my
changes, I plan to land my experimental API on trunk WebKit behind a
build flag, enabled it by default for Mac port (as done for shadow DOM
API).

Please let me know if you have any concerns and/or comments.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] NetworkSession

[Alex Christensen]

As part of this work, I’m also planning to remove the non-NetworkProcess code, 
both run time and compile time. Based on discussion at the WebKit contributors’ 
meeting I assume there is no objection to this.  See 
https://bugs.webkit.org/show_bug.cgi?id=151418 


Alex

> On Nov 9, 2015, at 11:32 AM, Alex Christensen  wrote:
> 
> I made new abstractions for loading in WebKit2: NetworkSession and 
> NetworkDataTask.  It is disabled by default right now, but if you switch 
> USE_NETWORK_SESSION to 1, it mostly works on Mac with features like 
> authentication challenges not implemented yet.  I believe these new 
> abstractions fit better with libsoup, but I’d like feedback on what an actual 
> libsoup implementation would need.  I’m planning on removing the 
> ResourceHandle use in WebKit2 and the async/continue callbacks in 
> ResourceHandleClient and ResourceHandle once this work is done.
> 
> Alex
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] How to deal with 1-pixel differences in reftests ?

[Alexey Proskuryakov]

> 18 нояб. 2015 г., в 11:50, Simon Fraser  написал(а):
> 
> There are some well-understood reasons why a test might not exactly match its 
> reference. One is that the test uses compositing layers to do clipping, but 
> the reference just clips with drawing, and these are not expected to give a 
> pixel-perfect match.
> 
> I proposed a way to allow a test to specify a custom tolerance in 
> https://bugs.webkit.org/show_bug.cgi?id=149828 
> . If we had this, it would 
> allow me to fix 142258  and 
> 146523 .

Bug 142258 also serves as an example for why we shouldn't do this. Both known 
reasons for it are actual bugs that needed to be discovered, and need to be 
fixed.

What are the causes for flakiness in bug 146523?

- Alexey

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev