Re: [webkit-dev] Suggesting to enable paint timing by default

[Noam Rosenthal]

On Thu, 20 Aug 2020 at 6:26 Maciej Stachowiak  wrote:

>
>
> On Jul 17, 2020, at 12:12 AM, Noam Rosenthal  wrote:
>
>
>
> On Thu, Jul 16, 2020 at 11:03 PM Keith Miller 
> wrote:
>
>> Results appear to be neutral on the page load time benchmark, so you
>> should be good on that front. I don’t know who the best person to vet the
>> maturity of the code is though, sorry.
>>
>
> Thanks a lot Keith, I appreciate it!
> @Maciej Stachowiak , what would be a good way to assert
> whether the code maturity is good enough to enable paint timing by default?
> The original code was reviewed by smfr and initially by zalan. It's
> covered by over 30 tests, mostly WPT, and A/B tests show no effect on load
> times as per Keith's check.
> Would asking for additional reviews be the next step? From whom?
>
>
> At this point, if a reviewer approves a patch to enable it by default on
> trunk, I think you are good to go.
>
> As a courtesy to Apple, I’d ask you to hold off on landing until
> mid-September, but that is optional.
>

Great
The patch has already been reviewed,
I will re-land it in mid September.
Thank you!

>
>  - Maciej
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Suggesting to enable paint timing by default

[Noam Rosenthal]

On Mon, Jul 13, 2020 at 9:15 PM Keith Miller  wrote:

> If you tell me how to enable paint timing by default, I can start an A/B
> task for you. I’m probably not qualified to review it for code maturity
> though.
>
Awesome, thanks!
It's an experimental runtime flag calledPaintTimingEnabled
I have a patch for enabling it by default here:
https://bugs.webkit.org/show_bug.cgi?id=211736
We mainly need to test that measuring paint timing doesn't (badly)
influence loading performance.



>
> Cheers,
> Keith
>
> On Jul 13, 2020, at 3:02 AM, Noam Rosenthal  wrote:
>
>
>
> On Wed, May 27, 2020 at 12:04 PM Yoav Weiss  wrote:
>
>> +Ryosuke Niwa  +Alex Christensen
>>  who were involved in the spec discussions.
>>
>> On Wed, May 27, 2020 at 10:29 AM Noam Rosenthal  wrote:
>>
>>>
>>>
>>> Following up on this.
>>>
>>>> FOn Tue, May 12, 2020 at 10:28 AM Maciej Stachowiak 
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On May 11, 2020, at 9:53 PM, Noam Rosenthal  wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Tue, May 12, 2020 at 1:36 AM Maciej Stachowiak 
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> I noticed from comments in one of the Radars that the patch may
>>>>>> result in an additional “fake paint”, so it should probably be 
>>>>>> performance
>>>>>> tested. Have you done any testing?
>>>>>>
>>>>> I've tested it locally, I haven't noticed any significant side effect,
>>>>> because in complex situations the fake paint only happens once per page 
>>>>> and
>>>>> bails early once contentfulness is detected. but I can run any additional
>>>>> test needed.
>>>>>
>>>>>
>>>>>> We’ll likely want to A/B some of Apple’s page load speed benchmarks.
>>>>>>
>>>>> A/B testing load speed sounds sensible. How do we go about doing that?
>>>>>
>>>>>
>>>>> Unfortunately our page load speed benchmarks are not public because
>>>>> they incorporate captured page content, which we can’t freely 
>>>>> redistribute.
>>>>>
>>>>> So, can someone else from Apple review that the code is mature enough
>>> for this? Simon had reviewed the original patch. Maybe Zalan/Darin?
>>>
>>> A helpful person from Apple may be able to set up an A/B test for this
>>>>> patch.
>>>>>
>>>> What's required to ask for help from a helpful person at Apple? :)
>>>
>> Hola
> Pinging about this again :)
> The code for paint timing API is sitting there in the repo, waiting either
> for internal Apple A/B tests, for an additional code maturity review, or
> for enabling it by default... I'm here if any changes in the code need to
> be made.
>
> Trying to figure out how we can proceed with this... @Maciej Stachowiak
> ?
> Cheers
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Suggesting to enable paint timing by default

[Noam Rosenthal]

On Wed, May 27, 2020 at 12:04 PM Yoav Weiss  wrote:

> +Ryosuke Niwa  +Alex Christensen
>  who were involved in the spec discussions.
>
> On Wed, May 27, 2020 at 10:29 AM Noam Rosenthal  wrote:
>
>>
>>
>> Following up on this.
>>
>>> FOn Tue, May 12, 2020 at 10:28 AM Maciej Stachowiak 
>>> wrote:
>>>
>>>>
>>>>
>>>> On May 11, 2020, at 9:53 PM, Noam Rosenthal  wrote:
>>>>
>>>>
>>>>
>>>> On Tue, May 12, 2020 at 1:36 AM Maciej Stachowiak 
>>>> wrote:
>>>>
>>>>>
>>>>> I noticed from comments in one of the Radars that the patch may result
>>>>> in an additional “fake paint”, so it should probably be performance 
>>>>> tested.
>>>>> Have you done any testing?
>>>>>
>>>> I've tested it locally, I haven't noticed any significant side effect,
>>>> because in complex situations the fake paint only happens once per page and
>>>> bails early once contentfulness is detected. but I can run any additional
>>>> test needed.
>>>>
>>>>
>>>>> We’ll likely want to A/B some of Apple’s page load speed benchmarks.
>>>>>
>>>> A/B testing load speed sounds sensible. How do we go about doing that?
>>>>
>>>>
>>>> Unfortunately our page load speed benchmarks are not public because
>>>> they incorporate captured page content, which we can’t freely redistribute.
>>>>
>>>> So, can someone else from Apple review that the code is mature enough
>> for this? Simon had reviewed the original patch. Maybe Zalan/Darin?
>>
>> A helpful person from Apple may be able to set up an A/B test for this
>>>> patch.
>>>>
>>> What's required to ask for help from a helpful person at Apple? :)
>>
> Hola
Pinging about this again :)
The code for paint timing API is sitting there in the repo, waiting either
for internal Apple A/B tests, for an additional code maturity review, or
for enabling it by default... I'm here if any changes in the code need to
be made.

Trying to figure out how we can proceed with this... @Maciej Stachowiak
?
Cheers
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Suggesting to enable paint timing by default

[Noam Rosenthal]

Following up on this.

> FOn Tue, May 12, 2020 at 10:28 AM Maciej Stachowiak  wrote:
>
>>
>>
>> On May 11, 2020, at 9:53 PM, Noam Rosenthal  wrote:
>>
>>
>>
>> On Tue, May 12, 2020 at 1:36 AM Maciej Stachowiak  wrote:
>>
>>>
>>> I noticed from comments in one of the Radars that the patch may result
>>> in an additional “fake paint”, so it should probably be performance tested.
>>> Have you done any testing?
>>>
>> I've tested it locally, I haven't noticed any significant side effect,
>> because in complex situations the fake paint only happens once per page and
>> bails early once contentfulness is detected. but I can run any additional
>> test needed.
>>
>>
>>> We’ll likely want to A/B some of Apple’s page load speed benchmarks.
>>>
>> A/B testing load speed sounds sensible. How do we go about doing that?
>>
>>
>> Unfortunately our page load speed benchmarks are not public because they
>> incorporate captured page content, which we can’t freely redistribute.
>>
>> So, can someone else from Apple review that the code is mature enough for
this? Simon had reviewed the original patch. Maybe Zalan/Darin?

A helpful person from Apple may be able to set up an A/B test for this
>> patch.
>>
> What's required to ask for help from a helpful person at Apple? :)
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Suggesting to enable paint timing by default

[Noam Rosenthal]

On Tue, May 12, 2020 at 10:28 AM Maciej Stachowiak  wrote:

>
>
> On May 11, 2020, at 9:53 PM, Noam Rosenthal  wrote:
>
>
>
> On Tue, May 12, 2020 at 1:36 AM Maciej Stachowiak  wrote:
>
>>
>> I noticed from comments in one of the Radars that the patch may result in
>> an additional “fake paint”, so it should probably be performance tested.
>> Have you done any testing?
>>
> I've tested it locally, I haven't noticed any significant side effect,
> because in complex situations the fake paint only happens once per page and
> bails early once contentfulness is detected. but I can run any additional
> test needed.
>
>
>> We’ll likely want to A/B some of Apple’s page load speed benchmarks.
>>
> A/B testing load speed sounds sensible. How do we go about doing that?
>
>
> Unfortunately our page load speed benchmarks are not public because they
> incorporate captured page content, which we can’t freely redistribute.
>
> A helpful person from Apple may be able to set up an A/B test for this
> patch.
>
 Understood. I'd be happy to facilitate/support/do what's necessary to move
this forward.
Thanks!
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Suggesting to enable paint timing by default

[Noam Rosenthal]

On Tue, May 12, 2020 at 1:36 AM Maciej Stachowiak  wrote:

>
> I noticed from comments in one of the Radars that the patch may result in
> an additional “fake paint”, so it should probably be performance tested.
> Have you done any testing?
>
I've tested it locally, I haven't noticed any significant side effect,
because in complex situations the fake paint only happens once per page and
bails early once contentfulness is detected. but I can run any additional
test needed.


> We’ll likely want to A/B some of Apple’s page load speed benchmarks.
>
A/B testing load speed sounds sensible. How do we go about doing that?


>
> I’d like to hear from others on maturity of the spec and readiness of the
> code.
>

>  - Maciej
>
> On May 11, 2020, at 11:44 AM, Noam Rosenthal  wrote:
>
> Following the discussion with Simon in
> https://bugs.webkit.org/show_bug.cgi?id=78011
>
> Since we have a pretty stable spec (https://w3c.github.io/paint-timing/),
> lots of passing web platform tests, other browser vendor support and a
> working implementation of first contentful paint, I am planning to submit a
> patch to enable paint timing by default.
>
> https://bugs.webkit.org/show_bug.cgi?id=211736
>
> Any objections? Other thoughts?
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Suggesting to enable paint timing by default

[Noam Rosenthal]

Following the discussion with Simon in
https://bugs.webkit.org/show_bug.cgi?id=78011

Since we have a pretty stable spec (https://w3c.github.io/paint-timing/),
lots of passing web platform tests, other browser vendor support and a
working implementation of first contentful paint, I am planning to submit a
patch to enable paint timing by default.

https://bugs.webkit.org/show_bug.cgi?id=211736

Any objections? Other thoughts?
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Thu, Mar 5, 2020 at 4:24 AM Maciej Stachowiak  wrote:

>
>
>> There has been a lot of work done on the spec front since the comments
> above, and I think we're ready for a re-review the state.
> * The next pull request: https://github.com/whatwg/html/pull/5112 has
> been thumbed up to make it into the HTML spec, and is separate from
> client-hints, but requires second implementor interest, which is what this
> email thread is for :)
>
>
> I don’t think it has? I’m seeing:
> "[ ]  At least two implementers are interested (and none opposed):”
>

> And satisfying this is a requirement for adding features to WHATWG Living
> Standards.
>
> And the review block at the bottom says “Review required” and “Merging is
> Blocked with big red X’s.
>
Yes, see the last comment from the reviewer: "I guess the main things
remaining are implementer interest and someone verifying the test coverage.
(Happy to help with the latter once the former is sorted.) "
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

On Tue, Mar 3, 2020 at 10:36 AM Ryosuke Niwa  wrote:

>
> By the way, do you know what the status / interests at Mozilla? Given
> WebKit's painting / navigation behavior / implementation is still pretty
> close to Blink, it would be a good idea to reach out to Mozilla to make
> sure whatever in the spec is something they can also implement.
>
> Mozilla have a bug open on this:
https://bugzilla.mozilla.org/show_bug.cgi?id=1518999, but unfortunately it
is not prioritized.
Since they're mainly a desktop browser web developers currently don't
prioritize them for a performance metric. I hope that changes.
~Noam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

On Tue, Mar 3, 2020 at 10:18 AM Ryosuke Niwa  wrote:

> Sorry for the delay. I had other other things to take care of first.
>
> Based on the discussion we had (between Maciej, Simon, Alan, and I), we
> should take the following items into account for WebKit's first meaningful
> paint heuristics:
>
>- Background image
>
> I've filed https://bugs.webkit.org/show_bug.cgi?id=208501 and can get it
done.
Btw if there's something I'm taking on myself but Apple would rather do or
vice versa, please let me know :)

>
>- SVG images
>- "Contentful" canvas once we rigorously defined what it means:
>https://github.com/w3c/paint-timing/issues/50
>- First frame / poster image of a video:
>https://github.com/w3c/paint-timing/issues/51
>
> Then as Maciej pointed out there are a few spec works to do:
>
>- WebKit takes any text regardless of whether they appear within UA
>shadow root or not into account for the first meaningful paint. The spec
>needs to clarify this behavior -
>https://github.com/w3c/paint-timing/issues/52
>- The exact timing of navigation should be defined -
>https://github.com/w3c/paint-timing/issues/19
>- Clarify whether "first paint" or "first content paint" ever happens
>to a blank page - https://github.com/w3c/paint-timing/issues/53
>- Clarify what happens to a page which consists of just an iframe -
>https://github.com/w3c/paint-timing/issues/54
>- Combination of paint timing and long tasks can expose precise paint
>timing - https://github.com/w3c/paint-timing/issues/55
>
> To supplement earlier Maciej's points, per our discussion, we don't think
> "first paint" is a good metric to expose to the Web since Safari / WebKit
> users would never see that. If any website optimize for the first paint
> metrics instead of or at the cost of first contentful paint, then such an
> optimization would only result in perceived regressions for our users.
>
I've spoken with the Wikipedia folks on this and they agree, first-paint is
not really that useful as a performance metric (I think it's useful to
catch bugs, e.g. in cases where it's vastly behind first-contentful-paint).
For now I'm focusing only on the first-contentful-paint metric, and adding
web platform tests to cover this situation (the current tests would fail in
the case where FP is not implemented).
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak  wrote:

>
>
> On Nov 11, 2019, at 4:37 AM, Noam Rosenthal  wrote:
>
>
>
> On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak  wrote:
>
>> - Is there a specification for Content-DPR? All I could find in bugzila
>> and elsewhere is links to <
>> https://httpwg.org/http-extensions/client-hints.html>, which does not
>> mention Content-DPR at all).
>>
> In a nutshell, the spec is in transition from IETF to HTML/fetch, and the
> client-hints aspects of it are still in progress (unlike conent-dpr, which
> is much simpler hasn't changed since introduced). It's documented here:
> https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
> .
> I will answer more about this separately if required, waiting for some
> info from the people working on the spec originally.
>
>
> It looks like these are the relevant Pull Requests:
>
>
> HTML: https://github.com/whatwg/html/pull/3774 |
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density
> Fetch: https://github.com/whatwg/fetch/pull/773 |
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density
>
> It looks like these are in a somewhat messy state. For example, Fetch
> places the Content-DPR value in an “image density” variable, but it doesn’t
> look like the Pull Request to HTML doesn’t use it anywhere. As another
> example, HTML directly references Content-DPR as setting the “current pixel
> density”, but does not specify that it would take priority over a density
> derived from srcset. There are also no diffs to CSS Images or to SVG, which
> provide distinct ways to reference images and which presumably should
> respect Content-DPR.
>



>
>
>
>> - Can you give us some examples of how the CDN would make the decision of
>> what density of image to serve without knowing the client’s DPR via
>>  client-hints?
>>
>  Some examples from  https://github.com/eeeps/content-dpr-explainer:
> - making decisions by user agent, like choosing to cap certain images for
> user-agents known to have smaller displays
> - making decisions by traffic/geo, like serving lower-resolution images
> when traffic is bogged down
> - A client may ask for "placeholder image" (e.g. ?placeholder=true in the
> URL), and the CDN would decide whether to serve a lower-quality JPEG or to
> scale it down, or do both.
>
>
> These seem like reasonable use cases.
>
>
> - Is this presuming situations where the CDN serves the images but not the
>> markup or CSS (which presumably could be rewritten to account for DPR)?
>>
> Correct.
>
>
>> - What happens if Content-DPR is provided, but the markup or CSS has
>> conflicting explicit info? For example, > image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x.
>> Or the analogous thing with CSS.
>>
> When image size is decided, css/markup takes precedence, then content DPR,
> and finally srcset-selection DPR. This is actually a bonus value of this
> feature - if the server happens to serve an image that differs from the
> ratio requested in the srcset tag, the intrinsic size match the served
> content and not the request (which is more of a recommendation).
>
> - Does Content-DPR have any effect on images where an explicit size is
>> provided, either in markup or in CSS? (I’m assuming no.)
>>
> No, it only has effect on intrinsic size.
>
>
> Overall, this seems like a feature with valid use cases. But
> unfortunately, it’s a (currently) Blink-only feature with no clear
> specification. To the extent there is a spec, it’s mixed in with multiple
> Pull Requests. These pull requests mix in Client Hints, a feature unlikely
> to gain multi implementor support, so they probably won’t land any time
> soon. And the language about Content-DPR is broken and does not not seem to
> match Chrome’s actual implementation.
>
> So in summary, there is no proper spec, and Chrome has shown willingness
> to have their implementation change faster than even draft spec language in
> PRs can keep up with.
>

> Although the use case seems valid, I don’t think it’s a good idea for
> WebKit to implement the feature in this state. It seems likely to cause
> interop headaches and a need to reverse engineer Blink rather than
> developing against specs and test cases.
>
> There has been a lot of work done on the spec front since the comments
above, and I think we're ready for a re-review the state.
* The next pull request: https://github.com/whatwg/html/pull/5112 has been
thumbed up to make it into the HTML spec, and is separate from
client-hints, bu

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

On Mon, Mar 2, 2020 at 2:37 AM Maciej Stachowiak  wrote:

>
>
> On Mar 1, 2020, at 4:19 PM, Noam Rosenthal  wrote:
> OK, to summarize what I got from this
> - we want the spec and webkit painting to be as close as possible
> - The spec needs to be clearer/less buggy about a few things, such as
> "White" canvas, spec issues to be files
> - WebKit should be closer to the spec wrt canvas, backgrounds and
> potentially pixel/character threshold (TBD)
> - FP is more sensitive than FCP, because it exposes browser differences
> and may lead to unwanted comparisons and misunderstanding. Thus, it should
> be exposed as a different runtime feature flag.
>
> One thing I'm wondering about - would it be better to change the rendering
> heuristics together with implementing the paint API reporting? Or would it
> be better to separate those concerns a bit in terms of implementation? I
> mean, having the performance APIs in the code behind 2 flags with failed
> tests conforming to the spec might help iterate on the actual rendering.
> What would you consider a better approach here?
>
>
> If I were doing this myself, I’d first change rendering heuristics
> (probably without a flag) and then add the web-facing API.
>
Okay, let's see how that goes.


> What you describe might be reasonable if there are WPT conformance tests
> that can distinguish cases where the FCP timing is too late or too soon.
> It’s probably possible to make such tests, but I am not sure if they
> already exist.
>
They exist to some extent, webkit currently fails 4 out of 13 painting-API
conformance tests, around some of the issues discussed here (e.g. one of
the tests fails because webkit doesn't consider background image to be
contentful). I think I'll work on a few more while working on the spec,
they're quite easy to write.

Cheers
Noam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

On Mon, Mar 2, 2020 at 1:18 AM Maciej Stachowiak  wrote:

>
>
> On Mar 1, 2020, at 2:57 PM, Noam Rosenthal  wrote:
>
>
>
> On Mon, Mar 2, 2020 at 12:21 AM Maciej Stachowiak  wrote:
>
>>
>>
>> On Mar 1, 2020, at 2:07 PM, Noam Rosenthal  wrote:
>>
>>
>>> The first visually non-empty milestone almost always happens way before
>>> this point. The above is just a fallback to make sure we eventually hit
>>> this milestone. For example, if a document is totally blank even after
>>> loading the document and all subresources, we want to paint it instead of
>>> waiting forever.
>>>
>>> The visually non-empty heuristic is elsewhere.
>>>
>>> Note that WebKit would consider the paint triggered by the above
>>> fallback code to be both a first paint and “first visually non-empty paint”
>>> or “first meaningful paint”, which somewhat corresponds to Chrome’s notion
>>> of “first contentful paint”.
>>>
>>> First paint can only happen earlier under even more unusual
>>> circumstances, I believe there is a timeout after which we will paint even
>>> if all we can paint is blank white or background color.
>>>
>>>
>>> But let's take the specifics to Slack/bugzilla?
>>>
>>>
>>> I think it would be good for you to talk to people who understand
>>> WebKit’s layout/paint milestones in detail before taking things to
>>> bugzilla. Ask on Slack, and I’ll point you to the right people.
>>>
>> Oops, just saw this, an initial (not for review) patch is already in
>> bugzilla :)
>> But I'll continue the discussion - I have better idea of what to ask now.
>> Who would be the right people to ask?
>>
>>
>> Alan, Simon, and Ryosuke should all know about this.
>>
> Awesomeness.
>
>
>>
>> A few of us sat down and reviewed this spec. We think that, as written,
>> the definition of “first contentful paint” is underspecified and in some
>> cases buggy. For example, it says a “non-white canvas” should count as
>> contentful. But canvases don’t start as white, they start as fully
>> transparent black (all zeros). And checking the color of every pixel in a
>> canvas is expensive. The intent here is probably that a canvas that has
>> ever been painted into is contentful (or one that has been painted into
>> more recently than the last time it was cleared).
>>
> Yes, I also thought that part of the spec was strange. I'll help revise it.
> btw the FirstMeaningfulPaint in webkit doesn't look at canvas content at
> all - rather on the creation of a RenderCanvasElement... maybe being closer
> to the spec here would be better rendering-wise? Also, the current layout
> milestones doen't consider background images as "rendered pixels". Is that
> on purpose?
>
>
> We think background images (and SVG, if not included yet) should count as
> meaningful content. And canvases should probably be limited to non-empty
> ones.
>
OK - that's more like the spec.


>
>
>
>>
>> In any case, it definitely does not match the WebKit notion of first
>> visually non-empty layout / first meaningful paint, in some cases in ways
>> that we regret.
>>
> Regret, as in - it was better if the spec was closer to what webkit was
> doing?
>
>
> No, I mean in some cases we think we should do what the spec says.
>
> I think it's OK if the spec's FCP and webkit's FMP are not the same, and
> if FCP is fired according to spec, and after the actual painting in webkit
> was done.
>
>
> I think we should try to align with the spec. Otherwise, because WebKit
> usually won’t paint at all until webkit-FMP time, FCP won’t fire until that
> time (since it is only fired when an actual paint happens).
>


>
>
>> Spec issues to follow.
>>
>> We also think exposing “first paint” is harmful and we’d rather not
>> implement it. We don’t agree that painting non-contentful content quickly
>> is a good idea, either for browsers or for websites. And this API will
>> inevitably be used to compare browsers, regardless of disclaimers that it
>> shouldn’t be.
>>
> Harmful in the sense of comparing browsers? I don't think it can,
> regardless of disclaimers - since the hardware used for the browsers (at
> least on mobile) is vastly different, and also the networks. How about
> exposing a prefixed entry? e.g. "-webkit-first-paint" - something that
> doesn't try to seem comparable? The goal here is to help people optimize
> their site and make sure it doesn't regress on webkit, rather than create
> meaningl

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

On Mon, Mar 2, 2020 at 12:21 AM Maciej Stachowiak  wrote:

>
>
> On Mar 1, 2020, at 2:07 PM, Noam Rosenthal  wrote:
>
>
>> The first visually non-empty milestone almost always happens way before
>> this point. The above is just a fallback to make sure we eventually hit
>> this milestone. For example, if a document is totally blank even after
>> loading the document and all subresources, we want to paint it instead of
>> waiting forever.
>>
>> The visually non-empty heuristic is elsewhere.
>>
>> Note that WebKit would consider the paint triggered by the above fallback
>> code to be both a first paint and “first visually non-empty paint” or
>> “first meaningful paint”, which somewhat corresponds to Chrome’s notion of
>> “first contentful paint”.
>>
>> First paint can only happen earlier under even more unusual
>> circumstances, I believe there is a timeout after which we will paint even
>> if all we can paint is blank white or background color.
>>
>>
>> But let's take the specifics to Slack/bugzilla?
>>
>>
>> I think it would be good for you to talk to people who understand
>> WebKit’s layout/paint milestones in detail before taking things to
>> bugzilla. Ask on Slack, and I’ll point you to the right people.
>>
> Oops, just saw this, an initial (not for review) patch is already in
> bugzilla :)
> But I'll continue the discussion - I have better idea of what to ask now.
> Who would be the right people to ask?
>
>
> Alan, Simon, and Ryosuke should all know about this.
>
Awesomeness.


>
> A few of us sat down and reviewed this spec. We think that, as written,
> the definition of “first contentful paint” is underspecified and in some
> cases buggy. For example, it says a “non-white canvas” should count as
> contentful. But canvases don’t start as white, they start as fully
> transparent black (all zeros). And checking the color of every pixel in a
> canvas is expensive. The intent here is probably that a canvas that has
> ever been painted into is contentful (or one that has been painted into
> more recently than the last time it was cleared).
>
Yes, I also thought that part of the spec was strange. I'll help revise it.
btw the FirstMeaningfulPaint in webkit doesn't look at canvas content at
all - rather on the creation of a RenderCanvasElement... maybe being closer
to the spec here would be better rendering-wise? Also, the current layout
milestones doen't consider background images as "rendered pixels". Is that
on purpose?


>
> In any case, it definitely does not match the WebKit notion of first
> visually non-empty layout / first meaningful paint, in some cases in ways
> that we regret.
>
Regret, as in - it was better if the spec was closer to what webkit was
doing?
I think it's OK if the spec's FCP and webkit's FMP are not the same, and if
FCP is fired according to spec, and after the actual painting in webkit was
done.

>
> Spec issues to follow.
>
> We also think exposing “first paint” is harmful and we’d rather not
> implement it. We don’t agree that painting non-contentful content quickly
> is a good idea, either for browsers or for websites. And this API will
> inevitably be used to compare browsers, regardless of disclaimers that it
> shouldn’t be.
>
Harmful in the sense of comparing browsers? I don't think it can,
regardless of disclaimers - since the hardware used for the browsers (at
least on mobile) is vastly different, and also the networks. How about
exposing a prefixed entry? e.g. "-webkit-first-paint" - something that
doesn't try to seem comparable? The goal here is to help people optimize
their site and make sure it doesn't regress on webkit, rather than create
meaningless cross-browser comparisons...

In any case even having first-contentful-paint would be useful - with or
without first-paint.


> It’s probably unwise to implement this until the spec is fixed. And once
> it is, we should change our “first visually non-empty layout” notion to
> match it before exposing the corresponding paint timing milestone.
>
Not sure that's necessary. We can match the spec without changing the way
we render. For example, “first visually non-empty layout" needs a minimum
amount of pixels/characters - do we want to change the spec to have this
heuristic, or change that heuristic in webkit, or neither? I'm actually
fine with any of these (speaking for what Wikipedia and other web-devs
would want), as long as we give web-devs some way to measure something that
they can count on to a degree :)


>
> I think Ryosuke took notes on the spec issues we need to file.
>
Ryusoke, will you share those notes with me? I can change the spec myself
(I'm on the web perf group now).

>
> Regards,
> Maciej
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

>
>
> The first visually non-empty milestone almost always happens way before
> this point. The above is just a fallback to make sure we eventually hit
> this milestone. For example, if a document is totally blank even after
> loading the document and all subresources, we want to paint it instead of
> waiting forever.
>
> The visually non-empty heuristic is elsewhere.
>
> Note that WebKit would consider the paint triggered by the above fallback
> code to be both a first paint and “first visually non-empty paint” or
> “first meaningful paint”, which somewhat corresponds to Chrome’s notion of
> “first contentful paint”.
>
> First paint can only happen earlier under even more unusual circumstances,
> I believe there is a timeout after which we will paint even if all we can
> paint is blank white or background color.
>
>
> But let's take the specifics to Slack/bugzilla?
>
>
> I think it would be good for you to talk to people who understand WebKit’s
> layout/paint milestones in detail before taking things to bugzilla. Ask on
> Slack, and I’ll point you to the right people.
>
Oops, just saw this, an initial (not for review) patch is already in
bugzilla :)
But I'll continue the discussion - I have better idea of what to ask now.
Who would be the right people to ask?
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

On Thu, Feb 27, 2020 at 12:46 PM Noam Rosenthal  wrote:

>
>
> On Thu, Feb 27, 2020 at 12:17 PM Yoav Weiss  wrote:
>
>>
>>
>> On Wed, Feb 26, 2020 at 11:33 PM Ryosuke Niwa  wrote:
>>
>>>
>>> I don't think we should do that. For starters, Chrome's painting
>>> strategy while loading a web page is very different from that of Safari /
>>> WebKit. We would freeze the painting of the previous page at the moment a
>>> new navigation is committed, and we wouldn't update the painting until the
>>> destination page has a meaningful content in it. This is a very much
>>> different from Chrome's model where the moment a new navigation is
>>> committed, Chrome will show a blank page then start incrementally painting
>>> the page throughout the navigation.
>>>
>> Body background color is still painted before any contentful paint... Is
> this a bug?
> Also, a web page might not have content at all (e.g. a bunch of divs with
> bgcolors). Would webkit not render that web page at all?
>

It seems like the code in FrameView does this:

// Ensure that we always fire visually non-empty milestone eventually.

*if* (finishedParsingMainDocument && frame().loader().isComplete())

*return* *true*;

I suggest splitting this to two milestones - the current one, that triggers
when the main document finished loading, and another one that only triggers
when content has actually been painted (which may never happen). I think
this would be a good split for first-paint/first-contentful-paint in WebKit.

But let's take the specifics to Slack/bugzilla?
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

On Thu, Feb 27, 2020 at 12:17 PM Yoav Weiss  wrote:

>
>
> On Wed, Feb 26, 2020 at 11:33 PM Ryosuke Niwa  wrote:
>
>>
>> I don't think we should do that. For starters, Chrome's painting strategy
>> while loading a web page is very different from that of Safari / WebKit. We
>> would freeze the painting of the previous page at the moment a new
>> navigation is committed, and we wouldn't update the painting until the
>> destination page has a meaningful content in it. This is a very much
>> different from Chrome's model where the moment a new navigation is
>> committed, Chrome will show a blank page then start incrementally painting
>> the page throughout the navigation.
>>
> Body background color is still painted before any contentful paint... Is
this a bug?
Also, a web page might not have content at all (e.g. a bunch of divs with
bgcolors). Would webkit not render that web page at all?

>
>> Second off, the point of specification is to allow multiple independent
>> implementations. If we had to reverse-engineer what Chrome is doing and
>> implement that, it defeats the point of having any standard at all.
>>
>
> With my WebPerfWG hat on, I agree. Would be good to find a model that
> works well for different implementations (even if not comparable between
> different implementations), while providing points in time for developers
> that can: a) provide a user meaningful visual metric and b) enable spotting
> regressions.
>
Note taken. Though this is hypothetical - let's get to the point where the
spec is lacking, and then we can decide how to go about it.
What I propose is to go as far as possible with the spec, and when we reach
an ambiguity tackle it on the spec front.

>
> Would WebKit folks be interested in helping exploration on that front?
>
I would be happy to help coordinate this (e.g. work on the details of pec
differences and prod Ryosuke and folks for feedback about internals).


>
>> I don't think we don't should that because we don't have an equivalent of
>> first-paint.
>>
>
> FWIW, I don't think it's a huge problem if WebKit will report FP and FCP
> as the same timestamp, as they are indeed the same point in time.
>
A simple test shows that safari does show the background color
(body=bgcolor) when a timeout is set to render everything else (e.g. add
text to the document). According to the spec, the first one should give
first-paint and the second one would be first-contentful-paint.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

On Wed, Feb 26, 2020 at 9:29 PM Maciej Stachowiak  wrote:

> I also suggest to start with "first-paint" as it's perhaps a bit less
> "internal" than FCP, and can provide a performance-regression metric with a
> lesser degree of risk regarding exposing internals / privacy.
>
>
> First paint that’s not first meaningful/contentful paint is not a very
> good performance metric IMO. Who cares that a paint happened if it doesn’t
> have any image or text content?
>
I think it's meaningful as a general regression metric, albeit less useful
than FCP. Web developers are scrambling for any RUM information they can
use...


> I also don’t think this exposes less. The privacy risk here is exposing
> timing data that might be usable for fingerprinting.
>
Yes, I think that privacy-relevant timing information is already available
through resource timing (window.performance.getEntries("resource")). But
we'll see what the privacy experts come up with.


> What was deprecated was "first meaningful paint" (FMP). FCP was not
> deprecated and has been in wide use for some time.
>
> What’s the difference between First Meaningful and First Contentful?
>
FMP had the notion of "primary" content; e.g. it was ignoring
navbars/headers, and trying to figure out when the main content of the site
has been rendered.  As noted, FMP is deprecated. FCP is a lot more
transparent in what it measures.

We will discuss with Apple’s privacy experts what they think of the privacy
> risk. I’m just giving you a rain check for results of this discussion.
>
Got it, Looking forward :)
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

(resending from correct address)
On Wed, Feb 26, 2020 at 8:08 PM Maciej Stachowiak  wrote:

>
> Some quick comments:
>

> the definition of First Contentful Paint here in the spec: <
> https://www.w3.org/TR/paint-timing/#sec-terminology> does not match the
> definition stated at . The
> Chrome definition on web.dev specifies that iframe content is not
> included, the spec does not have this limitation. Would an implementation
> that matches the spec match Chrome?
>
The draft version of the spec specifies that iframe content is not included
in FCP:  https://w3c.github.io/paint-timing/#sec-reporting-paint-timing,
and has a few more comprehensive details about this. I think it's a good
place to start.

I am also not sure this matches the layout milestones that already exist in
> non-Blink browser engines. Has this spec been implemented in Gecko, for
> example, to verity that it’s not exposing a concept that only exists in
> Blink?
>
No, this has not been implemented in Gecko, I'm tracking the bug on this:
https://bugzilla.mozilla.org/show_bug.cgi?id=1518999, there was some
movement recently.

I suggest to start from "first-paint", and to try to match chrome as much
as possible in how FCP is implemented, in the cases where the spec doesn't
give enough detail, if such places exist. I think that for the main
use-case of catching regressions for website code, it's ok (and almost
unpreventable) if the implementations have some variances between them,
what matters is that the metric is reliable for the particular browser.
I also suggest to start with "first-paint" as it's perhaps a bit less
"internal" than FCP, and can provide a performance-regression metric with a
lesser degree of risk regarding exposing internals / privacy.


>
> Chrome team themselves have been telling web developers that First
> Contentful Paint is deprecated in favor of Largest Contentful Paint. Should
> we concerned about this? It seems even harder to define LCP in an
> engine-independent way.
>
What was deprecated was "first meaningful paint" (FMP). FCP was not
deprecated and has been in wide use for some time.


>
> Finally, we should do a privacy review to consider whether exposing this
> info to webpages creates fingerprinting risk or otherwise exposes user data.
>
Great, what is needed for such review?
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)

[Noam Rosenthal]

Hola
I was approached by the Wikimedia foundation to implement the paint timing
API for WebKit (yay).
I think this is a good feature to have for webkit, and I wanted to hear
thoughts about it before I begin.

The feature was enabled in Chrome for quite a while, and is potentially
very useful for real user monitoring to see if changes to the code of the
website create performance regressions on different browsers.
We've been using it extensively (on Chrome) when I worked at Wix.com, and
always hoped that "some day" it will be available in other browsers.

I think it's a pretty lean spec and was stable since 2017, and that it fits
with the WebKit project goals, by allowing web developers to better
optimize their content and catch Safari/WebKit-specific regressions early.

I would like to get feedback on whether it's a desired feature for WebKit,
if there are caveats/pitfalls I should be thinking about, and anything else
that's worth mentioning at this point.

There are of course delicate decisions to be made as to the choice of
timing indicator, but I will get to that once we've sorted out the position
in general.

Bug: https://bugs.webkit.org/show_bug.cgi?id=78011
Spec: https://www.w3.org/TR/paint-timing/

Thanks!
Noam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak  wrote:

>
>
> On Nov 11, 2019, at 4:37 AM, Noam Rosenthal  wrote:
>
>
>
> On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak  wrote:
>
>> - Is there a specification for Content-DPR? All I could find in bugzila
>> and elsewhere is links to <
>> https://httpwg.org/http-extensions/client-hints.html>, which does not
>> mention Content-DPR at all).
>>
> In a nutshell, the spec is in transition from IETF to HTML/fetch, and the
> client-hints aspects of it are still in progress (unlike conent-dpr, which
> is much simpler hasn't changed since introduced). It's documented here:
> https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
> .
> I will answer more about this separately if required, waiting for some
> info from the people working on the spec originally.
>
>
> It looks like these are the relevant Pull Requests:
>
>
> HTML: https://github.com/whatwg/html/pull/3774 |
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density
> Fetch: https://github.com/whatwg/fetch/pull/773 |
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density
>
> It looks like these are in a somewhat messy state. For example, Fetch
> places the Content-DPR value in an “image density” variable, but it doesn’t
> look like the Pull Request to HTML doesn’t use it anywhere. As another
> example, HTML directly references Content-DPR as setting the “current pixel
> density”, but does not specify that it would take priority over a density
> derived from srcset. There are also no diffs to CSS Images or to SVG, which
> provide distinct ways to reference images and which presumably should
> respect Content-DPR.
>
>
>
>> - Can you give us some examples of how the CDN would make the decision of
>> what density of image to serve without knowing the client’s DPR via
>>  client-hints?
>>
>  Some examples from  https://github.com/eeeps/content-dpr-explainer:
> - making decisions by user agent, like choosing to cap certain images for
> user-agents known to have smaller displays
> - making decisions by traffic/geo, like serving lower-resolution images
> when traffic is bogged down
> - A client may ask for "placeholder image" (e.g. ?placeholder=true in the
> URL), and the CDN would decide whether to serve a lower-quality JPEG or to
> scale it down, or do both.
>
>
> These seem like reasonable use cases.
>
>
> - Is this presuming situations where the CDN serves the images but not the
>> markup or CSS (which presumably could be rewritten to account for DPR)?
>>
> Correct.
>
>
>> - What happens if Content-DPR is provided, but the markup or CSS has
>> conflicting explicit info? For example, > image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x.
>> Or the analogous thing with CSS.
>>
> When image size is decided, css/markup takes precedence, then content DPR,
> and finally srcset-selection DPR. This is actually a bonus value of this
> feature - if the server happens to serve an image that differs from the
> ratio requested in the srcset tag, the intrinsic size match the served
> content and not the request (which is more of a recommendation).
>
> - Does Content-DPR have any effect on images where an explicit size is
>> provided, either in markup or in CSS? (I’m assuming no.)
>>
> No, it only has effect on intrinsic size.
>
>
> Overall, this seems like a feature with valid use cases. But
> unfortunately, it’s a (currently) Blink-only feature with no clear
> specification. To the extent there is a spec, it’s mixed in with multiple
> Pull Requests. These pull requests mix in Client Hints, a feature unlikely
> to gain multi implementor support, so they probably won’t land any time
> soon. And the language about Content-DPR is broken and does not not seem to
> match Chrome’s actual implementation.
>
> So in summary, there is no proper spec, and Chrome has shown willingness
> to have their implementation change faster than even draft spec language in
> PRs can keep up with.
>
> Although the use case seems valid, I don’t think it’s a good idea for
> WebKit to implement the feature in this state. It seems likely to cause
> interop headaches and a need to reverse engineer Blink rather than
> developing against specs and test cases.
>
> I would like to see a clean spec, either separating Content-DPR PRs from
> client-hints or making a standalone spec. And either way it should be fixed
> to match what Chrome has implemented, and have WPT tests that reflect the
> spec and intended behavior.
>

I hear you. Let me get back to this thread once we're cleaner on the spec
front.
Thank you for the detailed response, it gives a clear path forward.
~Noam

>
> Regards,
> Maciej
>
>
>
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Tue, Nov 12, 2019 at 3:28 AM Alexey Proskuryakov  wrote:

>
>
> 10 нояб. 2019 г., в 1:16 AM, Noam Rosenthal  написал(а):
>
> Also, as a response to the view that this should be part of markup, I
> would argue that those transparent decisions by the servers should not
> concern web authors at all. It’s an optimization decision to be handled by
> the servers, with the user agent doing nothing about it but allow that
> decision to be transparent in terms of layout (the same way gzip and
> cache-control are transparent).
>
>
> I don't see this as a precedent, because cache control and compression are
> invisible to users.
>
Agreed, I was showing it as an example that HTTP headers are a valid place
to add instructions to/from transient layers of the internet (CDN/proxy).
Though, for example, the good old "Accept" header may result in receiving a
different image encoding for the same URL, resulting in different qualities
for the same URL.


> Whereas image quality most certainly is. Changing it behind both web
> developer and user back would cause lots of undesirable behaviors - say, I
> e-mail an image from a webpage to someone else, who doesn't have a small
> display, or just zoom in to see the details.
>
This basically results in the website being untestable by the author (or by
> UA implementers who will be getting bug reports about site behavior
> differences).
>

This can already happen, a URL for an image doesn't guarantee always
receiving the same resource. Content-DPR doesn't change that. On the
contrary, by providing Content-DPR we allow such optimizations (which
already take place) to avoid breaking layouts that are based on intrinsic
image sizes, which is the main way websites can break due to serving
different images to the same image request URL. The web today is based on
this premise - the same URL may lead to different resources in different
situations.

Also, we don't change it behind the web developer's back - who maybe does
that is the developer's chosen CDN, and it's a legitimate
experience/performance tradeoff that can be negotiated between them. What
we can do as a browser engine, is do the minimum to prevent that behavior
from breaking layouts.

>

> Also, maybe I'm just pessimistic here, but it seems almost inevitable that
> disagreements between markup, Content-DPR and dpi information embedded in
> images will be handled by UAs differently, even if the spec ends up being
> very detailed on this point.
>
I'm less pessimistic about that. Content-DPR is quite clear - it's a
description of the content, like an encoding

~Noam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak  wrote:

> - Is there a specification for Content-DPR? All I could find in bugzila
> and elsewhere is links to <
> https://httpwg.org/http-extensions/client-hints.html>, which does not
> mention Content-DPR at all).
>
In a nutshell, the spec is in transition from IETF to HTML/fetch, and the
client-hints aspects of it are still in progress (unlike conent-dpr, which
is much simpler hasn't changed since introduced). It's documented here:
https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
.
I will answer more about this separately if required, waiting for some info
from the people working on the spec originally.


> - Can you give us some examples of how the CDN would make the decision of
> what density of image to serve without knowing the client’s DPR via
>  client-hints?
>
 Some examples from  https://github.com/eeeps/content-dpr-explainer:
- making decisions by user agent, like choosing to cap certain images for
user-agents known to have smaller displays
- making decisions by traffic/geo, like serving lower-resolution images
when traffic is bogged down
- A client may ask for "placeholder image" (e.g. ?placeholder=true in the
URL), and the CDN would decide whether to serve a lower-quality JPEG or to
scale it down, or do both.

- Is this presuming situations where the CDN serves the images but not the
> markup or CSS (which presumably could be rewritten to account for DPR)?
>
Correct.


> - What happens if Content-DPR is provided, but the markup or CSS has
> conflicting explicit info? For example,  image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x.
> Or the analogous thing with CSS.
>
When image size is decided, css/markup takes precedence, then content DPR,
and finally srcset-selection DPR. This is actually a bonus value of this
feature - if the server happens to serve an image that differs from the
ratio requested in the srcset tag, the intrinsic size match the served
content and not the request (which is more of a recommendation).

- Does Content-DPR have any effect on images where an explicit size is
> provided, either in markup or in CSS? (I’m assuming no.)
>
No, it only has effect on intrinsic size.

~Noam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Sun, Nov 10, 2019 at 9:41 PM Maciej Stachowiak  wrote:

>
> Is this header useful without the DPR client-hint?
>
Absolutely.
A server/CDN can decide to serve an image with a particular content-dpr
without knowing the client's actual DPR.


> On Nov 10, 2019, at 1:16 AM, Noam Rosenthal  wrote:
>
> Hola
>
> I would like to open a discussion that has started a few years back and
> has never reached a consensus:
> https://bugs.webkit.org/show_bug.cgi?id=145380
>
> In a nutshell: content-dpr is a response-only header for images, that
> allows servers at transient layers (CDNs/proxies) to optimize images by
> changing their resolution, allowing the user agents to compensate for the
> change in intrinsic size caused by the resolution change - thus making the
> resolution change transparent to users. It's the header that makes the
> resolution-encoding transparent.
>
> The feature is already up and running in chrome since version 67, and is
> used by some CDNs.
>
> Since there was lack of consensus in the bug discussion, I wanted to make
> the case for it here, and see if opinions about the subject can be voiced
> before we reach a decision/consensus. I tried to find a good balance
> between being detailed and being concise.
>
> —
>
> Players in CDN, proxy and other transient parts of the internet world have
> innovated a lot in the past few years. There are lots of interesting
> companies and competition there. One of the areas of this innovation is in
> optimizing images in a transparent way at transient layers (proxy/CDN).
> This makes a lot of sense considering how much of internet traffic is taken
> by image download.
>
> What the research at the CDN companies found, was that modifying
> resolution at the transient layer can have a tremendous effect on
> performance/user-experience balance, for example by serving
> lower-resolution versions of the images when network traffic is costly/slow
> (the exact formula for that is part of where the CDN can innovate). More
> details on that innovation in the links below.
>
> The thing is, that modifying image resolution at the CDN/proxy is not
> inherently transparent, due to one reason - layout is affected by the
> image’s intrinsic size, and changing the image’s resolution inadvertently
> changes the image’s intrinsic size. To make this transparent, the user
> agent has to be involved, to compensate for this optimization when reading
> the image’s intrinsic size.
>
> The main case against this feature was that image resolution is a feature
> that should be handled at the markup layer and not at the transport layer (
> https://bugs.webkit.org/show_bug.cgi?id=145380#c7).
> I would argue that http-headers are not a transport layer (TCP/UDP etc.),
> but rather part of an application-layer protocol that is designed, in part,
> to pass information to the transient layers such as CDNs, and that
> resolution compression is a (new, image-specific) equivalent of
> transfer-encoding.
>
> Also, as a response to the view that this should be part of markup, I
> would argue that those transparent decisions by the servers should not
> concern web authors at all. It’s an optimization decision to be handled by
> the servers, with the user agent doing nothing about it but allow that
> decision to be transparent in terms of layout (the same way gzip and
> cache-control are transparent).
>
> What is offered here is a win-win in terms of delivering images, and would
> allow webkit-browser users to enjoy some of the innovations in the image
> delivery world without modifying their HTML content and without concern of
> breaking their layouts. Less battery and data used for downloading big
> images at certain situations, for example.
>
> I hope this provides enough background without being too tedious. I would
> be happy to shed more light on whatever is missing in this longish essay :)
>
> Additional info:
> https://github.com/eeeps/content-dpr-explainer
>
> https://cloudinary.com/blog/client_hints_and_responsive_images_what_changed_in_chrome_67
> https://www.chromestatus.com/metrics/feature/timeline/popularity/906
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

Hola


I would like to open a discussion that has started a few years back and has
never reached a consensus: https://bugs.webkit.org/show_bug.cgi?id=145380


In a nutshell: content-dpr is a response-only header for images, that
allows servers at transient layers (CDNs/proxies) to optimize images by
changing their resolution, allowing the user agents to compensate for the
change in intrinsic size caused by the resolution change - thus making the
resolution change transparent to users. It's the header that makes the
resolution-encoding transparent.


The feature is already up and running in chrome since version 67, and is
used by some CDNs.


Since there was lack of consensus in the bug discussion, I wanted to make
the case for it here, and see if opinions about the subject can be voiced
before we reach a decision/consensus. I tried to find a good balance
between being detailed and being concise.


—


Players in CDN, proxy and other transient parts of the internet world have
innovated a lot in the past few years. There are lots of interesting
companies and competition there. One of the areas of this innovation is in
optimizing images in a transparent way at transient layers (proxy/CDN).
This makes a lot of sense considering how much of internet traffic is taken
by image download.


What the research at the CDN companies found, was that modifying resolution
at the transient layer can have a tremendous effect on
performance/user-experience balance, for example by serving
lower-resolution versions of the images when network traffic is costly/slow
(the exact formula for that is part of where the CDN can innovate). More
details on that innovation in the links below.


The thing is, that modifying image resolution at the CDN/proxy is not
inherently transparent, due to one reason - layout is affected by the
image’s intrinsic size, and changing the image’s resolution inadvertently
changes the image’s intrinsic size. To make this transparent, the user
agent has to be involved, to compensate for this optimization when reading
the image’s intrinsic size.


The main case against this feature was that image resolution is a feature
that should be handled at the markup layer and not at the transport layer (
https://bugs.webkit.org/show_bug.cgi?id=145380#c7).

I would argue that http-headers are not a transport layer (TCP/UDP etc.),
but rather part of an application-layer protocol that is designed, in part,
to pass information to the transient layers such as CDNs, and that
resolution compression is a (new, image-specific) equivalent of
transfer-encoding.


Also, as a response to the view that this should be part of markup, I would
argue that those transparent decisions by the servers should not concern
web authors at all. It’s an optimization decision to be handled by the
servers, with the user agent doing nothing about it but allow that decision
to be transparent in terms of layout (the same way gzip and cache-control
are transparent).


What is offered here is a win-win in terms of delivering images, and would
allow webkit-browser users to enjoy some of the innovations in the image
delivery world without modifying their HTML content and without concern of
breaking their layouts. Less battery and data used for downloading big
images at certain situations, for example.


I hope this provides enough background without being too tedious. I would
be happy to shed more light on whatever is missing in this longish essay :)


Additional info:

https://github.com/eeeps/content-dpr-explainer

https://cloudinary.com/blog/client_hints_and_responsive_images_what_changed_in_chrome_67

https://www.chromestatus.com/metrics/feature/timeline/popularity/906
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Help regarding rendering in webkit2 port of haiku

[Noam Rosenthal]

On Mon, Jun 10, 2019 at 10:33 AM Adrien Destugues 
wrote:

> On Mon, Jun 10, 2019 at 09:37:06AM +0300, Noam Rosenthal wrote:
> > ~ (reposting from webkit email)
>
> Hi,
>
> Another limitation we hit on some websites is that it is not possible
> for us to create too many drawable offscreen bitmaps. There is a
> system-wide limit of 4096, which can be hit rather easily on Google
> Maps, for example. So we will probably need a way to limit the use of
> drawable bitmaps when possible. Maybe coordinated graphics would help
> here, if it allows to share a single drawing context and bitmap, rather
> than creating a separate one for each rendering layer.
>

I would consider using CoordinatedGraphics with something like a display
list, where the drawing commands are sent via webkit2 to the UI process and
then everything is painted in the UI process in one go. This would be the
most memory-stingy approach, though it would mean zero acceleration and
would cancel out any benefit of compositing - compositing uses memory to
reduce drawing commands but when memory is what's expensive there's no
point in that... you would need to play with this to get the best
memory/cpu cost/benefit.


>
> On the way drawing works for us: the drawing primitives are all
> implemented in our display server, there is no rendering done client
> side in webkit processes. This means the display server has a pointer to
> the bitmap data already, and ideally it would be able to pass this from
> one application to another without needing to copy all the data. We may
> need to make some changes to our display server to allow this, but
> that's fine (it's the nice thing about Haiku, if we need to change
> something in the OS, we can do that easily).
>
> Now we have to find out if coordinated graphics or some other way to do
> things makes sense, and for this we have to understand how the rendering
> work is split between web process and ui process, and what kind of data
> they expect to exchange between them. Especially in our case where the
> bitmap data is in fact already shared with the OS display server, so if
> UI process does not do any drawing by itself, it doesn't actually need
> to access the data.
>
> --
> Adrien.
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Help regarding rendering in webkit2 port of haiku

[Noam Rosenthal]

~ (reposting from webkit email)

Hello!
I suggest starting by learning one of the other ports - like GTK - how the
whole flow works from the windowing system to WebCore to the graphics
system. Read the code, run it in debug mode and put breakpoints, get those
aha moments of how it all works together.

Note that coordinated graphics is not automatically the right choice for
every port - it was created for mobile windowing systems where window
context-switching was expensive and single-context hardware acceleration
was cheap. If you're looking for a guide check out
https://trac.webkit.org/wiki/CoordinatedGraphicsSystem, hopefully it's
still somewhat valid. I would start there by understanding the flow and how
it fits with the Haiku architecture, seems like you've started doing that
already. you're on the right track there.

btw I might be able to help out (no promises), I haven't touched WebKit
code in a few years but I was responsible for TextureMapper and the
coordinated graphics system and I like bizarre projects (a Haiku port
sounds like one). Let's see how it goes :)


On Mon, Jun 10, 2019 at 9:05 AM Rajagopalan Gangadharan <
g.raju2...@gmail.com> wrote:

> Hello everybody,
>
> A quick look back on what we have done so far. We are trying to port
> webkit2 to haiku and we have a working IPC with us. So the next step i
> presume is to be rendering.
>
> I did few amount of research and have few basic ideas. Correct me if I am
> wrong.
>
> 1) The rendering takes place in webprocess over a shared bitmap which is
> then rendered onto the client area in UIProcess.
>
> 2) Webkit uses coordinated graphics system which relies on OPENGL.
>
> So we look forward to get the rendering step done or is there anything we
> should take care in between? .
> Our platform doesnt have hardware accelerated OpenGL yet we have software
> implementation vesa which we tend not to use.
> We have a working texture mapper and compositing coordinator from our
> webkit legacy port. I think this makes up the entire compositing
> coordinator.
>
> So it would be of great help if any tells how could we proceed like what
> should be our next move. Perhaps a small guide would be better.
>
> Thank you
>
> Regards
> G.Rajagopalan
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] The SrcN responsive images proposal

[Noam Rosenthal]

  Designing this proposal around code formatting is a non-issue in my
 opinion
  and it surely didn't stop SVG from providing just one d attribute for
  path. Following the your logic, it should be d-N. Sure, path d=… is
  primarily meant to be written by software.

 Please don't try to use reducto ad absurdum; it usually gives absurd
 results.  The reasoning for multiple attributes is not because it's a
 list, it's because it's a list of lists, and would require three
 different delimiters.


The style attribute is perhaps a better example, as it is in some cases a
list of lists, e.g.
div style=transform: rotate() translate(); background: red
url(foobar.png) /

A list of lists in markup is usually expressed as either sub-elements or as
non-markup syntax inside an attribute; Enumerated attributes are highly
unusual.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] The SrcN responsive images proposal

[Noam Rosenthal]

On Fri, Nov 8, 2013 at 12:50 AM, Timothy Hatcher timo...@apple.com wrote:

 On Nov 7, 2013, at 2:43 PM, Tab Atkins Jr. jackalm...@gmail.com wrote:

 I'm pretty sure the argument is almost entirely around hand-authoring,
 where experience with style='' shows that lists-of-lists are bad.


 Until you are dealing with dozens of images per tag. That is untenable one
 image heavy sites and tools will need to be written to auto generate these
 lists of lists from the original assets. Scripts can easily deal with a
 micro format in one attribute. They can't easily deal with a micro format
 spread across multiple src-n attributes in arbitrary HTML.\


If we start from the one-attribute approach, we can improve the situation
in a simiilar manner to how style is attached to an element.
For example:
head
srcset name=foobar
some list-of-lists-of-lists-of-lists
/srcset
/head
body
img srcset=ref(#foobar) /
or even
   img src=srcset(foobar) /
/body

We can even do this kind of thing after the basic img
srcset=all-kinds-of-stuff / is supported, solving the feature problem
first and the authoring convenience problem second.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] PSA: WebKit2 and DrawingAreaImpl

[Noam Rosenthal]

Thanks for the heads up, Anders
I've created a bug for this, https://bugs.webkit.org/show_bug.cgi?id=122207


On Wed, Oct 2, 2013 at 3:19 AM, Anders Carlsson ander...@apple.com wrote:

 Hello,

 I just wanted to let everyone know that we (Apple) are moving away from
 DrawingAreaImpl and always using our tiled drawing area. Longer term we’d
 like to remove DrawingAreaImpl completely since it was designed back when
 we didn’t run in accelerated compositing mode all the time.

 It’s my understanding that all the other non-mac ports use coordinated
 graphics now, and so the logical step would be to create a special
 DrawingArea subclass for coordinated graphics and move the relevant code
 there. That’d allow us to remove DrawingAreaImpl and LayerTreeHost.

 I’d be more than happy to review any patches to do this, and answer any
 questions.

 Thanks,
 - Anders



 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Current status of threaded compositor for WebKit Gtk

[Noam Rosenthal]

Replying from right address this time...


On Mon, Apr 15, 2013 at 9:23 AM, Noam Rosenthal n...@webkit.org wrote:

 Thanks Gwang-Yoon
 Yes, I would like to get rid of TextureMapperImageBuffer, and we can do
 that once Qt-WebKit1 can move to the threaded compositor.
 I would like to use the threaded compositor as a replacement for the
 direct TextureMapper approach in general, so that we have less unmaintained
 code paths.

 What concerns me right now is that this implementation would fragment code
 paths that we should focus on removing, such as non composited contents (
 https://bugs.webkit.org/show_bug.cgi?id=110355). But otherwise let's move
 forward as far as I'm concerned...



 On Mon, Apr 15, 2013 at 2:53 AM, Gwang-Yoon Hwang 
 ryum...@company100.netwrote:

 Thanks for respond.


 On Mon, Apr 15, 2013 at 1:10 AM, Martin Robinson mrobin...@webkit.orgwrote:

 On Sun, Apr 14, 2013 at 12:52 AM, Gwang-Yoon Hwang
 ryum...@company100.net wrote:

 Nice work!

  1. There are 3 accelerated compositing methods in WebKit1 Gtk. Cairo,
  Clutter, and GL. These patches will adds 1 more options, threaded
  compositing. I think we need to simplify/unite rendering paths to
 reduce
  complexity.

 I think that No'am expressed interest in ditching the ImageBuffer
 compositor, so that would simplify things a bit.

 Good. Let's discuss about that with No'am.


   2. In these patches, I attached threaded compositor into webkit1 gtk
 with
  untidy codes. (ex ChromeClientGtk, FrameLoaderClientGtk.. ) I think if
 we
  need to maintain 4 compositing paths at a time, we need more cleaner
  approach, with a single interface  factory.

 I'm surprised you didn't focus on WebKit2, since WebKit1 is in
 maintenance mode now.

 --Martin


 It was easier to prototype, debug in WebKit1. So I did it first. Now I am
 going to make it for WebKit2 Gtk.
 And, there are another reason for that, If we choose threaded compositor
 on WK2 only,  we need to add another compositing path to
 our maintenance list. I think it would be better if WK1 and WK2 uses same
 unify compositing paths to reduce maintenance issues.


 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 https://lists.webkit.org/mailman/listinfo/webkit-dev



___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New web-facing canvas feature: opaque attribute

[Noam Rosenthal]

On Fri, Mar 15, 2013 at 9:19 AM, Gregg Tavares g...@google.com wrote:

 I don't understand the opposition to alpha

 You set colors in Canvas2d with rgb or rgba. That 'a' in rgba stands for
 alpha.
 You can set a global alpha for drawing with context.globalAlpha
 You read data from getImageData that returns red, green, blue, alpha
 You write data to putImageData wit red, green, blue, alpha

 So why name an option that effectively removes the alpha something not
 called alpha?  I have no better idea what opaque means. As far as I
 know it just means blending won't happen during compositing but alpha still
 exists when I draw and when I read from the canvas. That seems more
 confusing to me.

 I also don't understand why if getContext(webgl, { alpha: false }) has
 been shipping for 3 years there's a need to make it different on canvas 2d?
 Picking a different name or picking a different method (a function, an
 attribute on canvas or context) seems like bikeshedding rather than some
 functional objection. There's a solution, it has existed and been shipping
 in WebKit for 3 years. Can't we just go with that? Developers will be no
 more confused by this than whether drawImage is drawImage(img, x, y) or
 drawImage(x, y, img). You look up once in a while.

 As far as the spec goes, the effect of setting alpha:false is as if every
 draw operation ended with

dest = RGBA | 0x00FF

 That means everything works as speced. getImageData will pull out RGBA but
 A will always be 255. Every draw call including putImageData follows the
 rule above.  How that effects compositing is not a normative part of the
 spec. It's just something UAs happen to know they can do.


For what it's worth, I'm with Greg on this.
First of all, this is not needed for other elements, only canvas - other
elements have enough information about whether or not they're opaque since
they're fully declarative. So I don't see value in trying to extend the
solution to areas where the problem doesn't exist.
Second, having an alpha option and other options on drawing buffers is
really a standard feature in toolkits, libraries, windowing systems etc. -
it feels natural and not like an optimization hack.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] CSS filter behavior change - what is our policy?

[Noam Rosenthal]

Hellos
How do we go about rendering behavior changes that affect features that are
enabled on shipping browsers?

I'm specifically referring to http://trac.webkit.org/changeset/139770
The brightness filter is enabled by default on chrome and Safari if I
remember correctly, and now pages that use brightness(0) would have their
element blackened, while before those elements would have been left
unchanged. This is of course correct so I can't claim it's a bug, but
still it would break existing websites, even if not many.

Do we see CSS filters as being bleeding edge enough where we don't care?
Do we have a way to warn web developers about this? They'd basically have
to check Chrome/Safari/Other version in order to work around the problem,
as there's no media query for check if brightness behaves correctly.

Guidance appreciated.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] CSS filter behavior change - what is our policy?

[Noam Rosenthal]

I'm not so worried about existing content, as there's not that much of it
and web developers can always adapt it, but unlike other bleeding edge
features this one is already enabled by default in several released
browsers, which means new content that wishes to use the brightness filter
would have to create cross-webkit-browser-version workarounds from day
one...
Maybe a feature that's enabled by default in more than one browser is
effectively no longer bleeding edge?

I'm afraid that the current situation renders the brightness filter useless
- people who use -webkit-filter: brightness(0) would know for sure that
using it would generate completely different results on different
(shipping) versions of webkit.

I see two options here, but maybe there are more I can't think of right now:
1. revert the change, and bring it back for the unprefixed version of the
filter CSS attributes, living with the incorrect brightness filter for
the lifetime of -webkit-filter.
2. Create a quirks media-query or something like that, where using CSS
the web developer can query for the new behavior.

If these aren't acceptable, what would be your recommendation to web
developers who want to use brightness and target both new browsers and
currently shipping browsers?


On Thu, Mar 14, 2013 at 9:07 PM, Noam Rosenthal n...@webkit.org wrote:

 I'm not so worried about existing content, as there's not that much of it
 and web developers can always adapt it, but unlike other bleeding edge
 features this one is already enabled by default in several released
 browsers, which means new content that wishes to use the brightness filter
 would have to create cross-webkit-browser-version workarounds from day
 one...
 Maybe a feature that's enabled by default in more than one browser is
 effectively no longer bleeding edge?

 I'm afraid that the current situation renders the brightness filter
 useless - people who use -webkit-filter: brightness(0) would know for sure
 that using it would generate completely different results on different
 (shipping) versions of webkit.

 I see two options here, but maybe there are more I can't think of right
 now:
 1. revert the change, and bring it back for the unprefixed version of the
 filter CSS attributes, living with the incorrect brightness filter for
 the lifetime of -webkit-filter.
 2. Create a quirks media-query or something like that, where using CSS
 the web developer can query for the new behavior.

 If these aren't acceptable, what would be your recommendation to web
 developers who want to use brightness and target both new browsers and
 currently shipping browsers?



 On Thu, Mar 14, 2013 at 7:35 PM, Dean Jackson d...@apple.com wrote:


 On 15/03/2013, at 4:45 AM, Noam Rosenthal n...@webkit.org wrote:

 How do we go about rendering behavior changes that affect features that
 are enabled on shipping browsers?

 I'm specifically referring to http://trac.webkit.org/changeset/139770
 The brightness filter is enabled by default on chrome and Safari if I
 remember correctly, and now pages that use brightness(0) would have their
 element blackened, while before those elements would have been left
 unchanged. This is of course correct so I can't claim it's a bug, but
 still it would break existing websites, even if not many.

 Do we see CSS filters as being bleeding edge enough where we don't
 care? Do we have a way to warn web developers about this? They'd basically
 have to check Chrome/Safari/Other version in order to work around the
 problem, as there's no media query for check if brightness behaves
 correctly.


 I think in this case it was enough of a combination of bleeding edge +
 definite bug (where bleeding edge is determined by it being a prefixed
 property that isn't even at the candidate recommendation stage as well has
 young enough to not have widespread use). But you raise a good point - I
 would have been more reluctant to make this change as time passed.

 Dean



___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

[noam . rosenthal]

From: ext Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org
Date: Thursday, February 14, 2013 8:07 AM
To: Dongsung Huang luxte...@company100.netmailto:luxte...@company100.net
Cc: webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org 
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org
Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction 
pixel-results on the fly

On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang 
luxte...@company100.netmailto:luxte...@company100.net wrote:
I like this idea. I cannot find any harm if we have this functionality.

Those changes are not harmless. There are people monitoring tests results full 
time in order to keep WebKit in good shape. No other part of WebKit require 
continuous attention.

Case 1: CSS Filters  Shaders
I wanted this test functionality when I commented http://webkit.org/b/97859#c19
If I want to make gaussian blur test, I prefer using 'getPixel' test as follows,

Why wasn't a ref-test a better solution in this particular case?
Because gaussian blurs on the GPU are not accurate, and look slightly different 
on different GPUs, but usually close enough.
We need a way to measure close enough for features where all you can get is 
close enough! Ref-tests and pixel-tests are way to rigid for that, and 
require constant rebaselines and headaches.


Case 2: Fixed Position Element
[...]
function repeatedlyCalledDuringScrolling() {
ASSERT(getPixel(15, 9) == white);
ASSERT(getPixel(15, 10) == green);
ASSERT(getPixel(9, 15) == white);
ASSERT(getPixel(10, 15) == green);

}

I think this shows what I said about correctness and readability:
-Asserting the correctness of the test and the result becomes close to 
impossible for the reader. One has to review the full code to have a chance of 
understanding an error.
We can generate PNGs when a snapshot is requested so that the tester can 
eyeball the results.

-You cannot cover non trivial cases (images, text, form elements, etc).
-And it is inefficient. You have to render each frame on the UIProcess, move it 
to the WebProcess, and box it for JavaScript to process (with pixel format 
conversions depending on your graphics system)
Another option is to do expose a partial comparison mechanism that runs in  the 
UI process.
E.g.
var snapshotHandle = testRunner.createSnapshot();
testRunner.comparePixel(snapshotHandle, 15, 20, 'white', { tolerance: 0.2 });

This way we can generate a visual representation of failures.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

[noam . rosenthal]

From: ext Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org
Date: Thursday, February 14, 2013 8:36 AM
To: Noam Rosenthal noam.rosent...@nokia.commailto:noam.rosent...@nokia.com
Cc: luxte...@company100.netmailto:luxte...@company100.net 
luxte...@company100.netmailto:luxte...@company100.net, 
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org 
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org
Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction 
pixel-results on the fly

On Wed, Feb 13, 2013 at 11:27 PM, 
noam.rosent...@nokia.commailto:noam.rosent...@nokia.com wrote:
Why wasn't a ref-test a better solution in this particular case?
Because gaussian blurs on the GPU are not accurate, and look slightly different 
on different GPUs, but usually close enough.
We need a way to measure close enough for features where all you can get is 
close enough! Ref-tests and pixel-tests are way to rigid for that, and 
require constant rebaselines and headaches.

There is _NO_ gaussian blur in that patch.
DongSung mentioned gaussian blur in the comment, which is what I was referring 
to.
But other bugs as well, like delegated scrolling, also have that problem of 
results not being 100% the same every time.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

[noam . rosenthal]

From: ext Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org
Date: Saturday, February 9, 2013 12:52 AM
To: Noam Rosenthal noam.rosent...@nokia.commailto:noam.rosent...@nokia.com
Cc: rn...@webkit.orgmailto:rn...@webkit.org 
rn...@webkit.orgmailto:rn...@webkit.org, 
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org 
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org
Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction 
pixel-results on the fly

On Fri, Feb 8, 2013 at 3:16 PM, 
noam.rosent...@nokia.commailto:noam.rosent...@nokia.com wrote:
The problem with dynamic features of the web like animations/interactions is 
that they're non-deterministic, or at least a lot less deterministic than 
static features of the web like layouts.
Ref tests, pixel tests etc. are tools built for deterministic testing: load a 
file, take a snapshot, compare against a result. Testing an animation (or a 
filter) needs to feel a lot more dynamic and expressive: Animate green boxes, 
make sure that they're within a particular range at particular points in time.

The tests also have to be deterministic and comprehensive. I am afraid of 
loosing both with the Render-to-Canvas approach.

Can you give concrete examples of the kind of bugs you are hunting, and why 
testing cannot use the two methods suggested?
I did not say they cannot be tested with the two methods suggested :) It's more 
 about a preference to move some of those decisions from the test 
infrastructure to the test itself, but potentially those bugs could be tested 
in either way.
Examples for bugs I've encountered/reviewed and can use better in-motion 
testing (note that those are specific to Qt/EFL, but I'm sure there are tons of 
bugs like this that come up for Apple/Google as well)
http://trac.webkit.org/changeset/140825
http://trac.webkit.org/changeset/142112
http://trac.webkit.org/changeset/134953
https://bugs.webkit.org/show_bug.cgi?id=109179

Controlling the clock and programmatically sampling the end result would 
definitely make those more testable, but of course any progress in this area 
would be beneficial and my preference to a canvas-based API is more of an 
opinion.

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

[noam . rosenthal]

From: ext Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org
This is the situation I am afraid with a solution where pixels would be 
evaluated from JavaScript. You can test, but you lack visibility why something 
is correct or incorrect. Text tests, ref-tests and pixel tests have a great 
readability, you can quickly evaluate their correctness. This is important in 
my opinion, I don't think we want more opaque output like the RenderTree dump.

I am not against your plan if the readability of the tests can be good. I'd be 
happy to review patches toward testing dynamic changes in webpages.
We're on the same page. I'm afraid of the same thing… Let's talk specifics when 
a patch is available to review.

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

[noam . rosenthal]

Since people seem to agree that this is a good problem to solve, I've created a 
bug for it: https://bugs.webkit.org/show_bug.cgi?id=109356
I can't promise to fix it myself right this moment as I'm spread a bit too 
thin, but if someone wants to help or pick it up before I get to it please feel 
free :)
Noam

From: ext Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org
Date: Saturday, February 9, 2013 10:57 AM
To: Noam Rosenthal noam.rosent...@nokia.commailto:noam.rosent...@nokia.com
Cc: rn...@webkit.orgmailto:rn...@webkit.org 
rn...@webkit.orgmailto:rn...@webkit.org, 
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org 
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org
Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction 
pixel-results on the fly

On Sat, Feb 9, 2013 at 1:24 AM, 
noam.rosent...@nokia.commailto:noam.rosent...@nokia.com wrote:
I did not say they cannot be tested with the two methods suggested :) It's more 
 about a preference to move some of those decisions from the test 
infrastructure to the test itself, but potentially those bugs could be tested 
in either way.
Examples for bugs I've encountered/reviewed and can use better in-motion 
testing (note that those are specific to Qt/EFL, but I'm sure there are tons of 
bugs like this that come up for Apple/Google as well)
http://trac.webkit.org/changeset/140825
http://trac.webkit.org/changeset/142112
http://trac.webkit.org/changeset/134953
https://bugs.webkit.org/show_bug.cgi?id=109179

Controlling the clock and programmatically sampling the end result would 
definitely make those more testable, but of course any progress in this area 
would be beneficial and my preference to a canvas-based API is more of an 
opinion.

To explain my concerns:
Sometime, I look at a failing test, and think: what the f**k is this supposed 
to test?. Then I have to dig a ton of logs, and finally read the change to 
understand a the single JS statement in the whole script that make the test 
useful.

This is the situation I am afraid with a solution where pixels would be 
evaluated from JavaScript. You can test, but you lack visibility why something 
is correct or incorrect. Text tests, ref-tests and pixel tests have a great 
readability, you can quickly evaluate their correctness. This is important in 
my opinion, I don't think we want more opaque output like the RenderTree dump.

I am not against your plan if the readability of the tests can be good. I'd be 
happy to review patches toward testing dynamic changes in webpages.

Benjamin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

[noam . rosenthal]

Oops, forgot to reply to the list.

From: Noam Rosenthal noam.rosent...@nokia.commailto:noam.rosent...@nokia.com
From: ext Simon Fraser simon.fra...@apple.commailto:simon.fra...@apple.com
The existing pauseAnimation DRT API can stop an animation mid-flight, to allow 
for reliable snapshotting. Why isn't this enough?
I would say that the main things that are missing are:

  1.  Sample and compare several images in the same test
  2.  Sample images heuristically, e.g. sample that a green box is close 
enough to a particular location – A full ImageDiff is very rigid in that way.
  3.  Sample images mid-flight without pausing. Pausing can mess with animation 
timing and thus skew the results.

I don't have a full list of what's missing – but I know for sure that when 
trying to hunt mid-flight bugs in the past I was not able to create sufficient 
tests with the current DRT tools.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

[noam . rosenthal]

Hellos

we don't currently have a solution in webkit's test infrastructure for testing 
animations on the fly, or in general for testing multiple image results in 
the same test.
(If this was discussed before and I'm unaware of that discussion, please stop 
me here...)

This is because ImageDiff works on the single-test level, and you can't 
explicitly make image comparisons in the same test at different points in time.
This has before caused us several regressions that were not caught by layout 
tests.

I'd like to propose a solution, and would welcome some feedback on whether it's 
a good one...
The idea is that you would be able to programatically retrieve the current 
snapshot into a canvas ImageData, and then compare the pixel results with 
JavaScript in the LayoutTest. Something like:

window.testRunner.getWindowSnapshot(function(snapshot) {
if (snapshot.data[snapshot.width * 100 + 10] != 127)
output.innerHTML = FAIL;
});

WDYT?
Noam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

[noam . rosenthal]

From: webkit-dev-boun...@lists.webkit.org [webkit-dev-boun...@lists.webkit.org] 
on behalf of ext Robert Kroeger [rjkro...@chromium.org]

 How would the test wait for the animation to happen? Tests that use
 timeouts seem more likely to be flaky from what I've seen. Could
 window.testRunner.getWindowSnapshot take a time delta of some kind?
Most of the bugs I've encountered in this area can be caught by a test that 
would have obvious results that can buffer the flakiness, e.g. by moving a 
big green rectangle and testing only its center.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

[noam . rosenthal]

From: ext Ryosuke Niwa rn...@webkit.orgmailto:rn...@webkit.org
I'd like to propose a solution, and would welcome some feedback on whether it's 
a good one...
The idea is that you would be able to programatically retrieve the current 
snapshot into a canvas ImageData, and then compare the pixel results with 
JavaScript in the LayoutTest. Something like:

I had similar thoughts but my counter proposal is to let DRT/WTR generate 
multiple actual results either in the form of multiple layers in PNG or 
multiple PNG images. The advantage of this latter approach is that we can have 
multiple reference files as well. e.g. if a green box is to be moved from (0,0) 
to (50,0) and to (100, 0), we could create three reference files that 
correspond to each state.

But perhaps there is a good reason you didn't choose this approach. Could you 
elaborate on the reason you picked this particular API?
The problem with dynamic features of the web like animations/interactions is 
that they're non-deterministic, or at least a lot less deterministic than 
static features of the web like layouts.
Ref tests, pixel tests etc. are tools built for deterministic testing: load a 
file, take a snapshot, compare against a result. Testing an animation (or a 
filter) needs to feel a lot more dynamic and expressive: Animate green boxes, 
make sure that they're within a particular range at particular points in time.
I think that when testing some of the dynamic features it's best to create a 
minimal mechanism such as retrieving snapshot pixels, giving more power to the 
tests themselves in a dynamic language like JavaScript, rather than try to 
force the tests to fit into a predefined set of features supported by the 
DRT/WTR infrastructure. Also this would allow some differences between the 
ports, which implement animations a bit differently, without constant 
rebaselines of reference results – e.g. A heuristic animation timing test may 
stay valid across platform even if the actual animation implementation is 
different.

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Wishes

[noam . rosenthal]

On 2/3/13 7:46 PM, ext Maciej Stachowiak m...@apple.com wrote:

If you're asking about phasing it out entirely, we don't have any
immediate plans to deprecate or remove the WebKit1 mac API. There are
quite a few Apple and third-party apps using it, and WebKit2 is not even
public API on Mac OS X currently.

As for implementing aspects of the WebKit1 API in a less efficient but
less WebCore-intrusive way, we could probably start doing that now, on a
case-by-case basis. Do you have any specific constraints you're concerned
about?
Maybe the problem is less about the WK1 API layer and more about
WebCore/platform.
In many other projects, there is a common pattern of platform adaptation
layer, where there is a clear semi-public and stable interface exposed by
the middleware, which allows platform adaptations to occur with less
intrusive dependencies. WebKit doesn't really have that notion, making it
impossible to port webkit without either participating in trunk or
forking.

Platform adaptations bring a lot of benefit since they allow flexibility,
but they come with the cost of not being able to always achieve the most
straightforward vertical integration.
Accelerated compositing is, in my view, a successful example of a platform
adaptation layer. GraphicsContext is perhaps less successful.
A successful platform adaptation is measured by how much access is needed
to the underlying platform-specific implementation. For GraphicsContext,
for example, we call platformContext() inside rendering code way too much.

I suggest that a good (but difficult) starting point to separating WebCore
and the platform adaptation is to remove all or most calls to
GraphicsContext::platformContext(), adding the appropriate functionality
as public interface to GraphicsContext, making the rendering code truly
platform agnostic, and perhaps later externalized as a true platform
adaptation interface.
But I would let others say whether this is feasible/desirable :)

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Wishes

[noam . rosenthal]

On 2/3/13 11:12 PM, ext Maciej Stachowiak m...@apple.com wrote:

 Maybe the problem is less about the WK1 API layer and more about
 WebCore/platform.

I'm sure that's one problem, but I am still curious what Eric had in mind.
Yes, me too, and I don't mean to hijack that conversation :)

 In many other projects, there is a common pattern of platform
adaptation
 layer, where there is a clear semi-public and stable interface exposed
by
 the middleware, which allows platform adaptations to occur with less
 intrusive dependencies. WebKit doesn't really have that notion, making
it
 impossible to port webkit without either participating in trunk or
 forking.

I think it would be a good direction to have a better platform adaptation
layer. Specific measures that I'd support:
- Incrementally move Source/WebCore/platform/ to Source/Platform/ as
suggested by Adam et al.
- Remove knowledge of WebCore-level concepts from platform/ where they
have crept in
- Avoid using PLATFORM() or the equivalent in WebCore as much as possible
(though ENABLE macros are fine of course)
Sounds like we're on the same page.

I was ready to agree with you about platformContext() before I looked at
the code. But checking WebCore/rendering/, it looks like the only calls
to platformContext() are in port-specific theme files. The theme
implementation is tricky because it needs knowledge of both rendering and
platform/port-specific styles and drawing code. It seems to me that to
have a truly clean interface for themes, you'd need to separate the part
of the code that cares about Rendering classes and the part of the code
that cares about platform stuff. Just adding a bunch of
single-platform-only code to GraphicsContext to avoid calling
platformContext() doesn't seem like it would actually make theme logic
separate from the core. It's a problem worth solving, but the solution is
not straightforward.
You're right, I think this used to be a bit worse but looking it again the
situation with WebCore/rendering is not too bad.
Still - GraphicsContext is quite inconsistent. Some of the functions of
that class are implemented in the port specific files (e.g. setAlpha) and
some in GraphicsContext.cpp; Some things are maintained by
GraphicsContextState, some only by the platform context. If there was a
clear contract about what belongs to the port-specific graphics context
and what belongs to the common GraphicsContext the differences between
the ports in that realm would be a lot more maintainable. And I think this
can be potentially extended to networking and to other places.

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Peripheral ports (was Re: WebKit Wishes)

[noam . rosenthal]

From: ext Eric Seidel e...@webkit.orgmailto:e...@webkit.org

I wish we didn’t have to worry about platforms we couldn’t test.

It can’t be the job of the core maintainers to care about all the peripheral 
ports which contribute very little core code. Our code needs to be structured 
in such a manner that its easy for the core to march forward, while letting the 
ports catch up as they need to asynchronously.  Platform support code shouldn’t 
even need to be in webkit.orghttp://webkit.org!  Porting 
webkit.orghttp://webkit.org’s platform abstractions should be trivial, but 
core developers (which probably 90% of them use only 2 ports Mac WK2 + Chromium 
Linux) shouldn’t need to worry about keeping all ports working.
As someone who works on the peripheral ports quite a bit I would have to 
agree that this is not a good situation. I would prefer a situation where my 
core contributions are considered valuable, and my peripheral ports 
contribution are not considered taxing.

 I wish we only had one build system
I think that having one meta-build system might help this quite a bit.
But what would really help me is if I could have a better understanding of 
which parts of the code are taxing for Apple and Google. For example, in 2011 
Oliver Hunt has communicated that the Qt JSC bindings were too taxing, and ever 
since we've done a lot of work to reduce that tax 
(https://bugs.webkit.org/show_bug.cgi?id=60842). This was, to me, a productive 
communication about that problem.

 I wish I felt like reviewers understood/trusted each other more.
I think that if there was a clear and detailed communication about taxing 
properties of the peripheral ports or platform abstraction, perhaps in the 
form of bugs on bugzilla like the one I've mentioned, we can trust the 
contributors from the peripheral ports, together with everyone else, to find 
the right solutions.
How would it feel for people if we had something like a bugzilla 
component/topic to track those issues?

I think that solving those issues rather than pushing the peripheral ports to 
the side is in the benefit of the WebKit project, and is one of the things that 
differentiates it from projects like Mozilla that only support one company's 
browser(s). I'm hoping that other people see this value as well…

~Noam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] RGBA8 and BGRA8 formats in WebKit

[noam . rosenthal]

I would say that the right way to go about improving pixel transfer (which is 
what you're trying to do, I believe...) is with OS-provided graphics surfaces;

On embedded systems those are usually provided and there are of course 
solutions for windows and Mac.
When using graphics surfaces, you don't need to swizzle the bits; The OS 
manages that for you at a lower level. Problem with graphics surfaces is that 
their characteristics are very platform dependent, and they have delicate 
memory, performance and security concerns. However, I would explore how we 
could use graphics surfaces better (e.g. decode images directly into a graphics 
surface, if the platform allows that), rather than try to fix this on the pixel 
format level.

If anyone has had experience with doing this, e.g. with IOSurface on Mac, your 
feedback is welcome...

Noam

From: webkit-dev-boun...@lists.webkit.org [webkit-dev-boun...@lists.webkit.org] 
on behalf of ext Zoltan Herczeg [zherc...@webkit.org]
Sent: Tuesday, January 22, 2013 5:14 PM
To: webkit-dev@lists.webkit.org
Subject: Re: [webkit-dev] RGBA8 and BGRA8 formats in WebKit

 Where in WebKit do you experience problems with color conversion?

As for me WebKit2 transmits BGRA images, which needs to be converted to
RGBA before it is uploaded to a texture on GLES 2.0. These conversions
seems computation heavy for certain animations, and I was wondered whether
do we really need to use BGRA here. It would be nice to invent something
to avoid that.

Regards,
Zoltan


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Adding Web MIDI API support to WebCore

[noam . rosenthal]

(Resending to list, having some email issues…)

From: ext Maciej Stachowiak m...@apple.commailto:m...@apple.com
 OK, but - sending low-level MIDI commands in binary form is extremely 
 unlikely to be the most effective way to get music out of your sound card. 
 While in the past, games or audio software may have used MIDI rendered by an 
 internal soundcard, my
 understanding is that nowadways this is almost never done. MIDI is mainly 
 used for truly external outputs and inputs.

I believe you're referring to the general MIDI built-in synthesizer common in 
sound cards, which is of course there but not what I'm referring to :)
You can use MIDI to control pure-software synthesizers that are connected via 
CoreMIDI (or other OS-specific interfaces), or to control specialized 
pure-software programs like Logic or Cubase.

I would say that using MIDI in a mainstream way requires either specialized 
hardware or specialized software, though it could potentially be used 
completely with WebAudio to create something like a software synthesizer.
Still I would have to agree that this is an obscure API, and in general I would 
prefer to maybe find a common place outside of the webkit project to implement 
random navigator.foobar APIs and have them work on top of some WebKit public 
APIs.

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] r- your own patches [was: Re: RenderArena: Teaching an old dog new tricks]

[noam . rosenthal]

From: ext Ryosuke Niwa rn...@webkit.orgmailto:rn...@webkit.org
 r+ and r- flags are supposed to be set only by reviewers. If you wanted to 
 withdraw your patch from the review queue, then you should be clearing  r? 
 flag, instead of setting r-. If you’re uploading a WIP patch, then it should 
 not bear either r?, r-, or r+
 flags. You can accomplish this by either not setting the flag when you upload 
 a patch on Bugzilla, clearing flag on the Bugzilla, or using --no-review 
 option on webkit-patch.

Regarding WIP patches, what I've seen a few times is us reviewers adding an r- 
flag to a WIP patch with no r?, when we think it's horribly wrong…
I think the flip side of the guideline for non-reviewers to avoid r- is to have 
reviewers use r- only when the patch is up for review. This will encourage 
people to use no flags instead of putting r- for WIP patches.
No'am

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Implement threaded model of Coordinated Graphics

[noam . rosenthal]

Slava, I think the new proposal is about threading in the web process, not in 
the UI process like we do in Qt.
I've posted some comments on the meta bug, thank you for the heads up on the 
mailing list!
No'am

From: ext Vyacheslav Ostapenko osta...@gmail.commailto:osta...@gmail.com
Date: Friday, November 16, 2012 10:12 AM
To: Jae Hyun Park jae.p...@company100.netmailto:jae.p...@company100.net
Cc: webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org 
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org
Subject: Re: [webkit-dev] Implement threaded model of Coordinated Graphics


Qt WK2 port already runs layer tree painting on separate thread.
Layer tree renderer is created on main thread, but rendering is performed only 
on paint thread.
Qt5 creates paint nodes for painting and QQuickWebPage paint node keeps 
threaded reference on LayerTreeRenderer. Messages to LayerTreeRenderer are 
delivered through dispatchUpdate/bind/renderQueue .
All updates from renderQueue to renderer are applied in 
QQuickWebPage::updatePaintNode call. QQuickItem::updatePaintNode is very 
special call. During this call main thread is locked and it is safe to access 
main thread objects from paint thread.
http://doc-snapshot.qt-project.org/5.0/qquickitem.html#updatePaintNode

So, the only thing that needs to be implemented for GTK is replacement for Qt 
paint node and sync point similar to QQuickItem::updatePaintNode .

Slava

On Fri, Nov 16, 2012 at 12:44 AM, Jae Hyun Park 
jae.p...@company100.netmailto:jae.p...@company100.net wrote:
Hi, webkit folks.

Our team is currently implementing threaded model of Coordinated Graphics in 
GTK+ port.
https://bugs.webkit.org/show_bug.cgi?id=100341
The purpose of sharing is to report our progress and make reviewers easier to 
understand the overview picture.

We have updated the design document in the link below:
https://docs.google.com/document/pub?id=1UoI1zk-6nTUFtz8i4evURM8aQIjkDRC8boO1zPdMMBg

Our prototype for this implementation is shared in the GitHub.
https://github.com/ryumiel/webkit-experimental

The prototype is still in development, and only implemented up to step 1 in our 
design document.

Any comments/concerns are appreciated.

Best regards,
Jae Hyun Park

___
webkit-dev mailing list
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] New WebKit Reviewer: Caio Marcelo de Oliveira Filho

[noam . rosenthal]

I am pleased to announce that Caio (@cmarcelo) is now a WebKit reviewer.
Caio has been working in many areas in the past years, starting with focus on 
the Qt port with improvements to the Qt/JS bridge, font test results and 
render-theme.
More recently has also work on the UndoManager and other aspects of editing 
code and CSS parsing, his latest contributions being around WTF HashMap 
iterators. 
Since Caio has been involved in so many parts of WebKit, I'm probably 
forgetting a few other contributions...

Please join me in congratulating Caio for his new reviewer status!
No'am
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Is there a rule to use UNUSED_PARAM ?

[noam . rosenthal]

How about ASSERT_USING and ASSERT_WITH_MESSAGE_USING?
If we use a variable in an assert it's no longer unused…

e.g. ASSERT_USING(timer, timer == m_resumeTimer);

On Oct 1, 2012, at 6:17 PM, ext Mike Lawther 
mikelawt...@google.commailto:mikelawt...@google.com wrote:

ASSERT_WITH_OTHERWISE_UNUSED.


On 2 October 2012 10:51, Ryosuke Niwa 
rn...@webkit.orgmailto:rn...@webkit.org wrote:
How about ASSERT_OR_UNUSED or ASSERT_USING_UNUSED?

On Mon, Oct 1, 2012 at 5:23 PM, Darin Adler 
da...@apple.commailto:da...@apple.com wrote:
On Oct 1, 2012, at 5:21 PM, Mike Lawther 
mikelawt...@google.commailto:mikelawt...@google.com wrote:

 my first reading of that name had me thinking it meant 'ASSERT that this 
 variable is unused' in the vein of macros I've seen in other places like 
 ASSERT_TRUE, ASSERT_NOT_NULL etc.

Yes, even though I named the macro I have had the same thought often. I love 
the macro, but a better name, even if a bit longer, would be welcome. If we can 
think of one we can fix this with a quick global replace.

-- Darin
___
webkit-dev mailing list
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


___
webkit-dev mailing list
webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] ANGLE on QNX / non-x11 unixoids

[noam . rosenthal]

A
On Sep 4, 2012, at 8:34 AM, ext Konstantin Tokarev annu...@yandex.ru wrote:

 
 To get it compiling, I had to patch ANGLE source code and Simon Hausmann told
 me he is not comfortable in reviewing these patches. So here I am asking for
 how to proceed. Should I first try to upstream these patches to ANGLE 
 somehow?
 
 Why do you compe ANGLE? Does QNX support DirectX?

ANGLE is used to validate and compile WebGL shaders to OpenGL/OpenGL ES2, so 
it's a dependency regardless of DirectX.
~No'am

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] [CoordinatedGraphics] bugs and what they mean

[noam . rosenthal]

Hello
Recently people have expressed confusion about bug reports in bugzilla that 
deal with the coordinated graphics system. I wanted to help a bit with the 
confusion.

The coordinated graphics system was contributed gradually by Qt in the last 
year or so, and is documented here: 
http://trac.webkit.org/wiki/CoordinatedGraphicsSystem. Earlier it was known as 
UI_SIDE_COMPOSITING, but that name was a bit awkward.

Lately, EFL has made some of the Qt-specific parts of that code cross-platform 
and have started using it together with TextureMapper, which makes it 
misleading to use the [Qt] prefix for coordinated-graphics bugs.

In general, there are several components in WebKit that are shared by several 
ports, but not all ports, e.g. GStreamer or Cairo. CoordinatedGraphics is 
simply one of them.

If there is still confusion about this, or if someone feels that we should use 
different prefixes or communicate about it in a different way, please let us 
know!

No'am
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Moving to Git?

[noam . rosenthal]

On Mar 9, 2012, at 1:43 PM, ext Maciej Stachowiak wrote:

It sounds like avoiding use of git-svn is the big benefit to git users and 
perhaps the reason this topic periodically comes up. Can anyone spell out in 
more detail the benefits of using straight git instead of git-svn?

My main pain point with the current workflow, which can potentially be repaired 
by using a real git workflow, is the manual changelog editing, rather than 
using commit messages.
With a git workflow we can review commit messages as part of the regular review 
process, which I understand we can't do with SVN (though I don't know SVN that 
well).

Wouldn't a world without manual changelog editing be slightly nicer?
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev