Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Ryosuke Niwa]

On Wed, Mar 4, 2020 at 10:53 PM Noam Rosenthal  wrote:
>
>
> On Thu, Mar 5, 2020 at 4:24 AM Maciej Stachowiak  wrote:
>>
>>>
>> There has been a lot of work done on the spec front since the comments 
>> above, and I think we're ready for a re-review the state.
>> * The next pull request: https://github.com/whatwg/html/pull/5112 has been 
>> thumbed up to make it into the HTML spec, and is separate from client-hints, 
>> but requires second implementor interest, which is what this email thread is 
>> for :)
>>
>>
>> I don’t think it has? I’m seeing:
>> "[ ]  At least two implementers are interested (and none opposed):”
>>
>>
>> And satisfying this is a requirement for adding features to WHATWG Living 
>> Standards.
>>
>> And the review block at the bottom says “Review required” and “Merging is 
>> Blocked with big red X’s.
>
> Yes, see the last comment from the reviewer: "I guess the main things 
> remaining are implementer interest and someone verifying the test coverage. 
> (Happy to help with the latter once the former is sorted.) "

Just to close the loop here, I don't think we're interested in
supporting this header given the layering violation we've pointed out
in the past at this point.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Thu, Mar 5, 2020 at 4:24 AM Maciej Stachowiak  wrote:

>
>
>> There has been a lot of work done on the spec front since the comments
> above, and I think we're ready for a re-review the state.
> * The next pull request: https://github.com/whatwg/html/pull/5112 has
> been thumbed up to make it into the HTML spec, and is separate from
> client-hints, but requires second implementor interest, which is what this
> email thread is for :)
>
>
> I don’t think it has? I’m seeing:
> "[ ]  At least two implementers are interested (and none opposed):”
>

> And satisfying this is a requirement for adding features to WHATWG Living
> Standards.
>
> And the review block at the bottom says “Review required” and “Merging is
> Blocked with big red X’s.
>
Yes, see the last comment from the reviewer: "I guess the main things
remaining are implementer interest and someone verifying the test coverage.
(Happy to help with the latter once the former is sorted.) "
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Maciej Stachowiak]

> On Mar 2, 2020, at 11:22 PM, Noam Rosenthal  wrote:
> 
> 
> 
> On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak  > wrote:
> 
> 
>> On Nov 11, 2019, at 4:37 AM, Noam Rosenthal > > wrote:
>> 
>> 
>> 
>> On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak > > wrote:
>> - Is there a specification for Content-DPR? All I could find in bugzila and 
>> elsewhere is links to > >, which does not 
>> mention Content-DPR at all).
>> In a nutshell, the spec is in transition from IETF to HTML/fetch, and the 
>> client-hints aspects of it are still in progress (unlike conent-dpr, which 
>> is much simpler hasn't changed since introduced). It's documented here: 
>> https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
>>  
>> .
>> I will answer more about this separately if required, waiting for some info 
>> from the people working on the spec originally.
> 
> It looks like these are the relevant Pull Requests:
> 
> 
> HTML: https://github.com/whatwg/html/pull/3774 
>  | 
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density
>  
> 
> Fetch: https://github.com/whatwg/fetch/pull/773 
>  | 
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density
>  
> 
> 
> It looks like these are in a somewhat messy state. For example, Fetch places 
> the Content-DPR value in an “image density” variable, but it doesn’t look 
> like the Pull Request to HTML doesn’t use it anywhere. As another example, 
> HTML directly references Content-DPR as setting the “current pixel density”, 
> but does not specify that it would take priority over a density derived from 
> srcset. There are also no diffs to CSS Images or to SVG, which provide 
> distinct ways to reference images and which presumably should respect 
> Content-DPR.
> 
>  
> 
>>  
>> - Can you give us some examples of how the CDN would make the decision of 
>> what density of image to serve without knowing the client’s DPR via  
>> client-hints? 
>>  Some examples from  https://github.com/eeeps/content-dpr-explainer 
>> :
>> - making decisions by user agent, like choosing to cap certain images for 
>> user-agents known to have smaller displays
>> - making decisions by traffic/geo, like serving lower-resolution images when 
>> traffic is bogged down
>> - A client may ask for "placeholder image" (e.g. ?placeholder=true in the 
>> URL), and the CDN would decide whether to serve a lower-quality JPEG or to 
>> scale it down, or do both.
> 
> These seem like reasonable use cases.
> 
>> 
>> - Is this presuming situations where the CDN serves the images but not the 
>> markup or CSS (which presumably could be rewritten to account for DPR)?
>> Correct.
>>  
>> - What happens if Content-DPR is provided, but the markup or CSS has 
>> conflicting explicit info? For example, > image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x. 
>> Or the analogous thing with CSS.
>> When image size is decided, css/markup takes precedence, then content DPR, 
>> and finally srcset-selection DPR. This is actually a bonus value of this 
>> feature - if the server happens to serve an image that differs from the 
>> ratio requested in the srcset tag, the intrinsic size match the served 
>> content and not the request (which is more of a recommendation).
>> 
>> - Does Content-DPR have any effect on images where an explicit size is 
>> provided, either in markup or in CSS? (I’m assuming no.)
>> No, it only has effect on intrinsic size. 
> 
> Overall, this seems like a feature with valid use cases. But unfortunately, 
> it’s a (currently) Blink-only feature with no clear specification. To the 
> extent there is a spec, it’s mixed in with multiple Pull Requests. These pull 
> requests mix in Client Hints, a feature unlikely to gain multi implementor 
> support, so they probably won’t land any time soon. And the language about 
> Content-DPR is broken and does not not seem to match Chrome’s actual 
> implementation.
> 
> So in summary, there is no proper spec, and Chrome has shown willingness to 
> have their implementation change faster than even draft spec language in PRs 
> can keep up with. 
> 
> Although the use case seems valid, I don’t think it’s a good idea for WebKit 
> to implement the feature in this state. It seems likely to cause interop 
> headaches and a need to reverse engineer Blink rather than developing against 
> spec

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak  wrote:

>
>
> On Nov 11, 2019, at 4:37 AM, Noam Rosenthal  wrote:
>
>
>
> On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak  wrote:
>
>> - Is there a specification for Content-DPR? All I could find in bugzila
>> and elsewhere is links to <
>> https://httpwg.org/http-extensions/client-hints.html>, which does not
>> mention Content-DPR at all).
>>
> In a nutshell, the spec is in transition from IETF to HTML/fetch, and the
> client-hints aspects of it are still in progress (unlike conent-dpr, which
> is much simpler hasn't changed since introduced). It's documented here:
> https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
> .
> I will answer more about this separately if required, waiting for some
> info from the people working on the spec originally.
>
>
> It looks like these are the relevant Pull Requests:
>
>
> HTML: https://github.com/whatwg/html/pull/3774 |
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density
> Fetch: https://github.com/whatwg/fetch/pull/773 |
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density
>
> It looks like these are in a somewhat messy state. For example, Fetch
> places the Content-DPR value in an “image density” variable, but it doesn’t
> look like the Pull Request to HTML doesn’t use it anywhere. As another
> example, HTML directly references Content-DPR as setting the “current pixel
> density”, but does not specify that it would take priority over a density
> derived from srcset. There are also no diffs to CSS Images or to SVG, which
> provide distinct ways to reference images and which presumably should
> respect Content-DPR.
>



>
>
>
>> - Can you give us some examples of how the CDN would make the decision of
>> what density of image to serve without knowing the client’s DPR via
>>  client-hints?
>>
>  Some examples from  https://github.com/eeeps/content-dpr-explainer:
> - making decisions by user agent, like choosing to cap certain images for
> user-agents known to have smaller displays
> - making decisions by traffic/geo, like serving lower-resolution images
> when traffic is bogged down
> - A client may ask for "placeholder image" (e.g. ?placeholder=true in the
> URL), and the CDN would decide whether to serve a lower-quality JPEG or to
> scale it down, or do both.
>
>
> These seem like reasonable use cases.
>
>
> - Is this presuming situations where the CDN serves the images but not the
>> markup or CSS (which presumably could be rewritten to account for DPR)?
>>
> Correct.
>
>
>> - What happens if Content-DPR is provided, but the markup or CSS has
>> conflicting explicit info? For example, > image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x.
>> Or the analogous thing with CSS.
>>
> When image size is decided, css/markup takes precedence, then content DPR,
> and finally srcset-selection DPR. This is actually a bonus value of this
> feature - if the server happens to serve an image that differs from the
> ratio requested in the srcset tag, the intrinsic size match the served
> content and not the request (which is more of a recommendation).
>
> - Does Content-DPR have any effect on images where an explicit size is
>> provided, either in markup or in CSS? (I’m assuming no.)
>>
> No, it only has effect on intrinsic size.
>
>
> Overall, this seems like a feature with valid use cases. But
> unfortunately, it’s a (currently) Blink-only feature with no clear
> specification. To the extent there is a spec, it’s mixed in with multiple
> Pull Requests. These pull requests mix in Client Hints, a feature unlikely
> to gain multi implementor support, so they probably won’t land any time
> soon. And the language about Content-DPR is broken and does not not seem to
> match Chrome’s actual implementation.
>
> So in summary, there is no proper spec, and Chrome has shown willingness
> to have their implementation change faster than even draft spec language in
> PRs can keep up with.
>

> Although the use case seems valid, I don’t think it’s a good idea for
> WebKit to implement the feature in this state. It seems likely to cause
> interop headaches and a need to reverse engineer Blink rather than
> developing against specs and test cases.
>
> There has been a lot of work done on the spec front since the comments
above, and I think we're ready for a re-review the state.
* The next pull request: https://github.com/whatwg/html/pull/5112 has been
thumbed up to make it into the HTML spec, and is separate from
client-hints, but requires second implementor interest, which is what this
email thread is for :)
* Web platform tests are comprehensive, separated from client-hints, and
match both the spec and the Chrome implementation:
https://github.com/web-platform-tests/wpt/tree/master/content-dpr. We can
always add more if we find more use-cases.

Would love to hear more thoughts!

Thanks
Noam
_

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak  wrote:

>
>
> On Nov 11, 2019, at 4:37 AM, Noam Rosenthal  wrote:
>
>
>
> On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak  wrote:
>
>> - Is there a specification for Content-DPR? All I could find in bugzila
>> and elsewhere is links to <
>> https://httpwg.org/http-extensions/client-hints.html>, which does not
>> mention Content-DPR at all).
>>
> In a nutshell, the spec is in transition from IETF to HTML/fetch, and the
> client-hints aspects of it are still in progress (unlike conent-dpr, which
> is much simpler hasn't changed since introduced). It's documented here:
> https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
> .
> I will answer more about this separately if required, waiting for some
> info from the people working on the spec originally.
>
>
> It looks like these are the relevant Pull Requests:
>
>
> HTML: https://github.com/whatwg/html/pull/3774 |
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density
> Fetch: https://github.com/whatwg/fetch/pull/773 |
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density
>
> It looks like these are in a somewhat messy state. For example, Fetch
> places the Content-DPR value in an “image density” variable, but it doesn’t
> look like the Pull Request to HTML doesn’t use it anywhere. As another
> example, HTML directly references Content-DPR as setting the “current pixel
> density”, but does not specify that it would take priority over a density
> derived from srcset. There are also no diffs to CSS Images or to SVG, which
> provide distinct ways to reference images and which presumably should
> respect Content-DPR.
>
>
>
>> - Can you give us some examples of how the CDN would make the decision of
>> what density of image to serve without knowing the client’s DPR via
>>  client-hints?
>>
>  Some examples from  https://github.com/eeeps/content-dpr-explainer:
> - making decisions by user agent, like choosing to cap certain images for
> user-agents known to have smaller displays
> - making decisions by traffic/geo, like serving lower-resolution images
> when traffic is bogged down
> - A client may ask for "placeholder image" (e.g. ?placeholder=true in the
> URL), and the CDN would decide whether to serve a lower-quality JPEG or to
> scale it down, or do both.
>
>
> These seem like reasonable use cases.
>
>
> - Is this presuming situations where the CDN serves the images but not the
>> markup or CSS (which presumably could be rewritten to account for DPR)?
>>
> Correct.
>
>
>> - What happens if Content-DPR is provided, but the markup or CSS has
>> conflicting explicit info? For example, > image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x.
>> Or the analogous thing with CSS.
>>
> When image size is decided, css/markup takes precedence, then content DPR,
> and finally srcset-selection DPR. This is actually a bonus value of this
> feature - if the server happens to serve an image that differs from the
> ratio requested in the srcset tag, the intrinsic size match the served
> content and not the request (which is more of a recommendation).
>
> - Does Content-DPR have any effect on images where an explicit size is
>> provided, either in markup or in CSS? (I’m assuming no.)
>>
> No, it only has effect on intrinsic size.
>
>
> Overall, this seems like a feature with valid use cases. But
> unfortunately, it’s a (currently) Blink-only feature with no clear
> specification. To the extent there is a spec, it’s mixed in with multiple
> Pull Requests. These pull requests mix in Client Hints, a feature unlikely
> to gain multi implementor support, so they probably won’t land any time
> soon. And the language about Content-DPR is broken and does not not seem to
> match Chrome’s actual implementation.
>
> So in summary, there is no proper spec, and Chrome has shown willingness
> to have their implementation change faster than even draft spec language in
> PRs can keep up with.
>
> Although the use case seems valid, I don’t think it’s a good idea for
> WebKit to implement the feature in this state. It seems likely to cause
> interop headaches and a need to reverse engineer Blink rather than
> developing against specs and test cases.
>
> I would like to see a clean spec, either separating Content-DPR PRs from
> client-hints or making a standalone spec. And either way it should be fixed
> to match what Chrome has implemented, and have WPT tests that reflect the
> spec and intended behavior.
>

I hear you. Let me get back to this thread once we're cleaner on the spec
front.
Thank you for the detailed response, it gives a clear path forward.
~Noam

>
> Regards,
> Maciej
>
>
>
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Tue, Nov 12, 2019 at 3:28 AM Alexey Proskuryakov  wrote:

>
>
> 10 нояб. 2019 г., в 1:16 AM, Noam Rosenthal  написал(а):
>
> Also, as a response to the view that this should be part of markup, I
> would argue that those transparent decisions by the servers should not
> concern web authors at all. It’s an optimization decision to be handled by
> the servers, with the user agent doing nothing about it but allow that
> decision to be transparent in terms of layout (the same way gzip and
> cache-control are transparent).
>
>
> I don't see this as a precedent, because cache control and compression are
> invisible to users.
>
Agreed, I was showing it as an example that HTTP headers are a valid place
to add instructions to/from transient layers of the internet (CDN/proxy).
Though, for example, the good old "Accept" header may result in receiving a
different image encoding for the same URL, resulting in different qualities
for the same URL.


> Whereas image quality most certainly is. Changing it behind both web
> developer and user back would cause lots of undesirable behaviors - say, I
> e-mail an image from a webpage to someone else, who doesn't have a small
> display, or just zoom in to see the details.
>
This basically results in the website being untestable by the author (or by
> UA implementers who will be getting bug reports about site behavior
> differences).
>

This can already happen, a URL for an image doesn't guarantee always
receiving the same resource. Content-DPR doesn't change that. On the
contrary, by providing Content-DPR we allow such optimizations (which
already take place) to avoid breaking layouts that are based on intrinsic
image sizes, which is the main way websites can break due to serving
different images to the same image request URL. The web today is based on
this premise - the same URL may lead to different resources in different
situations.

Also, we don't change it behind the web developer's back - who maybe does
that is the developer's chosen CDN, and it's a legitimate
experience/performance tradeoff that can be negotiated between them. What
we can do as a browser engine, is do the minimum to prevent that behavior
from breaking layouts.

>

> Also, maybe I'm just pessimistic here, but it seems almost inevitable that
> disagreements between markup, Content-DPR and dpi information embedded in
> images will be handled by UAs differently, even if the spec ends up being
> very detailed on this point.
>
I'm less pessimistic about that. Content-DPR is quite clear - it's a
description of the content, like an encoding

~Noam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Alexey Proskuryakov]

> 10 нояб. 2019 г., в 1:16 AM, Noam Rosenthal  написал(а):
> 
> Hola
> 
> I would like to open a discussion that has started a few years back and has 
> never reached a consensus: https://bugs.webkit.org/show_bug.cgi?id=145380 
> 
> 
> In a nutshell: content-dpr is a response-only header for images, that allows 
> servers at transient layers (CDNs/proxies) to optimize images by changing 
> their resolution, allowing the user agents to compensate for the change in 
> intrinsic size caused by the resolution change - thus making the resolution 
> change transparent to users. It's the header that makes the 
> resolution-encoding transparent.
> 
> The feature is already up and running in chrome since version 67, and is used 
> by some CDNs.
> 
> Since there was lack of consensus in the bug discussion, I wanted to make the 
> case for it here, and see if opinions about the subject can be voiced before 
> we reach a decision/consensus. I tried to find a good balance between being 
> detailed and being concise. 
> 
> —
> 
> Players in CDN, proxy and other transient parts of the internet world have 
> innovated a lot in the past few years. There are lots of interesting 
> companies and competition there. One of the areas of this innovation is in 
> optimizing images in a transparent way at transient layers (proxy/CDN). This 
> makes a lot of sense considering how much of internet traffic is taken by 
> image download.
> 
> What the research at the CDN companies found, was that modifying resolution 
> at the transient layer can have a tremendous effect on 
> performance/user-experience balance, for example by serving lower-resolution 
> versions of the images when network traffic is costly/slow (the exact formula 
> for that is part of where the CDN can innovate). More details on that 
> innovation in the links below.
> 
> The thing is, that modifying image resolution at the CDN/proxy is not 
> inherently transparent, due to one reason - layout is affected by the image’s 
> intrinsic size, and changing the image’s resolution inadvertently changes the 
> image’s intrinsic size. To make this transparent, the user agent has to be 
> involved, to compensate for this optimization when reading the image’s 
> intrinsic size.
> 
> The main case against this feature was that image resolution is a feature 
> that should be handled at the markup layer and not at the transport layer 
> (https://bugs.webkit.org/show_bug.cgi?id=145380#c7 
> ).
> I would argue that http-headers are not a transport layer (TCP/UDP etc.), but 
> rather part of an application-layer protocol that is designed, in part, to 
> pass information to the transient layers such as CDNs, and that resolution 
> compression is a (new, image-specific) equivalent of transfer-encoding. 
> 
> Also, as a response to the view that this should be part of markup, I would 
> argue that those transparent decisions by the servers should not concern web 
> authors at all. It’s an optimization decision to be handled by the servers, 
> with the user agent doing nothing about it but allow that decision to be 
> transparent in terms of layout (the same way gzip and cache-control are 
> transparent).

I don't see this as a precedent, because cache control and compression are 
invisible to users. Whereas image quality most certainly is. Changing it behind 
both web developer and user back would cause lots of undesirable behaviors - 
say, I e-mail an image from a webpage to someone else, who doesn't have a small 
display, or just zoom in to see the details.

This basically results in the website being untestable by the author (or by UA 
implementers who will be getting bug reports about site behavior differences).

Also, maybe I'm just pessimistic here, but it seems almost inevitable that 
disagreements between markup, Content-DPR and dpi information embedded in 
images will be handled by UAs differently, even if the spec ends up being very 
detailed on this point.

- Alexey


> What is offered here is a win-win in terms of delivering images, and would 
> allow webkit-browser users to enjoy some of the innovations in the image 
> delivery world without modifying their HTML content and without concern of 
> breaking their layouts. Less battery and data used for downloading big images 
> at certain situations, for example.
> 
> I hope this provides enough background without being too tedious. I would be 
> happy to shed more light on whatever is missing in this longish essay :)
> 
> Additional info:
> https://github.com/eeeps/content-dpr-explainer 
> 
> https://cloudinary.com/blog/client_hints_and_responsive_images_what_changed_in_chrome_67
>  
> 
> https://www.chromestatus.com/metrics/feature/timeline/popularity/906 
> 

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Maciej Stachowiak]

> On Nov 11, 2019, at 4:37 AM, Noam Rosenthal  wrote:
> 
> 
> 
> On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak  > wrote:
> - Is there a specification for Content-DPR? All I could find in bugzila and 
> elsewhere is links to  >, which does not 
> mention Content-DPR at all).
> In a nutshell, the spec is in transition from IETF to HTML/fetch, and the 
> client-hints aspects of it are still in progress (unlike conent-dpr, which is 
> much simpler hasn't changed since introduced). It's documented here: 
> https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
>  
> .
> I will answer more about this separately if required, waiting for some info 
> from the people working on the spec originally.

It looks like these are the relevant Pull Requests:


HTML: https://github.com/whatwg/html/pull/3774 | 
https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density
Fetch: https://github.com/whatwg/fetch/pull/773 | 
https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density

It looks like these are in a somewhat messy state. For example, Fetch places 
the Content-DPR value in an “image density” variable, but it doesn’t look like 
the Pull Request to HTML doesn’t use it anywhere. As another example, HTML 
directly references Content-DPR as setting the “current pixel density”, but 
does not specify that it would take priority over a density derived from 
srcset. There are also no diffs to CSS Images or to SVG, which provide distinct 
ways to reference images and which presumably should respect Content-DPR.

>  
> - Can you give us some examples of how the CDN would make the decision of 
> what density of image to serve without knowing the client’s DPR via  
> client-hints? 
>  Some examples from  https://github.com/eeeps/content-dpr-explainer 
> :
> - making decisions by user agent, like choosing to cap certain images for 
> user-agents known to have smaller displays
> - making decisions by traffic/geo, like serving lower-resolution images when 
> traffic is bogged down
> - A client may ask for "placeholder image" (e.g. ?placeholder=true in the 
> URL), and the CDN would decide whether to serve a lower-quality JPEG or to 
> scale it down, or do both.

These seem like reasonable use cases.

> 
> - Is this presuming situations where the CDN serves the images but not the 
> markup or CSS (which presumably could be rewritten to account for DPR)?
> Correct.
>  
> - What happens if Content-DPR is provided, but the markup or CSS has 
> conflicting explicit info? For example,  image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x. 
> Or the analogous thing with CSS.
> When image size is decided, css/markup takes precedence, then content DPR, 
> and finally srcset-selection DPR. This is actually a bonus value of this 
> feature - if the server happens to serve an image that differs from the ratio 
> requested in the srcset tag, the intrinsic size match the served content and 
> not the request (which is more of a recommendation).
> 
> - Does Content-DPR have any effect on images where an explicit size is 
> provided, either in markup or in CSS? (I’m assuming no.)
> No, it only has effect on intrinsic size. 

Overall, this seems like a feature with valid use cases. But unfortunately, 
it’s a (currently) Blink-only feature with no clear specification. To the 
extent there is a spec, it’s mixed in with multiple Pull Requests. These pull 
requests mix in Client Hints, a feature unlikely to gain multi implementor 
support, so they probably won’t land any time soon. And the language about 
Content-DPR is broken and does not not seem to match Chrome’s actual 
implementation.

So in summary, there is no proper spec, and Chrome has shown willingness to 
have their implementation change faster than even draft spec language in PRs 
can keep up with.

Although the use case seems valid, I don’t think it’s a good idea for WebKit to 
implement the feature in this state. It seems likely to cause interop headaches 
and a need to reverse engineer Blink rather than developing against specs and 
test cases.

I would like to see a clean spec, either separating Content-DPR PRs from 
client-hints or making a standalone spec. And either way it should be fixed to 
match what Chrome has implemented, and have WPT tests that reflect the spec and 
intended behavior.

Regards,
Maciej




___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak  wrote:

> - Is there a specification for Content-DPR? All I could find in bugzila
> and elsewhere is links to <
> https://httpwg.org/http-extensions/client-hints.html>, which does not
> mention Content-DPR at all).
>
In a nutshell, the spec is in transition from IETF to HTML/fetch, and the
client-hints aspects of it are still in progress (unlike conent-dpr, which
is much simpler hasn't changed since introduced). It's documented here:
https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
.
I will answer more about this separately if required, waiting for some info
from the people working on the spec originally.


> - Can you give us some examples of how the CDN would make the decision of
> what density of image to serve without knowing the client’s DPR via
>  client-hints?
>
 Some examples from  https://github.com/eeeps/content-dpr-explainer:
- making decisions by user agent, like choosing to cap certain images for
user-agents known to have smaller displays
- making decisions by traffic/geo, like serving lower-resolution images
when traffic is bogged down
- A client may ask for "placeholder image" (e.g. ?placeholder=true in the
URL), and the CDN would decide whether to serve a lower-quality JPEG or to
scale it down, or do both.

- Is this presuming situations where the CDN serves the images but not the
> markup or CSS (which presumably could be rewritten to account for DPR)?
>
Correct.


> - What happens if Content-DPR is provided, but the markup or CSS has
> conflicting explicit info? For example,  image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x.
> Or the analogous thing with CSS.
>
When image size is decided, css/markup takes precedence, then content DPR,
and finally srcset-selection DPR. This is actually a bonus value of this
feature - if the server happens to serve an image that differs from the
ratio requested in the srcset tag, the intrinsic size match the served
content and not the request (which is more of a recommendation).

- Does Content-DPR have any effect on images where an explicit size is
> provided, either in markup or in CSS? (I’m assuming no.)
>
No, it only has effect on intrinsic size.

~Noam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Maciej Stachowiak]

> On Nov 10, 2019, at 2:13 PM, Maciej Stachowiak  wrote:
> 
> 
> 
>> On Nov 10, 2019, at 12:05 PM, Noam Rosenthal > > wrote:
>> 
>> 
>> 
>> On Sun, Nov 10, 2019 at 9:41 PM Maciej Stachowiak > > wrote:
>> 
>> Is this header useful without the DPR client-hint?
>> Absolutely. 
>> A server/CDN can decide to serve an image with a particular content-dpr 
>> without knowing the client's actual DPR.
> 
> A few more questions (apologies if there are basic things that would be 
> explained by a spec somewhere).

Following up on one of these questions...

> 
> - Is there a specification for Content-DPR? All I could find in bugzila and 
> elsewhere is links to  >, which does not 
> mention Content-DPR at all).

Looks like the ChangeLog links here: 
https://tools.ietf.org/html/draft-ietf-httpbis-client-hints-06#section-3.1.1 
 
which is an expired Internet-Draft for Client Hints. The latest Internet-Draft 
(-07, as well was the in-progress -08) no longer mentions Content-DPR. Is there 
a new place where this is specified?

> - Can you give us some examples of how the CDN would make the decision of 
> what density of image to serve without knowing the client’s DPR via  
> client-hints? 
> - Is this presuming situations where the CDN serves the images but not the 
> markup or CSS (which presumably could be rewritten to account for DPR)?
> - What happens if Content-DPR is provided, but the markup or CSS has 
> conflicting explicit info? For example,  image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x. 
> Or the analogous thing with CSS.
> - Does Content-DPR have any effect on images where an explicit size is 
> provided, either in markup or in CSS? (I’m assuming no.)
> 
> Cheers,
> Maciej
> 
> 
>> 
>> 
>>> On Nov 10, 2019, at 1:16 AM, Noam Rosenthal >> > wrote:
>>> 
>>> Hola
>>> 
>>> I would like to open a discussion that has started a few years back and has 
>>> never reached a consensus: https://bugs.webkit.org/show_bug.cgi?id=145380 
>>> 
>>> 
>>> In a nutshell: content-dpr is a response-only header for images, that 
>>> allows servers at transient layers (CDNs/proxies) to optimize images by 
>>> changing their resolution, allowing the user agents to compensate for the 
>>> change in intrinsic size caused by the resolution change - thus making the 
>>> resolution change transparent to users. It's the header that makes the 
>>> resolution-encoding transparent.
>>> 
>>> The feature is already up and running in chrome since version 67, and is 
>>> used by some CDNs.
>>> 
>>> Since there was lack of consensus in the bug discussion, I wanted to make 
>>> the case for it here, and see if opinions about the subject can be voiced 
>>> before we reach a decision/consensus. I tried to find a good balance 
>>> between being detailed and being concise. 
>>> 
>>> —
>>> 
>>> Players in CDN, proxy and other transient parts of the internet world have 
>>> innovated a lot in the past few years. There are lots of interesting 
>>> companies and competition there. One of the areas of this innovation is in 
>>> optimizing images in a transparent way at transient layers (proxy/CDN). 
>>> This makes a lot of sense considering how much of internet traffic is taken 
>>> by image download.
>>> 
>>> What the research at the CDN companies found, was that modifying resolution 
>>> at the transient layer can have a tremendous effect on 
>>> performance/user-experience balance, for example by serving 
>>> lower-resolution versions of the images when network traffic is costly/slow 
>>> (the exact formula for that is part of where the CDN can innovate). More 
>>> details on that innovation in the links below.
>>> 
>>> The thing is, that modifying image resolution at the CDN/proxy is not 
>>> inherently transparent, due to one reason - layout is affected by the 
>>> image’s intrinsic size, and changing the image’s resolution inadvertently 
>>> changes the image’s intrinsic size. To make this transparent, the user 
>>> agent has to be involved, to compensate for this optimization when reading 
>>> the image’s intrinsic size.
>>> 
>>> The main case against this feature was that image resolution is a feature 
>>> that should be handled at the markup layer and not at the transport layer 
>>> (https://bugs.webkit.org/show_bug.cgi?id=145380#c7 
>>> ).
>>> I would argue that http-headers are not a transport layer (TCP/UDP etc.), 
>>> but rather part of an application-layer protocol that is designed, in part, 
>>> to pass information to the transient layers such as CDNs, and that 
>>> resolution compression is a (new, image-specific) equivalent of 
>>> transfer-encoding. 
>>> 
>>> Al

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Maciej Stachowiak]

> On Nov 10, 2019, at 12:05 PM, Noam Rosenthal  wrote:
> 
> 
> 
> On Sun, Nov 10, 2019 at 9:41 PM Maciej Stachowiak  > wrote:
> 
> Is this header useful without the DPR client-hint?
> Absolutely. 
> A server/CDN can decide to serve an image with a particular content-dpr 
> without knowing the client's actual DPR.

A few more questions (apologies if there are basic things that would be 
explained by a spec somewhere).

- Is there a specification for Content-DPR? All I could find in bugzila and 
elsewhere is links to >, which does not mention 
Content-DPR at all).
- Can you give us some examples of how the CDN would make the decision of what 
density of image to serve without knowing the client’s DPR via  client-hints? 
- Is this presuming situations where the CDN serves the images but not the 
markup or CSS (which presumably could be rewritten to account for DPR)?
- What happens if Content-DPR is provided, but the markup or CSS has 
conflicting explicit info? For example, , and image-2x.jpg is served with a Content-DPR header of 3x. Or the 
analogous thing with CSS.
- Does Content-DPR have any effect on images where an explicit size is 
provided, either in markup or in CSS? (I’m assuming no.)

Cheers,
Maciej


> 
> 
>> On Nov 10, 2019, at 1:16 AM, Noam Rosenthal > > wrote:
>> 
>> Hola
>> 
>> I would like to open a discussion that has started a few years back and has 
>> never reached a consensus: https://bugs.webkit.org/show_bug.cgi?id=145380 
>> 
>> 
>> In a nutshell: content-dpr is a response-only header for images, that allows 
>> servers at transient layers (CDNs/proxies) to optimize images by changing 
>> their resolution, allowing the user agents to compensate for the change in 
>> intrinsic size caused by the resolution change - thus making the resolution 
>> change transparent to users. It's the header that makes the 
>> resolution-encoding transparent.
>> 
>> The feature is already up and running in chrome since version 67, and is 
>> used by some CDNs.
>> 
>> Since there was lack of consensus in the bug discussion, I wanted to make 
>> the case for it here, and see if opinions about the subject can be voiced 
>> before we reach a decision/consensus. I tried to find a good balance between 
>> being detailed and being concise. 
>> 
>> —
>> 
>> Players in CDN, proxy and other transient parts of the internet world have 
>> innovated a lot in the past few years. There are lots of interesting 
>> companies and competition there. One of the areas of this innovation is in 
>> optimizing images in a transparent way at transient layers (proxy/CDN). This 
>> makes a lot of sense considering how much of internet traffic is taken by 
>> image download.
>> 
>> What the research at the CDN companies found, was that modifying resolution 
>> at the transient layer can have a tremendous effect on 
>> performance/user-experience balance, for example by serving lower-resolution 
>> versions of the images when network traffic is costly/slow (the exact 
>> formula for that is part of where the CDN can innovate). More details on 
>> that innovation in the links below.
>> 
>> The thing is, that modifying image resolution at the CDN/proxy is not 
>> inherently transparent, due to one reason - layout is affected by the 
>> image’s intrinsic size, and changing the image’s resolution inadvertently 
>> changes the image’s intrinsic size. To make this transparent, the user agent 
>> has to be involved, to compensate for this optimization when reading the 
>> image’s intrinsic size.
>> 
>> The main case against this feature was that image resolution is a feature 
>> that should be handled at the markup layer and not at the transport layer 
>> (https://bugs.webkit.org/show_bug.cgi?id=145380#c7 
>> ).
>> I would argue that http-headers are not a transport layer (TCP/UDP etc.), 
>> but rather part of an application-layer protocol that is designed, in part, 
>> to pass information to the transient layers such as CDNs, and that 
>> resolution compression is a (new, image-specific) equivalent of 
>> transfer-encoding. 
>> 
>> Also, as a response to the view that this should be part of markup, I would 
>> argue that those transparent decisions by the servers should not concern web 
>> authors at all. It’s an optimization decision to be handled by the servers, 
>> with the user agent doing nothing about it but allow that decision to be 
>> transparent in terms of layout (the same way gzip and cache-control are 
>> transparent).
>> 
>> What is offered here is a win-win in terms of delivering images, and would 
>> allow webkit-browser users to enjoy some of the innovations in the image 
>> delivery world without modifying their HTML content and without concern of 
>> breaking their lay

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Sun, Nov 10, 2019 at 9:41 PM Maciej Stachowiak  wrote:

>
> Is this header useful without the DPR client-hint?
>
Absolutely.
A server/CDN can decide to serve an image with a particular content-dpr
without knowing the client's actual DPR.


> On Nov 10, 2019, at 1:16 AM, Noam Rosenthal  wrote:
>
> Hola
>
> I would like to open a discussion that has started a few years back and
> has never reached a consensus:
> https://bugs.webkit.org/show_bug.cgi?id=145380
>
> In a nutshell: content-dpr is a response-only header for images, that
> allows servers at transient layers (CDNs/proxies) to optimize images by
> changing their resolution, allowing the user agents to compensate for the
> change in intrinsic size caused by the resolution change - thus making the
> resolution change transparent to users. It's the header that makes the
> resolution-encoding transparent.
>
> The feature is already up and running in chrome since version 67, and is
> used by some CDNs.
>
> Since there was lack of consensus in the bug discussion, I wanted to make
> the case for it here, and see if opinions about the subject can be voiced
> before we reach a decision/consensus. I tried to find a good balance
> between being detailed and being concise.
>
> —
>
> Players in CDN, proxy and other transient parts of the internet world have
> innovated a lot in the past few years. There are lots of interesting
> companies and competition there. One of the areas of this innovation is in
> optimizing images in a transparent way at transient layers (proxy/CDN).
> This makes a lot of sense considering how much of internet traffic is taken
> by image download.
>
> What the research at the CDN companies found, was that modifying
> resolution at the transient layer can have a tremendous effect on
> performance/user-experience balance, for example by serving
> lower-resolution versions of the images when network traffic is costly/slow
> (the exact formula for that is part of where the CDN can innovate). More
> details on that innovation in the links below.
>
> The thing is, that modifying image resolution at the CDN/proxy is not
> inherently transparent, due to one reason - layout is affected by the
> image’s intrinsic size, and changing the image’s resolution inadvertently
> changes the image’s intrinsic size. To make this transparent, the user
> agent has to be involved, to compensate for this optimization when reading
> the image’s intrinsic size.
>
> The main case against this feature was that image resolution is a feature
> that should be handled at the markup layer and not at the transport layer (
> https://bugs.webkit.org/show_bug.cgi?id=145380#c7).
> I would argue that http-headers are not a transport layer (TCP/UDP etc.),
> but rather part of an application-layer protocol that is designed, in part,
> to pass information to the transient layers such as CDNs, and that
> resolution compression is a (new, image-specific) equivalent of
> transfer-encoding.
>
> Also, as a response to the view that this should be part of markup, I
> would argue that those transparent decisions by the servers should not
> concern web authors at all. It’s an optimization decision to be handled by
> the servers, with the user agent doing nothing about it but allow that
> decision to be transparent in terms of layout (the same way gzip and
> cache-control are transparent).
>
> What is offered here is a win-win in terms of delivering images, and would
> allow webkit-browser users to enjoy some of the innovations in the image
> delivery world without modifying their HTML content and without concern of
> breaking their layouts. Less battery and data used for downloading big
> images at certain situations, for example.
>
> I hope this provides enough background without being too tedious. I would
> be happy to shed more light on whatever is missing in this longish essay :)
>
> Additional info:
> https://github.com/eeeps/content-dpr-explainer
>
> https://cloudinary.com/blog/client_hints_and_responsive_images_what_changed_in_chrome_67
> https://www.chromestatus.com/metrics/feature/timeline/popularity/906
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Maciej Stachowiak]

Is this header useful without the DPR client-hint?

> On Nov 10, 2019, at 1:16 AM, Noam Rosenthal  wrote:
> 
> Hola
> 
> I would like to open a discussion that has started a few years back and has 
> never reached a consensus: https://bugs.webkit.org/show_bug.cgi?id=145380 
> 
> 
> In a nutshell: content-dpr is a response-only header for images, that allows 
> servers at transient layers (CDNs/proxies) to optimize images by changing 
> their resolution, allowing the user agents to compensate for the change in 
> intrinsic size caused by the resolution change - thus making the resolution 
> change transparent to users. It's the header that makes the 
> resolution-encoding transparent.
> 
> The feature is already up and running in chrome since version 67, and is used 
> by some CDNs.
> 
> Since there was lack of consensus in the bug discussion, I wanted to make the 
> case for it here, and see if opinions about the subject can be voiced before 
> we reach a decision/consensus. I tried to find a good balance between being 
> detailed and being concise. 
> 
> —
> 
> Players in CDN, proxy and other transient parts of the internet world have 
> innovated a lot in the past few years. There are lots of interesting 
> companies and competition there. One of the areas of this innovation is in 
> optimizing images in a transparent way at transient layers (proxy/CDN). This 
> makes a lot of sense considering how much of internet traffic is taken by 
> image download.
> 
> What the research at the CDN companies found, was that modifying resolution 
> at the transient layer can have a tremendous effect on 
> performance/user-experience balance, for example by serving lower-resolution 
> versions of the images when network traffic is costly/slow (the exact formula 
> for that is part of where the CDN can innovate). More details on that 
> innovation in the links below.
> 
> The thing is, that modifying image resolution at the CDN/proxy is not 
> inherently transparent, due to one reason - layout is affected by the image’s 
> intrinsic size, and changing the image’s resolution inadvertently changes the 
> image’s intrinsic size. To make this transparent, the user agent has to be 
> involved, to compensate for this optimization when reading the image’s 
> intrinsic size.
> 
> The main case against this feature was that image resolution is a feature 
> that should be handled at the markup layer and not at the transport layer 
> (https://bugs.webkit.org/show_bug.cgi?id=145380#c7 
> ).
> I would argue that http-headers are not a transport layer (TCP/UDP etc.), but 
> rather part of an application-layer protocol that is designed, in part, to 
> pass information to the transient layers such as CDNs, and that resolution 
> compression is a (new, image-specific) equivalent of transfer-encoding. 
> 
> Also, as a response to the view that this should be part of markup, I would 
> argue that those transparent decisions by the servers should not concern web 
> authors at all. It’s an optimization decision to be handled by the servers, 
> with the user agent doing nothing about it but allow that decision to be 
> transparent in terms of layout (the same way gzip and cache-control are 
> transparent).
> 
> What is offered here is a win-win in terms of delivering images, and would 
> allow webkit-browser users to enjoy some of the innovations in the image 
> delivery world without modifying their HTML content and without concern of 
> breaking their layouts. Less battery and data used for downloading big images 
> at certain situations, for example.
> 
> I hope this provides enough background without being too tedious. I would be 
> happy to shed more light on whatever is missing in this longish essay :)
> 
> Additional info:
> https://github.com/eeeps/content-dpr-explainer 
> 
> https://cloudinary.com/blog/client_hints_and_responsive_images_what_changed_in_chrome_67
>  
> 
> https://www.chromestatus.com/metrics/feature/timeline/popularity/906 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev