LGTM2 On Wed, Feb 14, 2024 at 7:11 PM Nidhi Jaju <nidhij...@chromium.org> wrote:
> > > On Thu, Feb 15, 2024 at 12:50 AM David Benjamin <david...@chromium.org> > wrote: > >> On Wed, Feb 14, 2024 at 9:20 AM Yoav Weiss (@Shopify) < >> yoavwe...@chromium.org> wrote: >> >>> LGTM1 >>> >>> On Wednesday, February 14, 2024 at 2:36:10 AM UTC+1 Nidhi Jaju wrote: >>> >>> On Wed, Feb 14, 2024 at 2:48 AM James Hartig <fastest...@gmail.com> >>> wrote: >>> >>> My employer ran into the window size during our pre-production >>> validation and it was difficult to debug since it was working in cURL, the >>> zstd CLI, and only presented itself on certain URLs. I appreciate Nidhi >>> responding to our issue so quickly and updating Chrome to have a more >>> helpful error message in the future. The Go package we use already updated >>> their default <https://github.com/klauspost/compress/pull/913> to 8MB >>> (without any awareness to Chrome's size) which should help future users of >>> that package but there might be other packages out there that might not >>> have a low enough default. The updated Chrome error message will help but >>> only if you run into that error message when testing; which might not if >>> you happen to be testing with small responses. I'm not sure where >>> developers should be looking to be aware of the window size. Does it make >>> sense to mention in the Chrome Status entry? If the spec is updated that >>> might be good enough but I just wanted to discuss other avenues that might >>> be more developer-aware. >>> >>> >>> Thank you, I've included these details about the window size limits >>> under the "Interoperability and Compatibility Risks" section in the >>> ChromeStatus entry. >>> >>> On Tue, Feb 13, 2024 at 6:43 PM Yoav Weiss (@Shopify) < >>> yoavwe...@chromium.org> wrote: >>> >>> >>> >>> On Tue, Feb 13, 2024 at 9:29 AM Nidhi Jaju <nidhij...@chromium.org> >>> wrote: >>> >>> >>> >>> On Tue, Feb 13, 2024 at 4:18 PM Yoav Weiss (@Shopify) < >>> yoavwe...@chromium.org> wrote: >>> >>> Thanks for working on this!! This is extremely exciting! >>> >>> On Tue, Feb 13, 2024 at 1:11 AM Nidhi Jaju <nidhij...@chromium.org> >>> wrote: >>> >>> Contact emails >>> >>> nidhij...@chromium.org >>> >>> Explainer >>> >>> https://docs.google.com/document/d/1aDyUw4mAzRdLyZyXpVgWvO- >>> eLpc4ERz7I_7VDIPo9Hc/edit?usp=sharing >>> >>> Specification >>> >>> https://datatracker.ietf.org/doc/html/rfc8878 >>> >>> Design docs >>> >>> https://docs.google.com/document/d/14dbzMpsYPfkefAJos124uPrlkvW7j >>> yPJhzjujSWws2k/edit?usp=sharing >>> >>> Summary >>> >>> Zstandard, or “zstd”, is a data compression mechanism described in >>> RFC8878. It is a fast lossless compression algorithm, targeting real-time >>> compression scenarios at zlib-level and better compression ratios. The >>> "zstd" token was added as an IANA-registered Content-Encoding token as per >>> https://datatracker.ietf.org/doc/html/rfc8878#name-content-encoding. >>> >>> Adding support for "zstd" as a Content-Encoding will help load pages >>> faster and use less bandwidth, and spend less time and CPU/power on >>> compression on our servers, resulting in reduced server costs. >>> >>> Blink component >>> >>> Internals>Network >>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Internals%3ENetwork> >>> >>> TAG review >>> >>> https://github.com/w3ctag/design-reviews/issues/930 >>> >>> TAG review status >>> >>> Pending >>> >>> Risks >>> Interoperability and Compatibility >>> >>> Servers that have a broken implementation of zstd might exist, but the >>> risk of this is small. Additionally, middleware and middleboxes like virus >>> checkers that intercept HTTPS connections might not support zstd, but might >>> fail to remove it from the Accept-Encoding header in the request. >>> >>> Another known risk is interoperability between clients that support zstd >>> regarding window frame sizes. In Chrome, we limit the window frame size to >>> 8MB to prevent excessive memory usage, but this limit does not exist in >>> curl and when using zstd directly. We have seen very few sites that use a >>> window size larger than 8MB which causes decoding errors, but we have added >>> new net error codes and debugging messages to help them understand what to >>> do in this situation. >>> >>> >>> I know we discussed >>> <https://w3c.github.io/web-performance/meetings/2023/2023-09-TPAC/index.html#h.xn2d3li0b8op> >>> this at length at the WebPerfWG. >>> Can you summarize developments and/or findings since that discussion on >>> that front? >>> Should we expect the default output of CLI tools to be compatible with >>> what we want to ship here? >>> Should we expect interoperability between Chromium and e.g. curl? >>> >>> >>> We've been discussing it with the zstd team at Meta at >>> https://github.com/facebook/zstd/issues/2713. The plan is to take it to >>> the HTTP WG at the IETF and either file an errata or publish a new document >>> with more strict window size guidelines. The zstd CLI tool currently >>> supports up to 8MB as a default, so the same limit. The library will use >>> 128MB by default, however, and Curl currently supports up to 128MB windows. >>> We expect those defaults to change to match any spec changes. In practice, >>> we've seen very limited reports of sites running into this limit, and we've >>> added helpful messages in Chromium to guide anyone who does run into it. >>> >>> >>> Thanks! Pushing that limit into the standard and having curl (and other >>> tools) follow that makes sense and seems important. >>> >>> Thinking out loud, the main risk here is for folks to be testing their >>> content outside of Chromium (e.g. with curl) and then have that content >>> break in Chromium. At the same time if content is tested in Chromium, it >>> will work in another client that supports larger windows. >>> So the (seemingly small) risk here is one we take on ourselves, rather >>> than risk we externalize on the ecosystem. >>> >>> >>> Yes, that sounds right. We'll continue to push to standardize this >>> behavior across the ecosystem. >>> >>> >>> >>> >>> >>> >>> >>> >>> Gecko: Positive (https://github.com/mozilla/ >>> standards-positions/issues/775) >>> >>> WebKit: Positive (https://github.com/WebKit/ >>> standards-positions/issues/168) >>> >>> Web developers: Positive (https://crbug.com/1246971) Meta (Yann and >>> Felix) and Akamai (Nic) are positive about zstd content-encoding on the >>> browser. Meta has collaborated with us to improve the compression ratios >>> for Meta origins during the experiment and is seeing positive user-level >>> results. Alibaba is also supportive of shipping zstd support as they saw >>> massive savings on their origins in terms of server CPU cost. >>> >>> Other signals: >>> >>> Ergonomics >>> >>> While both Zstandard and Brotli are clear wins over gzip >>> content-encoding, which of Zstandard or Brotli to use depends on many >>> factors, and site authors may need to experiment to identify the optimal >>> choice for their content. >>> >>> Zstandard uses more memory for decompression than gzip. However, this is >>> also true for Brotli, and we haven't seen any problems in practice. >>> >>> Activation >>> >>> The "zstd" Content-Encoding is not as widely supported by HTTP servers >>> as gzip. Of the top 5 web servers, Nginx has a third-party module, which >>> should also work for OpenResty (untested). Apache, IIS, and LiteSpeed >>> appear to have no support. Explicit server support is often only necessary >>> for dynamic content. For static (pre-compressed) content, Zstandard can >>> often be supported just by configuration. >>> >>> Only one public CDN is known to be able to compress Zstandard itself, >>> and some CDN's may require custom configuration to pass-through Zstandard >>> correctly. >>> >>> Zstd support is not particularly difficult to implement for a server >>> that already implements multiple content encodings. The C implementation >>> has a straightforward API and there are implementations for many other >>> languages. There is also a lively community of Zstandard enthusiasts which >>> should help accelerate adoption. >>> >>> Security >>> >>> CRIME <https://en.wikipedia.org/wiki/CRIME> and BREACH >>> <https://en.wikipedia.org/wiki/BREACH> mean that the resource being >>> compressed can be considered readable by the document deploying them. That >>> is bad if any of them contains information that the document cannot already >>> obtain by other means. An attacker may provide correctly formed compressed >>> frames with unreasonable memory requirements, and dictionaries may interact >>> unexpectedly with a decoder, leading to possible memory or other >>> resource-exhaustion attacks. It is possible to store arbitrary user >>> metadata in skippable frames, so they can be used as a watermark to track >>> the path of the compressed payload. It is important to note that these >>> concerns apply to all compression formats, not just zstd. >>> >>> To mitigate these risks, similar to Brotli, we'll be advertising support >>> for "zstd" encoding only if transferred data is opaque to proxies, to >>> ensure that resources don't contain private data that the origin cannot >>> read otherwise. >>> >>> >>> I'm not sure what that means. Can you elaborate on that? >>> >>> >>> This essentially means that, like Brotli, Zstd is only available in >>> secure contexts >>> <https://source.chromium.org/chromium/chromium/src/+/main:net/http/http_request_headers.cc;l=284-290;drc=4787fce6c51383f5631643ac3d14cc512d656de6> >>> i.e. over https. >>> >>> >>> Limiting zstd support to secure contexts makes perfect sense. However I >>> believe the reason we're doing that for brotli is more around compatibility >>> concerns with old network-based proxies >>> <https://issues.chromium.org/issues/40930163> that aren't ready for >>> non-gzip content-encodings. >>> I don't think secure contexts do much to protect against BREACH if >>> attackers can control parts of the response. At the same time, I don't know >>> that we're doing anything on that front for other compression formats, so >>> that seems fine. >>> >> >> Right, secure contexts don't magically make dangerous features safe. The >> *only* thing secure contexts do is make the name in the URL bar >> meaningful. The user may still be talking to evil.example. >> >> It sounds like there are more risks discussed here than BREACH, so I >> think we need to examine them separately: >> >> 1. Information leaks when you compress together attacker-controlled data >> and secret data. (BREACH) >> 2. DoS risks from the decompressor >> 3. Watermarking from user-specific encodings of the resource >> >> For BREACH, the description of the document not being able to read it >> confused me a little. When you compress something, the *length* of the >> compressed resource, even when encrypted, gets leaked to all manner of >> attackers via all manner of ways. I'm guessing the reference to the >> document is that resource timing APIs allow the document to learn the >> length of resources it otherwise cannot read? That is one attack vector >> (not at all mitigated by secure contexts), but there are others. >> Ultimately, BREACH means the server cannot *just* transparently compress >> every resource it sends. In particular, any kind of dynamic HTML resource >> will likely contain some attacker controlled strings. >> >> That said, the mitigation is mostly on the server to do. Once the >> resource gets to us, the leak has already happened. The only connection to >> proxies, and where we can do something on the client, is that sometimes >> proxies will try to transparently compress all HTTP resources >> indiscriminately. If that proxy is part of the network path and not the >> site, it has no hope of mitigating this. So being opaque to proxies is good >> and cuts out that minor component of the problem, but doesn't actually >> address the broader issue. It's just fine because the broader issue is for >> the server to address. (Though it means that our documentation to use it >> should mention the server's responsibility here!) >> > > Thank you for the additional discussion about the different security > risks. I've added a note about the server's responsibility to the > ChromeStatus entry to take care with including attacker-controlled data in > compressed content. > > >> >> For DoS risks, secure contexts also don't do anything. We assume that the >> attacker can direct the user to visit any website under their control, so >> users could well visit https://evil.example and securely get a >> DoS-triggering payload from it. As decompression happens in the network >> service, shared across sites, that DoS would impact other sites too. So, in >> order to deploy zstd or any such compression scheme, we need to mitigate >> DoS attacks directly, usually by applying limits. It sounds like you all >> have applied a frame size limit? Is that sufficient to avoid DoS, or are >> the other avenues for a zstd decompression to consume excessive resources? >> > > Yes, we added a window size limit of 8MB, which means that Chromium will > use a maximum of 8MB memory buffer to decompress frames to protect it from > unreasonable requirements. In addition, for zip bomb-like attacks, Chromium > doesn't decompress faster than the renderer consumes data, so we won't > accumulate excessive amounts of data in the network process. > > >> Finally, the watermarking concerns also aren't mitigated by secure >> contexts, but I think that's fine. This doesn't really apply to the web's >> security model because we already assume the resource may be user-specific >> in all manner of ways. (I mean, it can contain a Set-Cookie header!) >> Rather, when we want two contexts to be uncorrelated, we control whether >> they can communicate at all, rather than making assumptions on the encoding >> mechanism. (Network state partitioning, cookie controls, etc.) >> > > Agreed, for the content itself, we’ll continue to rely on the existing > partitioning present in Chromium, as with other content encodings. > > >> >> Adding zstd to third_party/ in Chromium adds a large new code surface >>> that processes untrusted data, which inevitably brings risks of new >>> security holes. However, this is mitigated by the extensive fuzzing and >>> security analysis done on zstd by Google and other community members. >>> >>> Furthermore, zstd is implemented in C, which is not a memory-safe >>> language, and the network service is not yet sandboxed on all platforms. >>> >>> WebView application risks >>> >>> Does this intent deprecate or change behavior of existing APIs, such >>> that it has potentially high risk for Android WebView-based applications? >>> >>> Apps which use a WebView to display content from Meta's servers will >>> suddenly start using Zstandard. Since we've already extensively tested our >>> implementation against Meta's servers in Chrome, no problems are expected. >>> There is a killswitch. No special treatment should be needed. >>> >>> >>> Debuggability >>> >>> No special support needed. >>> >>> Zstd content-encoding support is exposed to the devtools protocol, so >>> developers are able to override it and view the headers from the inspector. >>> >>> A new net error has been added for decoding errors related to window >>> frame size. >>> >>> Will this feature be supported on all six Blink platforms (Windows, Mac, >>> Linux, ChromeOS, Android, and Android WebView)? >>> >>> Yes >>> >>> Is this feature fully tested by web-platform-tests >>> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >>> ? >>> >>> Yes (https://wpt.fyi/results/fetch/content-encoding/zstd >>> <https://wpt.fyi/results/fetch/content-encoding/zstd?label=experimental&label=master&aligned> >>> ) >>> >>> Flag name on chrome://flags >>> >>> enable-zstd-content-encoding >>> >>> Finch feature name >>> >>> ZstdContentEncoding >>> >>> Requires code in //chrome? >>> >>> False >>> >>> Tracking bug >>> >>> https://bugs.chromium.org/p/chromium/issues/detail?id=1246971 >>> >>> Launch bug >>> >>> https://launch.corp.google.com/launch/4266275 >>> >>> Measurement >>> >>> https://chromestatus.com/metrics/feature/timeline/popularity/4629 >>> >>> Adoption plan >>> >>> In our experimental group, around 1% of responses use "zstd" >>> content-encoding. Given the significant benefits of zstandard over gzip, >>> we'd like to see it increase to 10% within 2 years. >>> >>> Estimated milestones >>> >>> Shipping on desktop >>> >>> 123 >>> >>> DevTrial on desktop >>> >>> 117 >>> >>> Shipping on Android >>> >>> 123 >>> >>> DevTrial on Android >>> >>> 117 >>> >>> Shipping on WebView >>> >>> 123 >>> >>> >>> Anticipated spec changes >>> >>> Open questions about a feature may be a source of future web compat or >>> interop issues. Please list open issues (e.g. links to known github issues >>> in the project for the feature specification) whose resolution may >>> introduce web compat/interop risk (e.g., changing to naming or structure of >>> the API in a non-backward-compatible way). >>> >>> The current standard, RFC8878, doesn't require a limit on the window >>> size used by HTTP servers when compressing Zstandard. An update of some >>> form will be needed to ensure interoperability. >>> >>> Link to entry on the Chrome Platform Status >>> >>> https://chromestatus.com/feature/6186023867908096 >>> >>> Links to previous Intent discussions >>> >>> Intent to Prototype: https://groups.google.com/a/ >>> chromium.org/g/blink-dev/c/GDsI0Hw-jYk/m/Yc5QZWD-AwAJ >>> >>> Intent to Experiment: https://groups.google.com/a/ >>> chromium.org/g/blink-dev/c/I6IWfl95gRU >>> >>> >>> This intent message was generated by Chrome Platform Status >>> <https://chromestatus.com/>. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+unsubscr...@chromium.org. >>> To view this discussion on the web visit https://groups.google.com/a/ >>> chromium.org/d/msgid/blink-dev/CAMZNYAN7VRca4VfRqP7pi% >>> 2BnqwDuor4ZVjF9yDNH1mZcXteQURw%40mail.gmail.com >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMZNYAN7VRca4VfRqP7pi%2BnqwDuor4ZVjF9yDNH1mZcXteQURw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+unsubscr...@chromium.org. >>> To view this discussion on the web visit >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f823c2bc-f224-4ff7-9f78-e9eba9a4949cn%40chromium.org >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f823c2bc-f224-4ff7-9f78-e9eba9a4949cn%40chromium.org?utm_medium=email&utm_source=footer> >>> . >> >> -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to blink-dev+unsubscr...@chromium.org. > To view this discussion on the web visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMZNYAMDwUHi%3DLzPafbdg5CgQm4t27ZFLg3am53-no0D3Fg%2B%2BQ%40mail.gmail.com > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMZNYAMDwUHi%3DLzPafbdg5CgQm4t27ZFLg3am53-no0D3Fg%2B%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-U7PCX0Yms-yTZo3uX2zRzjfV3q0nJ9%3DvG2oiYMQx4Nw%40mail.gmail.com.
