[
https://issues.apache.org/jira/browse/YUNIKORN-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wilfred Spiegelenburg updated YUNIKORN-3341:
--------------------------------------------
Labels: pull-request-available release-notes (was: pull-request-available)
> gzip middleware double-compresses responses that handlers already encoded
> -------------------------------------------------------------------------
>
> Key: YUNIKORN-3341
> URL: https://issues.apache.org/jira/browse/YUNIKORN-3341
> Project: Apache YuniKorn
> Issue Type: Bug
> Affects Versions: 1.9.0
> Reporter: Ilia Lazebnik
> Assignee: Ilia Lazebnik
> Priority: Major
> Labels: pull-request-available, release-notes
> Fix For: 1.10.0
>
>
> The gzip middleware on the REST router compresses any response whose buffered
> body exceeds \{{minCompressionSize}}, without checking whether the wrapped
> handler had already encoded the body itself.
> {\{/ws/v1/metrics}} is served by \{{promhttp.Handler()}}, which performs its
> own gzip content negotiation: when the client sends \{{Accept-Encoding:
> gzip}} it compresses the payload and sets \{{Content-Encoding: gzip}}. The
> middleware then sees the same \{{Accept-Encoding}}, buffers promhttp's
> already-gzipped bytes, finds them over the threshold, and compresses a second
> time. \{{switchToGzip}} uses \{{Header().Set}}, so the response still
> advertises a single \{{Content-Encoding: gzip}} while carrying a doubly
> encoded body.
> Clients decode once and get gzip framing instead of the payload. For
> Prometheus this means every scrape of \{{/ws/v1/metrics}} fails to parse and
> the target is marked down, so scheduler metrics are lost entirely for any
> scraper that advertises gzip support — which is the default.
> Reproduced against a running 1.9.0 scheduler:
> {code}
> $ curl -s -D- -o body.bin http://scheduler:9080/ws/v1/metrics
> 200 OK, 69714 bytes, "# HELP go_gc_duration_seconds ..."
> $ curl -s -D- -o body.gz -H 'Accept-Encoding: gzip'
> http://scheduler:9080/ws/v1/metrics
> 200 OK, Content-Encoding: gzip, 8839 bytes
> $ gunzip -c body.gz | file -
> /dev/stdin: gzip compressed data # still gzip after one decode
> $ gunzip -c body.gz | gunzip -c | head -1
> # HELP go_gc_duration_seconds ... # payload only after two
> {code}
> This is a regression in 1.9.0: \{{pkg/webservice/gzip.go}} is new in that
> release and \{{StartWebApp}} changed from \{{Handler: router}} to \{{Handler:
> compressResponse(router)}}. The route itself and port 9080 are unchanged from
> 1.8.0.
> *Proposed fix*
> Treat a \{{Content-Encoding}} set by the handler as a signal that the body is
> already encoded, and pass the response through untouched. Metrics responses
> stay compressed — promhttp still gzips them, just once — and handlers that do
> not set \{{Content-Encoding}} are unaffected.
> Fixing \{{getMetrics}} alone (via \{{promhttp.HandlerOpts{DisableCompression:
> true}}}) would patch only the one endpoint that self-encodes today; the
> middleware would still corrupt any future handler that does the same.
> PR: https://github.com/apache/yunikorn-core/pull/1115
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
