Re: compression in defaults happens twice with 1.9.0
Le 07/01/2019 à 22:08, PiBa-NL a écrit : Hi Christopher, Op 7-1-2019 om 16:32 schreef Christopher Faulet: Le 06/01/2019 à 16:22, PiBa-NL a écrit : Hi List, Using both 1.9.0 and 2.0-dev0-909b9d8 compression happens twice when configured in defaults. This was noticed by user walle303 on IRC. Seems like a bug to me as 1.8.14 does not show this behavior. Attached a little regtest that reproduces the issue. Can someone take a look, thanks in advance. Hi Pieter, Here is the patch that should fix this issue. Could you confirm please ? Works for me. Thanks! Thanks, now merged! -- Christopher Faulet
Re: compression in defaults happens twice with 1.9.0
Hi Christopher, Op 7-1-2019 om 16:32 schreef Christopher Faulet: Le 06/01/2019 à 16:22, PiBa-NL a écrit : Hi List, Using both 1.9.0 and 2.0-dev0-909b9d8 compression happens twice when configured in defaults. This was noticed by user walle303 on IRC. Seems like a bug to me as 1.8.14 does not show this behavior. Attached a little regtest that reproduces the issue. Can someone take a look, thanks in advance. Hi Pieter, Here is the patch that should fix this issue. Could you confirm please ? Thanks Works for me. Thanks! Regards, PiBa-NL (Pieter)
Re: compression in defaults happens twice with 1.9.0
Le 06/01/2019 à 16:22, PiBa-NL a écrit :
Hi List,
Using both 1.9.0 and 2.0-dev0-909b9d8 compression happens twice when
configured in defaults.
This was noticed by user walle303 on IRC.
Seems like a bug to me as 1.8.14 does not show this behavior. Attached a
little regtest that reproduces the issue.
Can someone take a look, thanks in advance.
Hi Pieter,
Here is the patch that should fix this issue. Could you confirm please ?
Thanks
--
Christopher Faulet
>From ff3c04e40ab0f4a7176ef25835b40f0d068150cf Mon Sep 17 00:00:00 2001
From: Christopher Faulet
Date: Mon, 7 Jan 2019 14:41:59 +0100
Subject: [PATCH] BUG/MINOR: compression: Disable it if another one is already
in progress
Since the commit 9666720c8 ("BUG/MEDIUM: compression: Use the right buffer
pointers to compress input data"), the compression can be done twice. The first
time on the frontend and the second time on the backend. This may happen by
configuring the compression in a default section.
To fix the bug, when the response is checked to know if it should be compressed
or not, if the flag HTTP_MSGF_COMPRESSING is set, the compression is not
performed. It means it is already handled by a previous compression filter.
Thanks to Pieter (PiBa-NL) to report this bug.
This patch must be backported to 1.9.
---
src/flt_http_comp.c | 9 -
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/src/flt_http_comp.c b/src/flt_http_comp.c
index 380366b06..f8fbf4c9d 100644
--- a/src/flt_http_comp.c
+++ b/src/flt_http_comp.c
@@ -808,6 +808,10 @@ http_select_comp_reshdr(struct comp_state *st, struct stream *s, struct http_msg
if (st->comp_algo == NULL)
goto fail;
+ /* compression already in progress */
+ if (msg->flags & HTTP_MSGF_COMPRESSING)
+ goto fail;
+
/* HTTP < 1.1 should not be compressed */
if (!(msg->flags & HTTP_MSGF_VER_11) || !(txn->req.flags & HTTP_MSGF_VER_11))
goto fail;
@@ -902,6 +906,10 @@ htx_select_comp_reshdr(struct comp_state *st, struct stream *s, struct http_msg
if (st->comp_algo == NULL)
goto fail;
+ /* compression already in progress */
+ if (msg->flags & HTTP_MSGF_COMPRESSING)
+ goto fail;
+
/* HTTP < 1.1 should not be compressed */
if (!(msg->flags & HTTP_MSGF_VER_11) || !(txn->req.flags & HTTP_MSGF_VER_11))
goto fail;
@@ -1391,7 +1399,6 @@ check_implicit_http_comp_flt(struct proxy *proxy)
fconf->conf = NULL;
fconf->ops = _ops;
LIST_ADDQ(>filter_configs, >list);
-
end:
return err;
}
--
2.20.1
compression in defaults happens twice with 1.9.0
Hi List,
Using both 1.9.0 and 2.0-dev0-909b9d8 compression happens twice when
configured in defaults.
This was noticed by user walle303 on IRC.
Seems like a bug to me as 1.8.14 does not show this behavior. Attached a
little regtest that reproduces the issue.
Can someone take a look, thanks in advance.
Regards,
PiBa-NL (Pieter)
s1 0.0
txresp|!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_
s1 0.0 txresp|"#$%&'()*+,-./0123456789:;<=>?@ABCD
*** s1 0.0 shutting fd 4
** s1 0.0 Ending
*** h1 0.0 debug|:b1.srvrep[000a:adfd]: HTTP/1.1 200 OK
*** h1 0.0 debug|:b1.srvhdr[000a:adfd]: Content-Length: 100
*** h1 0.0 debug|:b1.srvcls[000a:adfd]
c1 0.0 rxhdr|HTTP/1.1 200 OK\r
c1 0.0 rxhdr|Content-Encoding: gzip\r
c1 0.0 rxhdr|Transfer-Encoding: chunked\r
c1 0.0 rxhdr|Co57\r
c1 0.0 rxhdr|\037\213\010
c1 0.0 rxhdrlen = 78
c1 0.0 http[ 0] |HTTP/1.1
c1 0.0 http[ 1] |200
c1 0.0 http[ 2] |OK
c1 0.0 http[ 3] |Content-Encoding: gzip
c1 0.0 http[ 4] |Transfer-Encoding: chunked
c1 0.0 http[ 5] |Co57
c1 0.0 http[ 6] |\037\213\010
c1 10.2 HTTP rx timeout (fd:7 1 ms)
# Checks compression defined in defaults doesnt happen twice
varnishtest "Compression in defaults"
feature ignore_unknown_macro
server s1 {
rxreq
txresp -bodylen 100
} -start
haproxy h1 -conf {
defaults
mode http
compression algo gzip
frontend fe1
bind "fd@${fe_1}"
default_backend b1
backend b1
server srv1 ${s1_addr}:${s1_port}
} -start
client c1 -connect ${h1_fe_1_sock} {
txreq -url "/" -hdr "Accept-Encoding: gzip"
rxresp
expect resp.status == 200
expect resp.http.content-encoding == "gzip"
expect resp.http.transfer-encoding == "chunked"
gunzip
expect resp.bodylen == 100
} -run
server s1 -wait
