Re: compression and q-values?

[Willy Tarreau]

Hi Jay,

On Fri, Mar 14, 2014 at 01:51:13PM -0700, Jay Christopherson wrote:
 It appears that HAproxy is not honoring q-values in the Accept-Encoding
 header, when requesting compression.

Indeed. The basic fix is not hard to do, but we're normally supposed to
also support q=0, and ideally to respect the client's choice, so that
becomes slightly more complicated. I started doing it when reading your
e-mail and stopped since I don't have the time right now.

If someone wants to take a look at it, it's in proto_http.c, function
select_compression_request_header(). We need to cut the accept-encoding
token on the semi-colon, check if the token is known, then if a q-value
is present, either it's zero and we must skip it, or we must compare it
to the previous q-value and remember it only if it's strictly greater
than any previous one. If no q-value is specified, it equals 1.000.

I think we can increase the ratio of compressed responses using this
because it's very likely that some browsers send q=something that
is not matched.

Willy

Re: compression and q-values?

[Willy Tarreau]

Hi again,

On Wed, Mar 19, 2014 at 07:32:30AM +0100, Willy Tarreau wrote:
 If someone wants to take a look at it, it's in proto_http.c, function
 select_compression_request_header(). We need to cut the accept-encoding
 token on the semi-colon, check if the token is known, then if a q-value
 is present, either it's zero and we must skip it, or we must compare it
 to the previous q-value and remember it only if it's strictly greater
 than any previous one. If no q-value is specified, it equals 1.000.

Finally I did it this morning. It was not very long and makes us more
compliant to the spec. It's merged in the git tree.

Regards,
Willy

compression and q-values?

[Jay Christopherson]

It appears that HAproxy is not honoring q-values in the Accept-Encoding
header, when requesting compression.

When I run a curl command with Accept-Encoding:gzip (no q-value), I get
back an expected response:

#curl -svo /dev/null
http://something/js/global-main-7b2ca6b.jshttp://origin.cdn.creativelive.com/js/global-main-7b2ca6b.js
-HAccept-Encoding: gzip

*Date: Fri, 14 Mar 2014 19:20:28 GMT*
*Content-Type: application/x-javascript*
*Last-Modified: Fri, 14 Mar 2014 05:35:33 GMT*
*ETag: 53229525-56f6d*
*Access-Control-Allow-Origin: **
*Access-Control-Allow-Credentials: true*
*Access-Control-Allow-Methods: GET, POST, OPTIONS*
*Access-Control-Allow-Headers:
DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type*
*Accept-Ranges: bytes*
*Transfer-Encoding: chunked*
*Content-Encoding: gzip*

However, if I run a 'curl' command with a q-value on the Accept-Encoding
parameter, I get:

# curl -svo /dev/null
http://something/js/global-main-7b2ca6b.jshttp://origin.cdn.creativelive.com/js/global-main-7b2ca6b.js
-HAccept-Encoding: gzip;q=1.000

*Date: Fri, 14 Mar 2014 19:47:10 GMT*

*Content-Type: application/x-javascript*

*Content-Length: 356205*

*Last-Modified: Fri, 14 Mar 2014 05:35:30 GMT*

*ETag: 53229525-56f6d*
*Access-Control-Allow-Origin: **
*Access-Control-Allow-Credentials: true*
*Access-Control-Allow-Methods: GET, POST, OPTIONS*
*Access-Control-Allow-Headers:
DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type*
*Accept-Ranges: bytes*


On HAproxy, I have this setup on my frontend:

frontend web
mode http
bind *:80
bind *:443 ssl crt /etc/haproxy/certs/web.pem crt
/etc/haproxy/certs/
log-format %ST|%fp|%CC|%Ts|%ci|%r|%hr
capture cookie cc= len 63
option http-server-close
option forwardfor
capture request header User-Agent len 128
capture request header Host len 64
errorloc303 504 /live.html
errorloc303 503 /live.html
errorloc303 502 /live.html
maxconn 5000
*compression algo gzip*
*compression type text/html text/plain text/css text/javascript
application/javascript application/x-javascript*


Am I missing something or does HAproxy not support q-values?


Thanks!

Jay