Hi Team,
We are having one issue with routing traffic through HAProxy.
Have configured IPv6 for our Google load balancer backed by HAProxy. The
architecture looks like below:
GLB --> L1 HAProxy --> Kong Gateway --> L2 HAProxy --> App node
L1 HAProxy has the logic for compression.
# zlib Compression
compression algo gzip
compression type text/html text/tab-separated-values text/plain
text/xml text/csv text/css text/javascript application/xml
application/x-javascript application/javascript
But while evaluating IPv6, we are seeing one peculiar error. For IPV6
request for a css file, we see that HAProxy injects a "Content-Encoding:
gzip" Header, but does not really compress the file thus corrupting the
response.
We tried doing a tcpdump on L1 HAProxy and Kong to check.
Besides the `x-forwarded-for` header, HAProxy sends the same request to
Kong for both IPv4 and IPv6.
Also Kong responds with the same header values in response to HAProxy for
both IPv4 and IPv6.
Content length also is the same in Kong's response to HAProxy for both IPv4
and IPv6. So Kong is not doing any compression.
HAProxy version is : HAProxy version 2.6.4-2a2078c 2022/08/22
Attaching the tcpdump details for HAProxy and Kong for both IPv4 and IPv6.
Have masked a few values for compliance.
P.S: We are on GCP and the IPv6 setup looks like this :
https://cloud.google.com/load-balancing/docs/ipv6#ipv6_termination_and_proxy
Any directions on this would be really appreciated.
Thanks!
Regards,
Narendra Patel
!.0.1..#PROXY TCP4 106.194.224.166 130.211.46.71 5358 443
GET /css/font-awesome/css/font-awesome.min.css HTTP/1.1
Host: dsautomationqa.qa-test.com
User-Agent: curl/7.84.0
Accept: */*
Accept-Encoding: gzip
05:35:43.662182 IP 10.29.144.30.80 > 35.191.3.208.56684: Flags [.], seq 1:4225, ack 208, win 229, options [nop,nop,TS val 825101877 ecr 553922719], length 4224: HTTP: HTTP/1.1 200
E....]@.@.|.
...#....P.lM..y.........p.....
1..5!.0.HTTP/1.1 200
Content-Type: text/css;charset=UTF-8
test-pod: 3000
x-test-update: faf454c28f889eace83180d6e0c08ec742113b9d
test-node: test
x-robots-tag: none
accept-ranges: bytes
etag: W/"26711-1666321007000"
last-modified: Fri, 21 Oct 2022 02:56:47 GMT
date: Fri, 21 Oct 2022 05:35:42 GMT
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
strict-transport-security: max-age=31536000; includeSubDomains
content-security-policy-report-only: default-src 'self' 'unsafe-eval'; script-src 'report-sample' 'self' 'unsafe-eval' 'unsafe-inline' https://content.p-a.test.com/ https://data.p-a.test.com/ https://ssl.google-analytics.com/ga.js https://www.gstatic.com/ https://www.recaptcha.net; style-src 'report-sample' 'self' 'unsafe-inline' blob: https://content.p-a.test.com; object-src 'none'; base-uri 'self'; connect-src 'self' https://api-js.mixpanel.com; font-src 'self' https://fonts.gstatic.com data: ; frame-src 'self'; img-src 'self' https://content.p-a.test.com https://data.p-a.test.com https://test-www-static.test.com https://ssl.google-analytics.com data: ; manifest-src 'self'; media-src 'self'; report-uri https://csp-reporter.qa-test.com/ ;
x-test-request-id: 6AC2E0A6:14EE_82D32E47:01BB_--_410D|qa-gcp-l1webuitest01+http_l1_webui
cache-control: public,max-age=2592000
x-test-override-cache: true
content-encoding: gzip
transfer-encoding: chunked
vary: Accept-Encoding
.|.a1+..PROXY TCP6 2401:4900:56f0:a175:9541:3ef8:2fae:2c62 2600:1901:0:174e:: 41730 443
GET /css/font-awesome/css/font-awesome.min.css HTTP/1.1
Host: dsautomationqa.qa-test.com
User-Agent: curl/7.84.0
Accept: */*
Accept-Encoding: gzip
05:33:28.168939 IP 10.29.144.30.80 > 35.191.14.7.62628: Flags [.], seq 1:7041, ack 238, win 229, options [nop,nop,TS val 824966384 ecr 394037345], length 7040: HTTP: HTTP/1.1 200
E.....@.@.=.
...#....P..8..[.,Y............
1+...|.aHTTP/1.1 200
Content-Type: text/css;charset=UTF-8
test-pod: 3000
x-test-update: faf454c28f889eace83180d6e0c08ec742113b9d
test-node: test
x-robots-tag: none
accept-ranges: bytes
etag: W/"26711-1666321007000"
last-modified: Fri, 21 Oct 2022 02:56:47 GMT
date: Fri, 21 Oct 2022 05:33:27 GMT
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
strict-transport-security: max-age=31536000; includeSubDomains
content-security-policy-report-only: default-src 'self' 'unsafe-eval'; script-src 'report-sample' 'self' 'unsafe-eval' 'unsafe-inline' https://content.p-a.test.com/ https://data.p-a.test.com/ https://ssl.google-analytics.com/ga.js https://www.gstatic.com/ https://www.recaptcha.net; style-src 'report-sample' 'self' 'unsafe-inline' blob: https://content.p-a.test.com; object-src 'none'; base-uri 'self'; connect-src 'self' https://api-js.mixpanel.com; font-src 'self' https://fonts.gstatic.com data: ; frame-src 'self'; img-src 'self' https://content.p-a.test.com https://data.p-a.test.com https://test-www-static.test.com https://ssl.google-analytics.com data: ; manifest-src 'self'; media-src 'self'; report-uri https://csp-reporter.qa-test.com/ ;
x-test-request-id: 2401490056F0A17595413EF82FAE2C62:A302_260019010000174E0000000000000000:01BB_--_3E4F|qa-gcp-l1webuitest01+http_l1_webui
cache-control: public,max-age=2592000
x-test-override-cache: true
content-encoding: gzip
transfer-encoding: chunked
(...(...GET /css/font-awesome/css/font-awesome.min.css HTTP/1.1
host: dsautomationqa.qa-test.com
user-agent: curl/7.84.0
accept: */*
accept-encoding: gzip
x-test-original-host: dsautomationqa.qa-test.com
x-test-original-proto: https
x-original-user-agent: curl/7.84.0
x-forwarded-proto: https
x-test-proxy: http://haproxy:15000
x-test-request-id: C08CDDCE:9D36_82D32E47:01BB_--_129C4
x-test-subdomain: dsautomationqa
x-test-kong-upstream: http://test:7180
x-test-kong-apply-rules: true
x-forwarded-for: 192.140.221.206
connection: close
10:56:23.719933 IP 127.0.0.1.8000 > 127.0.0.1.59930: Flags [.], ack 611, win 351, options [nop,nop,TS val 671541935 ecr 671541935], length 0
E..4.[@[email protected][email protected]( .<N.p..._.(.....
(...(...
10:56:23.727669 IP 127.0.0.1.8000 > 127.0.0.1.59930: Flags [.], seq 1:21889, ack 611, win 351, options [nop,nop,TS val 671541943 ecr 671541935], length 21888
E.U..\@.@[email protected]( .<N.p..._S......
(...(...HTTP/1.1 200
Content-Type: text/css;charset=UTF-8
Content-Length: 26711
Connection: close
test-pod: 3000
x-test-update: 4f6c350f74ef64364dddd02beccbdca06a1aa1fa
test-node: test
x-robots-tag: none
x-test-via: test
accept-ranges: bytes
etag: W/"26711-1666174383000"
last-modified: Wed, 19 Oct 2022 10:13:03 GMT
date: Wed, 19 Oct 2022 10:56:23 GMT
X-Kong-Upstream-Latency: 5
X-Kong-Proxy-Latency: 3
Via: kong/2.5.0
(..D(..DGET /css/font-awesome/css/font-awesome.min.css HTTP/1.1
host: dsautomationqa.qa-test.com
user-agent: curl/7.84.0
accept: */*
accept-encoding: gzip
x-test-original-host: dsautomationqa.qa-test.com
x-test-original-proto: https
x-original-user-agent: curl/7.84.0
x-forwarded-proto: https
x-test-proxy: http://haproxy:15000
x-test-request-id: 24050201003D407C040623C9EDFC687B:B55C_260019010000174E0000000000000000:01BB_--_13920
x-test-subdomain: dsautomationqa
x-test-kong-upstream: http://test:7180
x-test-kong-apply-rules: true
x-forwarded-for: 2405:201:3d:407c:406:23c9:edfc:687b
connection: close
11:09:03.164638 IP 127.0.0.1.8000 > 127.0.0.1.43314: Flags [.], ack 679, win 352, options [nop,nop,TS val 672301380 ecr 672301380], length 0
E..4..@.@.'[email protected]....{.5....`.(.....
(..D(..D
11:09:03.182299 IP 127.0.0.1.8000 > 127.0.0.1.43314: Flags [.], seq 1:21889, ack 679, win 352, options [nop,nop,TS val 672301398 ecr 672301380], length 21888
E.U...@[email protected][email protected]....{.5....`S......
(..V(..DHTTP/1.1 200
Content-Type: text/css;charset=UTF-8
Content-Length: 26711
Connection: close
test-pod: 3000
x-test-update: 4f6c350f74ef64364dddd02beccbdca06a1aa1fa
test-node: test
x-robots-tag: none
x-test-via: test
accept-ranges: bytes
etag: W/"26711-1666174383000"
last-modified: Wed, 19 Oct 2022 10:13:03 GMT
date: Wed, 19 Oct 2022 11:09:02 GMT
X-Kong-Upstream-Latency: 5
X-Kong-Proxy-Latency: 13
Via: kong/2.5.0
