Hi List,
Didn't have time yet to bisect when it went wrong. But attached testfile
produces the following output after 3 curl requests at different speeds,
this seems to trigger a problem as the hash of the downloaded content is
nolonger the same as it should be, (in my actual environment its a 2MB
javascript file that comes from a iis server behind haproxy.). Took
already a few hours more than desired to come up with a seemingly
reliable reproduction.
1.9-dev10 is the first one i put on my production environment as i think
release is imminent so it 'should' be pretty stable ;), (yes i know..i
shouldn't assume..) before it was using 1.8.14.. So was quick to revert
to that 1.8 again :).
Using these settings:
compression algo gzip
compression type text/html text/plain application/json
application/javascript
compression offload
When these compression settings are disabled, it completes successfully..
**** top 2.4 shell_cmd| exit 1
**** top 2.4 shell_cmd| fi
**** top 2.5 shell_out|File1 all OK
**** top 2.5 shell_out|File2 not OK 7798551c02a37ce89c77fc18fc415e5b
**** top 2.5 shell_out|File3 not OK 3146c4c9fce4da750558bfd9387ffc3b
**** top 2.5 shell_status = 0x0001
---- top 2.5 shell_exit not as expected: got 0x0001 wanted 0x0000
* top 2.5 RESETTING after ./PB-TEST/ulticompres/b00005.vtc
** h1 2.5 Reset and free h1 haproxy 51853
** h1 2.5 Wait
** h1 2.5 Stop HAproxy pid=51853
**** h1 2.5 STDOUT poll 0x11
**** h1 2.5 Kill(2)=0: No error: 0
** h1 2.6 WAIT4 pid=51853 status=0x0002 (user 0.253496 sys 0.000000)
* top 2.6 TEST ./PB-TEST/ulticompres/b00005.vtc FAILED
# top TEST ./PB-TEST/ulticompres/b00005.vtc FAILED (2.581) exit=2
haproxy -v
HA-Proxy version 1.9-dev10-3815b22 2018/12/11
Copyright 2000-2018 Willy Tarreau <[email protected]>
Can anyone confirm? Or perhaps even fix ;) Ill try and dig a little more
tomorrow evening :).
Thanks in advance,
PiBa-NL (Pieter)
local data = "abcdefghijklmnopqrstuvwxyz"
local responseblob = ""
math.randomseed(1)
for i = 1,10000 do
responseblob = responseblob .. "\r\n" .. i .. data:sub(1,
math.floor(math.random(4,26)))
end
http01applet = function(applet)
local response = responseblob
applet:set_status(200)
applet:add_header("Content-Type", "application/javascript")
applet:add_header("Content-Length", string.len(response)*10)
applet:start_response()
for i = 1,10 do
applet:send(response)
end
end
core.register_service("fileloader-http01", "http", http01applet)
# Checks that compression doesnt cause corruption..
varnishtest "Compression validation"
feature ignore_unknown_macro
haproxy h1 -conf {
global
# log stdout format short daemon
lua-load ${testdir}/b00005.lua
defaults
mode http
log global
option httplog
frontend main-https
bind "fd@${fe1}" ssl crt ${testdir}/common.pem
compression algo gzip
compression type text/html text/plain application/json
application/javascript
compression offload
use_backend TestBack if TRUE
backend TestBack
server LocalSrv ${h1_fe2_addr}:${h1_fe2_port}
listen fileloader
mode http
bind "fd@${fe2}"
http-request use-service lua.fileloader-http01
} -start
shell {
HOST=${h1_fe1_addr}
if [ "${h1_fe1_addr}" = "::1" ] ; then
HOST="\[::1\]"
fi
curl --compressed -k "https://$HOST:${h1_fe1_port}"; -o
${tmpdir}/outputfile1.bin
curl --compressed -k "https://$HOST:${h1_fe1_port}"; -o
${tmpdir}/outputfile3.bin --limit-rate 300K
curl --compressed -k "https://$HOST:${h1_fe1_port}"; -o
${tmpdir}/outputfile2.bin --limit-rate 500K
} -run
shell {
md5sum=$(md5 -q ${tmpdir}/outputfile1.bin)
if [ "$md5sum" = "f0d51d274ebc7696237efec272a38c41" ]
then
echo "File1 all OK"
else
echo "File1 not OK $md5sum "
testfailed=1
fi
md5sum=$(md5 -q ${tmpdir}/outputfile2.bin)
if [ "$md5sum" = "f0d51d274ebc7696237efec272a38c41" ]
then
echo "File2 all OK"
else
echo "File2 not OK $md5sum "
testfailed=1
fi
md5sum=$(md5 -q ${tmpdir}/outputfile3.bin)
if [ "$md5sum" = "f0d51d274ebc7696237efec272a38c41" ]
then
echo "File3 all OK"
else
echo "File3 not OK $md5sum "
testfailed=1
fi
if [ -n "$testfailed" ]; then
exit 1
fi
} -run
