Re: Lua testcase.. some 'random' data returned when loading a image.. 1.6dev2
On Fri, 19 Jun 2015 02:05:50 +0200 PiBa-NL piba.nl@gmail.com wrote: Hi guys, I'm sure i am abusing lua for completely wrong thing here. But i do not understand why the result isn't at least consistent.. Ive got a Pinguïns.jpg of 759kB (Default Windows 7 example image).. And have the configuration listed below. When requesting the image from a browser the top of the image looks normal, but further down it starts morphing, or becomes complete garbage.. Increasing bufsize to 30 makes the image show normal when reading and writing the whole image in 1 variable.. Though the buffer is still less than half of the image size.?. What im wondering though is how can it be that the results with smaller bufsizes vary..?? Is it overflowing some memory? With 1.6dev1 it would crash after a few requests, dev2 seems to have fixed that.. Though the random behavior is still strange.. I would expect every time the same image is send it to be cut short at buffsize, or perhaps just work if lua might use its own memory buffers not limited to haproxy's buffsize? Is it a bug? Or just a wrong lua script? Should files / sockets be closed by the script? I get an error if i do..(attempt to use a closed file) Hi, thank you for the repport. I'm currently trying to reproduce it, I'm not finished. I see two errors in your code: First, io.* are real blocking functions. HAProxy is really blocked waiting for the data access. It is a very bad to use it. However, how that you write for the file acces should work. Second, your variable f is not declared as local. This variable is keep between two requests. Maybe is the reason of your problem. To serve static file with Lua, the best practice is loading the full file during the initialisation and store it into a global variable. During the runtime, you will use only the variable and not the file. Thierry global #tune.bufsize 30 tune.lua.session-timeout 10 tune.lua.task-timeout 10 lua-load /var/etc/haproxy/hello_world.lua listen proxy bind :10001 tcp-request content lua hello_image function hello_image(txn) txn.res:send(HTTP/1.1 200 OK\n\n) f = io.open(/var/etc/haproxy/Penguins.jpg, rb) local block = 5000 while true do local bytes = f:read(block) if not bytes then break end txn.res:send(bytes) --core.msleep(25); ## changes behavior to be somewhat more succesfull.. end txn:close() --f:close() ## [ALERT] 168/232309 (74397) : Lua function 'hello_image': runtime error: /var/etc/haproxy/hello_world.lua:8: attempt to use a closed file. end root@OPNsense:/usr/ports/net/haproxy-devel # haproxy -vv HA-Proxy version 1.6-dev2-ad90f0d 2015/06/17 Copyright 2000-2015 Willy Tarreau wi...@haproxy.org Build options : TARGET = freebsd CPU = generic CC = cc CFLAGS = -O2 -pipe -fstack-protector -fno-strict-aliasing -DFREEBSD_PORTS OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_OPENSSL=1 USE_STATIC_PCRE=1 USE_PCRE_JIT=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200 Encrypted password support via crypt(3): yes Built with zlib version : 1.2.8 Compression algorithms supported : identity(identity), deflate(deflate), raw-deflate(deflate), gzip(gzip) Built with OpenSSL version : OpenSSL 1.0.2b 11 Jun 2015 Running on OpenSSL version : OpenSSL 1.0.2b 11 Jun 2015 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports prefer-server-ciphers : yes Built with PCRE version : 8.35 2014-04-04 PCRE library supports JIT : yes Built with Lua version : Lua 5.3.0 Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY Available polling systems : kqueue : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use kqueue. -- Thierry FOURNIER thierry.fourn...@arpalert.org
Re: Lua testcase.. some 'random' data returned when loading a image.. 1.6dev2
Ok changed the the lua implementation a little, still seeing 'different' images/colors appear when requesting the page repeatedly with a single browser though.. Assuming the 'penguinsimage' content doesnt change nor move in memory,, i suspect that somehow the txn.res:send is somehow subject to random buffer content overwriting.. function myinit(txn) local f = io.open(/var/etc/haproxy/Penguins.jpg, rb) penguinsimage = f:read(*all) f:close() end core.register_init(myinit) function hello_world(txn) txn.res:send(penguinsimage) txn:close() end Thierry FOURNIER schreef op 19-6-2015 om 14:22: On Fri, 19 Jun 2015 02:05:50 +0200 PiBa-NL piba.nl@gmail.com wrote: Hi guys, I'm sure i am abusing lua for completely wrong thing here. But i do not understand why the result isn't at least consistent.. Ive got a Pinguïns.jpg of 759kB (Default Windows 7 example image).. And have the configuration listed below. When requesting the image from a browser the top of the image looks normal, but further down it starts morphing, or becomes complete garbage.. Increasing bufsize to 30 makes the image show normal when reading and writing the whole image in 1 variable.. Though the buffer is still less than half of the image size.?. What im wondering though is how can it be that the results with smaller bufsizes vary..?? Is it overflowing some memory? With 1.6dev1 it would crash after a few requests, dev2 seems to have fixed that.. Though the random behavior is still strange.. I would expect every time the same image is send it to be cut short at buffsize, or perhaps just work if lua might use its own memory buffers not limited to haproxy's buffsize? Is it a bug? Or just a wrong lua script? Should files / sockets be closed by the script? I get an error if i do..(attempt to use a closed file) Hi, thank you for the repport. I'm currently trying to reproduce it, I'm not finished. I see two errors in your code: First, io.* are real blocking functions. HAProxy is really blocked waiting for the data access. It is a very bad to use it. However, how that you write for the file acces should work. Second, your variable f is not declared as local. This variable is keep between two requests. Maybe is the reason of your problem. To serve static file with Lua, the best practice is loading the full file during the initialisation and store it into a global variable. During the runtime, you will use only the variable and not the file. Thierry global #tune.bufsize 30 tune.lua.session-timeout 10 tune.lua.task-timeout 10 lua-load /var/etc/haproxy/hello_world.lua listen proxy bind :10001 tcp-request content lua hello_image function hello_image(txn) txn.res:send(HTTP/1.1 200 OK\n\n) f = io.open(/var/etc/haproxy/Penguins.jpg, rb) local block = 5000 while true do local bytes = f:read(block) if not bytes then break end txn.res:send(bytes) --core.msleep(25); ## changes behavior to be somewhat more succesfull.. end txn:close() --f:close() ## [ALERT] 168/232309 (74397) : Lua function 'hello_image': runtime error: /var/etc/haproxy/hello_world.lua:8: attempt to use a closed file. end root@OPNsense:/usr/ports/net/haproxy-devel # haproxy -vv HA-Proxy version 1.6-dev2-ad90f0d 2015/06/17 Copyright 2000-2015 Willy Tarreau wi...@haproxy.org Build options : TARGET = freebsd CPU = generic CC = cc CFLAGS = -O2 -pipe -fstack-protector -fno-strict-aliasing -DFREEBSD_PORTS OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_OPENSSL=1 USE_STATIC_PCRE=1 USE_PCRE_JIT=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200 Encrypted password support via crypt(3): yes Built with zlib version : 1.2.8 Compression algorithms supported : identity(identity), deflate(deflate), raw-deflate(deflate), gzip(gzip) Built with OpenSSL version : OpenSSL 1.0.2b 11 Jun 2015 Running on OpenSSL version : OpenSSL 1.0.2b 11 Jun 2015 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports prefer-server-ciphers : yes Built with PCRE version : 8.35 2014-04-04 PCRE library supports JIT : yes Built with Lua version : Lua 5.3.0 Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY Available polling systems : kqueue : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use kqueue.
Re: Lua testcase.. some 'random' data returned when loading a image.. 1.6dev2
On Fri, 19 Jun 2015 20:35:50 +0200 PiBa-NL piba.nl@gmail.com wrote: Ok changed the the lua implementation a little, still seeing 'different' images/colors appear when requesting the page repeatedly with a single browser though.. Assuming the 'penguinsimage' content doesnt change nor move in memory,, i suspect that somehow the txn.res:send is somehow subject to random buffer content overwriting.. function myinit(txn) local f = io.open(/var/etc/haproxy/Penguins.jpg, rb) penguinsimage = f:read(*all) f:close() end core.register_init(myinit) function hello_world(txn) txn.res:send(penguinsimage) txn:close() end I reproduce the problem. I see an object with missing data when I send more than one connections. I currently looking for a solution. Thierry Thierry FOURNIER schreef op 19-6-2015 om 14:22: On Fri, 19 Jun 2015 02:05:50 +0200 PiBa-NL piba.nl@gmail.com wrote: Hi guys, I'm sure i am abusing lua for completely wrong thing here. But i do not understand why the result isn't at least consistent.. Ive got a Pinguïns.jpg of 759kB (Default Windows 7 example image).. And have the configuration listed below. When requesting the image from a browser the top of the image looks normal, but further down it starts morphing, or becomes complete garbage.. Increasing bufsize to 30 makes the image show normal when reading and writing the whole image in 1 variable.. Though the buffer is still less than half of the image size.?. What im wondering though is how can it be that the results with smaller bufsizes vary..?? Is it overflowing some memory? With 1.6dev1 it would crash after a few requests, dev2 seems to have fixed that.. Though the random behavior is still strange.. I would expect every time the same image is send it to be cut short at buffsize, or perhaps just work if lua might use its own memory buffers not limited to haproxy's buffsize? Is it a bug? Or just a wrong lua script? Should files / sockets be closed by the script? I get an error if i do..(attempt to use a closed file) Hi, thank you for the repport. I'm currently trying to reproduce it, I'm not finished. I see two errors in your code: First, io.* are real blocking functions. HAProxy is really blocked waiting for the data access. It is a very bad to use it. However, how that you write for the file acces should work. Second, your variable f is not declared as local. This variable is keep between two requests. Maybe is the reason of your problem. To serve static file with Lua, the best practice is loading the full file during the initialisation and store it into a global variable. During the runtime, you will use only the variable and not the file. Thierry global #tune.bufsize 30 tune.lua.session-timeout 10 tune.lua.task-timeout 10 lua-load /var/etc/haproxy/hello_world.lua listen proxy bind :10001 tcp-request content lua hello_image function hello_image(txn) txn.res:send(HTTP/1.1 200 OK\n\n) f = io.open(/var/etc/haproxy/Penguins.jpg, rb) local block = 5000 while true do local bytes = f:read(block) if not bytes then break end txn.res:send(bytes) --core.msleep(25); ## changes behavior to be somewhat more succesfull.. end txn:close() --f:close() ## [ALERT] 168/232309 (74397) : Lua function 'hello_image': runtime error: /var/etc/haproxy/hello_world.lua:8: attempt to use a closed file. end root@OPNsense:/usr/ports/net/haproxy-devel # haproxy -vv HA-Proxy version 1.6-dev2-ad90f0d 2015/06/17 Copyright 2000-2015 Willy Tarreau wi...@haproxy.org Build options : TARGET = freebsd CPU = generic CC = cc CFLAGS = -O2 -pipe -fstack-protector -fno-strict-aliasing -DFREEBSD_PORTS OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_OPENSSL=1 USE_STATIC_PCRE=1 USE_PCRE_JIT=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200 Encrypted password support via crypt(3): yes Built with zlib version : 1.2.8 Compression algorithms supported : identity(identity), deflate(deflate), raw-deflate(deflate), gzip(gzip) Built with OpenSSL version : OpenSSL 1.0.2b 11 Jun 2015 Running on OpenSSL version : OpenSSL 1.0.2b 11 Jun 2015 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports prefer-server-ciphers : yes Built with PCRE version : 8.35 2014-04-04 PCRE library supports JIT : yes Built with Lua version : Lua 5.3.0 Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY Available polling systems : kqueue : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result
Lua testcase.. some 'random' data returned when loading a image.. 1.6dev2
Hi guys, I'm sure i am abusing lua for completely wrong thing here. But i do not understand why the result isn't at least consistent.. Ive got a Pinguïns.jpg of 759kB (Default Windows 7 example image).. And have the configuration listed below. When requesting the image from a browser the top of the image looks normal, but further down it starts morphing, or becomes complete garbage.. Increasing bufsize to 30 makes the image show normal when reading and writing the whole image in 1 variable.. Though the buffer is still less than half of the image size.?. What im wondering though is how can it be that the results with smaller bufsizes vary..?? Is it overflowing some memory? With 1.6dev1 it would crash after a few requests, dev2 seems to have fixed that.. Though the random behavior is still strange.. I would expect every time the same image is send it to be cut short at buffsize, or perhaps just work if lua might use its own memory buffers not limited to haproxy's buffsize? Is it a bug? Or just a wrong lua script? Should files / sockets be closed by the script? I get an error if i do..(attempt to use a closed file) global #tune.bufsize 30 tune.lua.session-timeout 10 tune.lua.task-timeout 10 lua-load /var/etc/haproxy/hello_world.lua listen proxy bind :10001 tcp-request content lua hello_image function hello_image(txn) txn.res:send(HTTP/1.1 200 OK\n\n) f = io.open(/var/etc/haproxy/Penguins.jpg, rb) local block = 5000 while true do local bytes = f:read(block) if not bytes then break end txn.res:send(bytes) --core.msleep(25); ## changes behavior to be somewhat more succesfull.. end txn:close() --f:close() ## [ALERT] 168/232309 (74397) : Lua function 'hello_image': runtime error: /var/etc/haproxy/hello_world.lua:8: attempt to use a closed file. end root@OPNsense:/usr/ports/net/haproxy-devel # haproxy -vv HA-Proxy version 1.6-dev2-ad90f0d 2015/06/17 Copyright 2000-2015 Willy Tarreau wi...@haproxy.org Build options : TARGET = freebsd CPU = generic CC = cc CFLAGS = -O2 -pipe -fstack-protector -fno-strict-aliasing -DFREEBSD_PORTS OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_OPENSSL=1 USE_STATIC_PCRE=1 USE_PCRE_JIT=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200 Encrypted password support via crypt(3): yes Built with zlib version : 1.2.8 Compression algorithms supported : identity(identity), deflate(deflate), raw-deflate(deflate), gzip(gzip) Built with OpenSSL version : OpenSSL 1.0.2b 11 Jun 2015 Running on OpenSSL version : OpenSSL 1.0.2b 11 Jun 2015 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports prefer-server-ciphers : yes Built with PCRE version : 8.35 2014-04-04 PCRE library supports JIT : yes Built with Lua version : Lua 5.3.0 Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY Available polling systems : kqueue : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use kqueue.
