Re: BUG: Lua tasks can't use client sockets after bf89ff3d
On 11/30/18 2:42 AM, PiBa-NL wrote: Hi Frederic, Adis, Hi Pieter, Op 29-11-2018 om 14:53 schreef Frederic Lecaille: Hi Adis, On 11/29/18 10:03 AM, Adis Nezirovic wrote: On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote: OK thanks, I'll take a look at it once I've flushed my pending stuff on H2+HTX :-( Great, I had my morning coffee and visited my optometrist, so here is a fixed test script (correctly setting Host header). P.S. Lua usually suffers trying to do things in tasks, I don't think this is the first time something gets broken. Can we make reg test with Lua script (maybe strip out LuaSocket requirement)? Yes. There already exist LUA reg tests in reg-tests/lua directory. Fred. Indeed some LUA tests already exists, but it didn't check to use a socket from a task. Attached a new test which does, and does indeed fail on versions since the mentioned commit. Great job! Should i make a patch out of it for inclusion in git? Or can you guys do that once the fix is also ready.? i think it was the preferred to get bugfix+regtest 'linked' then ? We do not take care anymore of linking a VTC file with a bugfix. I will soon modify the documentation when we will switch to a new standalone version of varnishtest program (named vtest, see https://github.com/vtest/VTest). Regards. Fred
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
Hi Pieter, On Fri, Nov 30, 2018 at 02:42:26AM +0100, PiBa-NL wrote: > Attached a new test which does, and does indeed fail on versions since the > mentioned commit. Thanks! > Should i make a patch out of it for inclusion in git? Or can you guys do > that once the fix is also ready.? i think it was the preferred to get > bugfix+regtest 'linked' then ? I prefer to have the tests ASAP, as they are helpful to test bugs. In my opinion, tests should not be limited to known bugs, since the only purpose they provide in this case is to help with backports of fixes. So please send a patch for inclusion, I'll take it. Thanks! Willy
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
Hi Frederic, Adis, Op 29-11-2018 om 14:53 schreef Frederic Lecaille: Hi Adis, On 11/29/18 10:03 AM, Adis Nezirovic wrote: On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote: OK thanks, I'll take a look at it once I've flushed my pending stuff on H2+HTX :-( Great, I had my morning coffee and visited my optometrist, so here is a fixed test script (correctly setting Host header). P.S. Lua usually suffers trying to do things in tasks, I don't think this is the first time something gets broken. Can we make reg test with Lua script (maybe strip out LuaSocket requirement)? Yes. There already exist LUA reg tests in reg-tests/lua directory. Fred. Indeed some LUA tests already exists, but it didn't check to use a socket from a task. Attached a new test which does, and does indeed fail on versions since the mentioned commit. Should i make a patch out of it for inclusion in git? Or can you guys do that once the fix is also ready.? i think it was the preferred to get bugfix+regtest 'linked' then ? Regards, PiBa-NL (Pieter) varnishtest "Lua: txn:get_priv() scope" feature ignore_unknown_macro #REQUIRE_OPTIONS=LUA #REQUIRE_VERSION=1.6 server s1 { rxreq txresp -bodylen 20 } -start haproxy h1 -conf { global lua-load ${testdir}/b4.lua frontend fe1 mode http bind "fd@${fe1}" default_backend b1 backend b1 mode http http-request use-service lua.fakeserv } -start client c0 -connect ${h1_fe1_sock} { txreq -url "/" -hdr "vtcport: ${s1_port}" rxresp expect resp.status == 200 } -run server s1 -wait local vtc_port = 0 core.register_service("fakeserv", "http", function(applet) vtc_port = applet.headers["vtcport"][0] core.Info("APPLET START") local response = "OK" applet:add_header("Server", "haproxy/webstats") applet:add_header("Content-Length", string.len(response)) applet:add_header("Content-Type", "text/html") applet:start_response() applet:send(response) core.Info("APPLET DONE") end) local function cron() -- wait for until the correct port is set through the c0 request.. while vtc_port == 0 do core.msleep(1) end core.Debug('CRON port:' .. vtc_port) local socket = core.tcp() local success = socket:connect("127.0.0.1", vtc_port) core.Info("SOCKET MADE ".. (success or "??")) if success ~= 1 then core.Info("CONNECT SOCKET FAILED?") return end local request = "GET / HTTP/1.1\r\n\r\n" core.Info("SENDING REQUEST") socket:send(request) local result = "" repeat core.Info("4") local d = socket:receive("*a") if d ~= nil then result = result .. d end until d == nil or d == 0 core.Info("Received: "..result) end core.register_task(cron)
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
Hi Adis, On 11/29/18 10:03 AM, Adis Nezirovic wrote: On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote: OK thanks, I'll take a look at it once I've flushed my pending stuff on H2+HTX :-( Great, I had my morning coffee and visited my optometrist, so here is a fixed test script (correctly setting Host header). P.S. Lua usually suffers trying to do things in tasks, I don't think this is the first time something gets broken. Can we make reg test with Lua script (maybe strip out LuaSocket requirement)? Yes. There already exist LUA reg tests in reg-tests/lua directory. Fred.
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote: > OK thanks, I'll take a look at it once I've flushed my pending stuff on > H2+HTX :-( Great, I had my morning coffee and visited my optometrist, so here is a fixed test script (correctly setting Host header). P.S. Lua usually suffers trying to do things in tasks, I don't think this is the first time something gets broken. Can we make reg test with Lua script (maybe strip out LuaSocket requirement)? Best regards, -- Adis Nezirovic Software Engineer HAProxy Technologies - Powering your uptime! 375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US +1 (844) 222-4340 | https://www.haproxy.com local http = require 'socket.http' local ltn12 = require 'ltn12' local function main(applet) end local function cron() local headers = { host = 'www.haproxy.org' } local content = {} while true do core.Debug('CRON') local b, c, h = http.request { method='GET', headers=headers, url = 'http://51.15.8.218', sink = ltn12.sink.table(content), redirect = false, create = core.tcp } core.Debug(tostring(c)) core.msleep(1000) end end core.register_service("cron", "http", main) core.register_task(cron) global log /dev/log local0 debug nbproc 1 daemon lua-load cron.lua defaults log global mode http option httplog timeout connect 5s timeout client 10s timeout server 10s frontend cron bind 127.0.0.1:9000 http-request use-service lua.cron
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
Hi Adis, On Wed, Nov 28, 2018 at 05:35:28PM +0100, Adis Nezirovic wrote: > Hey guys, > > After commit bf89ff3db8be1a8f87de305c144467bbc2503036 > "MEDIUM: stream-int: make stream_int_update() aware of the lower layers" > I'm not able to use client sockets from tasks created with Lua (see > attached configuration and Lua script) Ah crap :-( I tried to be careful, but was worried at the same time, seeing that Lua directly manipulates the stream interface's update functions. But I don't know how to test for this, so I only relied on the reg tests at the moment. > You will need LuaSocket library to run the test code, but I also get the > same issue without it (using raw core.tcp() and custom http library). > > Before the commit cron works fine, connection and http layer (returns > 404 since http.socket is a little bit stupid, but ignore that). > > After the commit, http.socket reports 'Can't connect' (and 'connection > closed' afterwards). Debug log shows significant time spent in > 'stream_int_chk_rcv_applet()' OK thanks, I'll take a look at it once I've flushed my pending stuff on H2+HTX :-( Cheers, Willy