Re: BUG: Lua tasks can't use client sockets after bf89ff3d
On 11/30/18 2:42 AM, PiBa-NL wrote: Hi Frederic, Adis, Hi Pieter, Op 29-11-2018 om 14:53 schreef Frederic Lecaille: Hi Adis, On 11/29/18 10:03 AM, Adis Nezirovic wrote: On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote: OK thanks, I'll take a look at it once I've flushed my pending stuff on H2+HTX :-( Great, I had my morning coffee and visited my optometrist, so here is a fixed test script (correctly setting Host header). P.S. Lua usually suffers trying to do things in tasks, I don't think this is the first time something gets broken. Can we make reg test with Lua script (maybe strip out LuaSocket requirement)? Yes. There already exist LUA reg tests in reg-tests/lua directory. Fred. Indeed some LUA tests already exists, but it didn't check to use a socket from a task. Attached a new test which does, and does indeed fail on versions since the mentioned commit. Great job! Should i make a patch out of it for inclusion in git? Or can you guys do that once the fix is also ready.? i think it was the preferred to get bugfix+regtest 'linked' then ? We do not take care anymore of linking a VTC file with a bugfix. I will soon modify the documentation when we will switch to a new standalone version of varnishtest program (named vtest, see https://github.com/vtest/VTest). Regards. Fred
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
Hi Pieter, On Fri, Nov 30, 2018 at 02:42:26AM +0100, PiBa-NL wrote: > Attached a new test which does, and does indeed fail on versions since the > mentioned commit. Thanks! > Should i make a patch out of it for inclusion in git? Or can you guys do > that once the fix is also ready.? i think it was the preferred to get > bugfix+regtest 'linked' then ? I prefer to have the tests ASAP, as they are helpful to test bugs. In my opinion, tests should not be limited to known bugs, since the only purpose they provide in this case is to help with backports of fixes. So please send a patch for inclusion, I'll take it. Thanks! Willy
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
Hi Frederic, Adis,
Op 29-11-2018 om 14:53 schreef Frederic Lecaille:
Hi Adis,
On 11/29/18 10:03 AM, Adis Nezirovic wrote:
On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote:
OK thanks, I'll take a look at it once I've flushed my pending stuff on
H2+HTX :-(
Great, I had my morning coffee and visited my optometrist, so here is
a fixed test script (correctly setting Host header).
P.S.
Lua usually suffers trying to do things in tasks, I don't think this is
the first time something gets broken. Can we make reg test with Lua
script (maybe strip out LuaSocket requirement)?
Yes. There already exist LUA reg tests in reg-tests/lua directory.
Fred.
Indeed some LUA tests already exists, but it didn't check to use a
socket from a task.
Attached a new test which does, and does indeed fail on versions since
the mentioned commit.
Should i make a patch out of it for inclusion in git? Or can you guys do
that once the fix is also ready.? i think it was the preferred to get
bugfix+regtest 'linked' then ?
Regards,
PiBa-NL (Pieter)
varnishtest "Lua: txn:get_priv() scope"
feature ignore_unknown_macro
#REQUIRE_OPTIONS=LUA
#REQUIRE_VERSION=1.6
server s1 {
rxreq
txresp -bodylen 20
} -start
haproxy h1 -conf {
global
lua-load ${testdir}/b4.lua
frontend fe1
mode http
bind "fd@${fe1}"
default_backend b1
backend b1
mode http
http-request use-service lua.fakeserv
} -start
client c0 -connect ${h1_fe1_sock} {
txreq -url "/" -hdr "vtcport: ${s1_port}"
rxresp
expect resp.status == 200
} -run
server s1 -wait
local vtc_port = 0
core.register_service("fakeserv", "http", function(applet)
vtc_port = applet.headers["vtcport"][0]
core.Info("APPLET START")
local response = "OK"
applet:add_header("Server", "haproxy/webstats")
applet:add_header("Content-Length", string.len(response))
applet:add_header("Content-Type", "text/html")
applet:start_response()
applet:send(response)
core.Info("APPLET DONE")
end)
local function cron()
-- wait for until the correct port is set through the c0 request..
while vtc_port == 0 do
core.msleep(1)
end
core.Debug('CRON port:' .. vtc_port)
local socket = core.tcp()
local success = socket:connect("127.0.0.1", vtc_port)
core.Info("SOCKET MADE ".. (success or "??"))
if success ~= 1 then
core.Info("CONNECT SOCKET FAILED?")
return
end
local request = "GET / HTTP/1.1\r\n\r\n"
core.Info("SENDING REQUEST")
socket:send(request)
local result = ""
repeat
core.Info("4")
local d = socket:receive("*a")
if d ~= nil then
result = result .. d
end
until d == nil or d == 0
core.Info("Received: "..result)
end
core.register_task(cron)
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
Hi Adis, On 11/29/18 10:03 AM, Adis Nezirovic wrote: On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote: OK thanks, I'll take a look at it once I've flushed my pending stuff on H2+HTX :-( Great, I had my morning coffee and visited my optometrist, so here is a fixed test script (correctly setting Host header). P.S. Lua usually suffers trying to do things in tasks, I don't think this is the first time something gets broken. Can we make reg test with Lua script (maybe strip out LuaSocket requirement)? Yes. There already exist LUA reg tests in reg-tests/lua directory. Fred.
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote:
> OK thanks, I'll take a look at it once I've flushed my pending stuff on
> H2+HTX :-(
Great, I had my morning coffee and visited my optometrist, so here is
a fixed test script (correctly setting Host header).
P.S.
Lua usually suffers trying to do things in tasks, I don't think this is
the first time something gets broken. Can we make reg test with Lua
script (maybe strip out LuaSocket requirement)?
Best regards,
--
Adis Nezirovic
Software Engineer
HAProxy Technologies - Powering your uptime!
375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US
+1 (844) 222-4340 | https://www.haproxy.com
local http = require 'socket.http'
local ltn12 = require 'ltn12'
local function main(applet)
end
local function cron()
local headers = {
host = 'www.haproxy.org'
}
local content = {}
while true do
core.Debug('CRON')
local b, c, h = http.request {
method='GET',
headers=headers,
url = 'http://51.15.8.218',
sink = ltn12.sink.table(content),
redirect = false,
create = core.tcp
}
core.Debug(tostring(c))
core.msleep(1000)
end
end
core.register_service("cron", "http", main)
core.register_task(cron)
global
log /dev/log local0 debug
nbproc 1
daemon
lua-load cron.lua
defaults
log global
mode http
option httplog
timeout connect 5s
timeout client 10s
timeout server 10s
frontend cron
bind 127.0.0.1:9000
http-request use-service lua.cron
Re: BUG: Lua tasks can't use client sockets after bf89ff3d
Hi Adis, On Wed, Nov 28, 2018 at 05:35:28PM +0100, Adis Nezirovic wrote: > Hey guys, > > After commit bf89ff3db8be1a8f87de305c144467bbc2503036 > "MEDIUM: stream-int: make stream_int_update() aware of the lower layers" > I'm not able to use client sockets from tasks created with Lua (see > attached configuration and Lua script) Ah crap :-( I tried to be careful, but was worried at the same time, seeing that Lua directly manipulates the stream interface's update functions. But I don't know how to test for this, so I only relied on the reg tests at the moment. > You will need LuaSocket library to run the test code, but I also get the > same issue without it (using raw core.tcp() and custom http library). > > Before the commit cron works fine, connection and http layer (returns > 404 since http.socket is a little bit stupid, but ignore that). > > After the commit, http.socket reports 'Can't connect' (and 'connection > closed' afterwards). Debug log shows significant time spent in > 'stream_int_chk_rcv_applet()' OK thanks, I'll take a look at it once I've flushed my pending stuff on H2+HTX :-( Cheers, Willy
BUG: Lua tasks can't use client sockets after bf89ff3d
Hey guys,
After commit bf89ff3db8be1a8f87de305c144467bbc2503036
"MEDIUM: stream-int: make stream_int_update() aware of the lower layers"
I'm not able to use client sockets from tasks created with Lua (see
attached configuration and Lua script)
You will need LuaSocket library to run the test code, but I also get the
same issue without it (using raw core.tcp() and custom http library).
Before the commit cron works fine, connection and http layer (returns
404 since http.socket is a little bit stupid, but ignore that).
After the commit, http.socket reports 'Can't connect' (and 'connection
closed' afterwards). Debug log shows significant time spent in
'stream_int_chk_rcv_applet()'
Best regards,
--
Adis Nezirovic
Software Engineer
HAProxy Technologies - Powering your uptime!
375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US
+1 (844) 222-4340 | https://www.haproxy.com
global
log /dev/log local0 debug
nbproc 1
daemon
lua-load cron.lua
defaults
log global
mode http
option httplog
timeout connect 5s
timeout client 10s
timeout server 10s
frontend cron
bind 127.0.0.1:9000
http-request use-service lua.cron
local http = require 'socket.http'
local ltn12 = require 'ltn12'
local function main(applet)
end
local function cron()
local headers = {
host = 'www.haproxy.org'
}
local content = {}
while true do
core.Debug('CRON')
local b, c, h = http.request {
method='GET',
headers=h,
url = 'http://51.15.8.218',
sink = ltn12.sink.table(content),
redirect = false,
create = core.tcp
}
core.Debug(tostring(c)) -- http.request doesn't use Host header
-- so we expect 404 response here
core.msleep(1000)
end
end
core.register_service("cron", "http", main)
core.register_task(cron)
