Re: [BUG] core.msleep yields forever
thank you for the tests. Thierry On Sat, 20 Feb 2016 17:33:14 + Robert Samuel Newsonwrote: > Hi, > > With only the new patch to hlua_applet_tcp_fct, I've altered my msleep call > to 200 and use apachebench to confirm close to 5 requests per second. > > Requests per second:4.88 [#/sec] (mean) > > So, new patch works for me. > > B. > > > > On 20 Feb 2016, at 17:10, Thierry FOURNIER wrote: > > > > Hi, > > > > Thank you Robert for the bug repport, I reproduced the bug perfectly. > > Thank you Cyril for the analysis. > > > > For information, the flag CTRLYIELD indicates that the yield is > > required by lua automatic interrupt system and not by some function. So > > the core must give back the hand to the haproxy core and resume > > immediately the Lua execution. > > > > This system allow the execution time control, and allow the haproxy > > core to accept connections or forward data. > > > > This explain why your patch doesn't run perfectly, it resume the Lua > > code, but immediately. > > > > I join anothe fix. Can you test it ? > > > > Thierry > > > > > > On Sat, 20 Feb 2016 15:54:49 + > > Robert Samuel Newson wrote: > > > >> with those changes, and altering my sleep to 1200; > >> > >> time curl localhost:6000 > >> hello > >> curl localhost:6000 0.01s user 0.00s system 0% cpu 1.215 total > >> > >> I'd say you're on to something :) > >> > >> > >>> On 19 Feb 2016, at 23:25, Cyril Bonté wrote: > >>> > >>> Hi Robert, > >>> > >>> I add Thierry to the discussion (see below for the details). > >>> > >>> Le 19/02/2016 20:15, Robert Samuel Newson a écrit : > Hi, > > I think I've found a bug in core.msleep (and core.sleep); > > foo.lua; > > core.register_service("foo", "http", function(applet) > core.msleep(1) > local body = "hello" > applet:set_status(200) > applet:add_header("Content-Length", string.len(body)) > applet:start_response() > applet:send(body) > end) > > haproxy.cfg > > global > lua-load foo.lua > > defaults > mode http > timeout client 15 > timeout server 360 > timeout connect 5000 > timeout queue 5000 > > listen l > bind 127.0.0.1:6000 > http-request use-service lua.foo > > -- > > steps to reproduce; > > curl 127.0.0.1:6000 > > this will not respond at all. > > If you comment out the core.msleep(1) line, you get the expected 200 > response. > > This seems to occurs wherever core.msleep is used but I've only > confirmed this behaviour in register_service and register_action > functions. > >>> > >>> I'm not expert in the lua code area, but after some tests, I wonder if > >>> the calls to hlua_yieldk() shoudln't provide the HLUA_CTRLYIELD flag in > >>> this functions : > >>> - hlua_sleep_yield > >>> - hlua_sleep > >>> - hlua_msleep > >>> > >>> Giving something like : > >>> diff --git a/src/hlua.c b/src/hlua.c > >>> index 5cf2320..022c107 100644 > >>> --- a/src/hlua.c > >>> +++ b/src/hlua.c > >>> @@ -4983,7 +4983,7 @@ __LJMP static int hlua_sleep_yield(lua_State *L, > >>> int status, lua_KContext ctx) > >>> { > >>> int wakeup_ms = lua_tointeger(L, -1); > >>> if (now_ms < wakeup_ms) > >>> - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); > >>> + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, > >>> HLUA_CTRLYIELD)); > >>> return 0; > >>> } > >>> > >>> @@ -4998,7 +4998,7 @@ __LJMP static int hlua_sleep(lua_State *L) > >>> wakeup_ms = tick_add(now_ms, delay); > >>> lua_pushinteger(L, wakeup_ms); > >>> > >>> - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); > >>> + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, > >>> HLUA_CTRLYIELD)); > >>> return 0; > >>> } > >>> > >>> @@ -5013,7 +5013,7 @@ __LJMP static int hlua_msleep(lua_State *L) > >>> wakeup_ms = tick_add(now_ms, delay); > >>> lua_pushinteger(L, wakeup_ms); > >>> > >>> - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); > >>> + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, > >>> HLUA_CTRLYIELD)); > >>> return 0; > >>> } > >>> > >>> Also, I'm not sure about the wake_time type in "struct lua" : shouldn't > >>> it be declared as an unsigned int ? > >>> Which then implies some type changes in other parts of the code, for > >>> example : > >>> - in hlua_sleep_yield() : unsigned int wakeup_ms > >>> - or in hlua_yieldk() : shouldn't "int timeout" be declared as an > >>> unsigned int also ? > >>> > >>> Sorry, I can't have a more longer look on this tonight. > >>> > >>> -- > >>> Cyril Bonté > >> > > > > > > -- > > > > <0001-BUG-MAJOR-lua-applets-can-t-sleep.patch> > --
Re: [BUG] core.msleep yields forever
Hi, With only the new patch to hlua_applet_tcp_fct, I've altered my msleep call to 200 and use apachebench to confirm close to 5 requests per second. Requests per second:4.88 [#/sec] (mean) So, new patch works for me. B. > On 20 Feb 2016, at 17:10, Thierry FOURNIERwrote: > > Hi, > > Thank you Robert for the bug repport, I reproduced the bug perfectly. > Thank you Cyril for the analysis. > > For information, the flag CTRLYIELD indicates that the yield is > required by lua automatic interrupt system and not by some function. So > the core must give back the hand to the haproxy core and resume > immediately the Lua execution. > > This system allow the execution time control, and allow the haproxy > core to accept connections or forward data. > > This explain why your patch doesn't run perfectly, it resume the Lua > code, but immediately. > > I join anothe fix. Can you test it ? > > Thierry > > > On Sat, 20 Feb 2016 15:54:49 + > Robert Samuel Newson wrote: > >> with those changes, and altering my sleep to 1200; >> >> time curl localhost:6000 >> hello >> curl localhost:6000 0.01s user 0.00s system 0% cpu 1.215 total >> >> I'd say you're on to something :) >> >> >>> On 19 Feb 2016, at 23:25, Cyril Bonté wrote: >>> >>> Hi Robert, >>> >>> I add Thierry to the discussion (see below for the details). >>> >>> Le 19/02/2016 20:15, Robert Samuel Newson a écrit : Hi, I think I've found a bug in core.msleep (and core.sleep); foo.lua; core.register_service("foo", "http", function(applet) core.msleep(1) local body = "hello" applet:set_status(200) applet:add_header("Content-Length", string.len(body)) applet:start_response() applet:send(body) end) haproxy.cfg global lua-load foo.lua defaults mode http timeout client 15 timeout server 360 timeout connect 5000 timeout queue 5000 listen l bind 127.0.0.1:6000 http-request use-service lua.foo -- steps to reproduce; curl 127.0.0.1:6000 this will not respond at all. If you comment out the core.msleep(1) line, you get the expected 200 response. This seems to occurs wherever core.msleep is used but I've only confirmed this behaviour in register_service and register_action functions. >>> >>> I'm not expert in the lua code area, but after some tests, I wonder if the >>> calls to hlua_yieldk() shoudln't provide the HLUA_CTRLYIELD flag in this >>> functions : >>> - hlua_sleep_yield >>> - hlua_sleep >>> - hlua_msleep >>> >>> Giving something like : >>> diff --git a/src/hlua.c b/src/hlua.c >>> index 5cf2320..022c107 100644 >>> --- a/src/hlua.c >>> +++ b/src/hlua.c >>> @@ -4983,7 +4983,7 @@ __LJMP static int hlua_sleep_yield(lua_State *L, int >>> status, lua_KContext ctx) >>> { >>> int wakeup_ms = lua_tointeger(L, -1); >>> if (now_ms < wakeup_ms) >>> - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); >>> + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, >>> HLUA_CTRLYIELD)); >>> return 0; >>> } >>> >>> @@ -4998,7 +4998,7 @@ __LJMP static int hlua_sleep(lua_State *L) >>> wakeup_ms = tick_add(now_ms, delay); >>> lua_pushinteger(L, wakeup_ms); >>> >>> - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); >>> + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, >>> HLUA_CTRLYIELD)); >>> return 0; >>> } >>> >>> @@ -5013,7 +5013,7 @@ __LJMP static int hlua_msleep(lua_State *L) >>> wakeup_ms = tick_add(now_ms, delay); >>> lua_pushinteger(L, wakeup_ms); >>> >>> - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); >>> + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, >>> HLUA_CTRLYIELD)); >>> return 0; >>> } >>> >>> Also, I'm not sure about the wake_time type in "struct lua" : shouldn't it >>> be declared as an unsigned int ? >>> Which then implies some type changes in other parts of the code, for >>> example : >>> - in hlua_sleep_yield() : unsigned int wakeup_ms >>> - or in hlua_yieldk() : shouldn't "int timeout" be declared as an unsigned >>> int also ? >>> >>> Sorry, I can't have a more longer look on this tonight. >>> >>> -- >>> Cyril Bonté >> > > > -- > > <0001-BUG-MAJOR-lua-applets-can-t-sleep.patch>
Re: [BUG] core.msleep yields forever
Hi, Thank you Robert for the bug repport, I reproduced the bug perfectly. Thank you Cyril for the analysis. For information, the flag CTRLYIELD indicates that the yield is required by lua automatic interrupt system and not by some function. So the core must give back the hand to the haproxy core and resume immediately the Lua execution. This system allow the execution time control, and allow the haproxy core to accept connections or forward data. This explain why your patch doesn't run perfectly, it resume the Lua code, but immediately. I join anothe fix. Can you test it ? Thierry On Sat, 20 Feb 2016 15:54:49 + Robert Samuel Newsonwrote: > with those changes, and altering my sleep to 1200; > > time curl localhost:6000 > hello > curl localhost:6000 0.01s user 0.00s system 0% cpu 1.215 total > > I'd say you're on to something :) > > > > On 19 Feb 2016, at 23:25, Cyril Bonté wrote: > > > > Hi Robert, > > > > I add Thierry to the discussion (see below for the details). > > > > Le 19/02/2016 20:15, Robert Samuel Newson a écrit : > >> Hi, > >> > >> I think I've found a bug in core.msleep (and core.sleep); > >> > >> foo.lua; > >> > >> core.register_service("foo", "http", function(applet) > >> core.msleep(1) > >> local body = "hello" > >> applet:set_status(200) > >> applet:add_header("Content-Length", string.len(body)) > >> applet:start_response() > >> applet:send(body) > >> end) > >> > >> haproxy.cfg > >> > >> global > >> lua-load foo.lua > >> > >> defaults > >> mode http > >> timeout client 15 > >> timeout server 360 > >> timeout connect 5000 > >> timeout queue 5000 > >> > >> listen l > >> bind 127.0.0.1:6000 > >> http-request use-service lua.foo > >> > >> -- > >> > >> steps to reproduce; > >> > >> curl 127.0.0.1:6000 > >> > >> this will not respond at all. > >> > >> If you comment out the core.msleep(1) line, you get the expected 200 > >> response. > >> > >> This seems to occurs wherever core.msleep is used but I've only confirmed > >> this behaviour in register_service and register_action functions. > > > > I'm not expert in the lua code area, but after some tests, I wonder if the > > calls to hlua_yieldk() shoudln't provide the HLUA_CTRLYIELD flag in this > > functions : > > - hlua_sleep_yield > > - hlua_sleep > > - hlua_msleep > > > > Giving something like : > > diff --git a/src/hlua.c b/src/hlua.c > > index 5cf2320..022c107 100644 > > --- a/src/hlua.c > > +++ b/src/hlua.c > > @@ -4983,7 +4983,7 @@ __LJMP static int hlua_sleep_yield(lua_State *L, int > > status, lua_KContext ctx) > > { > > int wakeup_ms = lua_tointeger(L, -1); > > if (now_ms < wakeup_ms) > > - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); > > + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, > > HLUA_CTRLYIELD)); > > return 0; > > } > > > > @@ -4998,7 +4998,7 @@ __LJMP static int hlua_sleep(lua_State *L) > > wakeup_ms = tick_add(now_ms, delay); > > lua_pushinteger(L, wakeup_ms); > > > > - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); > > + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, > > HLUA_CTRLYIELD)); > > return 0; > > } > > > > @@ -5013,7 +5013,7 @@ __LJMP static int hlua_msleep(lua_State *L) > > wakeup_ms = tick_add(now_ms, delay); > > lua_pushinteger(L, wakeup_ms); > > > > - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); > > + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, > > HLUA_CTRLYIELD)); > > return 0; > > } > > > > Also, I'm not sure about the wake_time type in "struct lua" : shouldn't it > > be declared as an unsigned int ? > > Which then implies some type changes in other parts of the code, for > > example : > > - in hlua_sleep_yield() : unsigned int wakeup_ms > > - or in hlua_yieldk() : shouldn't "int timeout" be declared as an unsigned > > int also ? > > > > Sorry, I can't have a more longer look on this tonight. > > > > -- > > Cyril Bonté > -- >From 3c10a4d8320731d1093cddc405043d335862d074 Mon Sep 17 00:00:00 2001 From: Thierry Fournier Date: Sat, 20 Feb 2016 17:47:43 +0100 Subject: [PATCH] BUG/MAJOR: lua: applets can't sleep. This patch must be backported in 1.6 hlua_yield() function returns the required sleep time. The Lua core must be resume the execution after the required time. The core dedicated to the http and tcp applet doesn't implement the wake up function. It is a miss. This patch fix this. --- src/hlua.c | 4 1 file changed, 4 insertions(+) diff --git a/src/hlua.c b/src/hlua.c index f1c937a..fe58b9d 100644 --- a/src/hlua.c +++ b/src/hlua.c @@ -5747,6 +5747,8 @@ static void hlua_applet_tcp_fct(struct appctx *ctx) /* yield. */ case HLUA_E_AGAIN: + if (hlua->wake_time != TICK_ETERNITY) + task_schedule(ctx->ctx.hlua_apptcp.task, hlua->wake_time); return; /* finished
Re: [BUG] core.msleep yields forever
with those changes, and altering my sleep to 1200; time curl localhost:6000 hello curl localhost:6000 0.01s user 0.00s system 0% cpu 1.215 total I'd say you're on to something :) > On 19 Feb 2016, at 23:25, Cyril Bontéwrote: > > Hi Robert, > > I add Thierry to the discussion (see below for the details). > > Le 19/02/2016 20:15, Robert Samuel Newson a écrit : >> Hi, >> >> I think I've found a bug in core.msleep (and core.sleep); >> >> foo.lua; >> >> core.register_service("foo", "http", function(applet) >> core.msleep(1) >> local body = "hello" >> applet:set_status(200) >> applet:add_header("Content-Length", string.len(body)) >> applet:start_response() >> applet:send(body) >> end) >> >> haproxy.cfg >> >> global >> lua-load foo.lua >> >> defaults >> mode http >> timeout client 15 >> timeout server 360 >> timeout connect 5000 >> timeout queue 5000 >> >> listen l >> bind 127.0.0.1:6000 >> http-request use-service lua.foo >> >> -- >> >> steps to reproduce; >> >> curl 127.0.0.1:6000 >> >> this will not respond at all. >> >> If you comment out the core.msleep(1) line, you get the expected 200 >> response. >> >> This seems to occurs wherever core.msleep is used but I've only confirmed >> this behaviour in register_service and register_action functions. > > I'm not expert in the lua code area, but after some tests, I wonder if the > calls to hlua_yieldk() shoudln't provide the HLUA_CTRLYIELD flag in this > functions : > - hlua_sleep_yield > - hlua_sleep > - hlua_msleep > > Giving something like : > diff --git a/src/hlua.c b/src/hlua.c > index 5cf2320..022c107 100644 > --- a/src/hlua.c > +++ b/src/hlua.c > @@ -4983,7 +4983,7 @@ __LJMP static int hlua_sleep_yield(lua_State *L, int > status, lua_KContext ctx) > { > int wakeup_ms = lua_tointeger(L, -1); > if (now_ms < wakeup_ms) > - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); > + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, > HLUA_CTRLYIELD)); > return 0; > } > > @@ -4998,7 +4998,7 @@ __LJMP static int hlua_sleep(lua_State *L) > wakeup_ms = tick_add(now_ms, delay); > lua_pushinteger(L, wakeup_ms); > > - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); > + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, > HLUA_CTRLYIELD)); > return 0; > } > > @@ -5013,7 +5013,7 @@ __LJMP static int hlua_msleep(lua_State *L) > wakeup_ms = tick_add(now_ms, delay); > lua_pushinteger(L, wakeup_ms); > > - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); > + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, > HLUA_CTRLYIELD)); > return 0; > } > > Also, I'm not sure about the wake_time type in "struct lua" : shouldn't it be > declared as an unsigned int ? > Which then implies some type changes in other parts of the code, for example : > - in hlua_sleep_yield() : unsigned int wakeup_ms > - or in hlua_yieldk() : shouldn't "int timeout" be declared as an unsigned > int also ? > > Sorry, I can't have a more longer look on this tonight. > > -- > Cyril Bonté
Re: [BUG] core.msleep yields forever
Hi Robert, I add Thierry to the discussion (see below for the details). Le 19/02/2016 20:15, Robert Samuel Newson a écrit : Hi, I think I've found a bug in core.msleep (and core.sleep); foo.lua; core.register_service("foo", "http", function(applet) core.msleep(1) local body = "hello" applet:set_status(200) applet:add_header("Content-Length", string.len(body)) applet:start_response() applet:send(body) end) haproxy.cfg global lua-load foo.lua defaults mode http timeout client 15 timeout server 360 timeout connect 5000 timeout queue 5000 listen l bind 127.0.0.1:6000 http-request use-service lua.foo -- steps to reproduce; curl 127.0.0.1:6000 this will not respond at all. If you comment out the core.msleep(1) line, you get the expected 200 response. This seems to occurs wherever core.msleep is used but I've only confirmed this behaviour in register_service and register_action functions. I'm not expert in the lua code area, but after some tests, I wonder if the calls to hlua_yieldk() shoudln't provide the HLUA_CTRLYIELD flag in this functions : - hlua_sleep_yield - hlua_sleep - hlua_msleep Giving something like : diff --git a/src/hlua.c b/src/hlua.c index 5cf2320..022c107 100644 --- a/src/hlua.c +++ b/src/hlua.c @@ -4983,7 +4983,7 @@ __LJMP static int hlua_sleep_yield(lua_State *L, int status, lua_KContext ctx) { int wakeup_ms = lua_tointeger(L, -1); if (now_ms < wakeup_ms) - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, HLUA_CTRLYIELD)); return 0; } @@ -4998,7 +4998,7 @@ __LJMP static int hlua_sleep(lua_State *L) wakeup_ms = tick_add(now_ms, delay); lua_pushinteger(L, wakeup_ms); - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, HLUA_CTRLYIELD)); return 0; } @@ -5013,7 +5013,7 @@ __LJMP static int hlua_msleep(lua_State *L) wakeup_ms = tick_add(now_ms, delay); lua_pushinteger(L, wakeup_ms); - WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, 0)); + WILL_LJMP(hlua_yieldk(L, 0, 0, hlua_sleep_yield, wakeup_ms, HLUA_CTRLYIELD)); return 0; } Also, I'm not sure about the wake_time type in "struct lua" : shouldn't it be declared as an unsigned int ? Which then implies some type changes in other parts of the code, for example : - in hlua_sleep_yield() : unsigned int wakeup_ms - or in hlua_yieldk() : shouldn't "int timeout" be declared as an unsigned int also ? Sorry, I can't have a more longer look on this tonight. -- Cyril Bonté