Hi Aurelien, I really appreciate the response. This confirms, that "everything in runtime mode" uses the same thread pool as HTTP workers. This also says that the onlytime we can use "blocking IO" is in "initialization mode", ie, at HAproxy startup/reloads.
I shared the ACL example, but this is what I was really doing:- * Periodically, every 5 seconds, read the content of a file on disk via io.read() in a "lua task", save it in a global variable. * In a "lua service", return the value of global variable as body in response. Considering what we've already established in this email, it looks like HAProxy has no way to serve a "static file" which changes after HAproxy has been initialized. @Aurelien, please confirm! This seems like a big limitation, there has to be a workaround. Is it just acceptable in HAproxy world to "reload instead", every time a file is modified on disk? The only non-blocking functions available to use is via "socket class". Is it too crazy if we use "socket object" to do network IO to a process we write that acts as a proxy to do disk operations? Thanks, Abhijeet On Mon, Oct 17, 2022 at 1:13 AM Aurelien DARRAGON <[email protected]> wrote: > Hi, > > it feels like that shouldn't be true for "background tasks" as it is > mentioned that they run in separate threads > > That's not 100% true. Lua tasks do run concurrently with the rest of > HAProxy processing. > But they don't run in separate "threads": they run in separate haproxy > tasks. > What this mean is that lua task must follow haproxy task constraints: > > Within HAProxy, a task is implemented as a function that gets called from > the scheduler. > If the function needs to wait for some event/data to continue processing, > it must give the hand back and might be rescheduled later to continue > processing. > If the function fails to do so (blocks for too long), haproxy watchdog > will suspect that a thread is stuck, and will cause the whole process to > crash unconditionally. > That is because of HAProxy event-driven nature. > > From lua task: > > - core.* methods do respect this constraint > - io.* methods don't > > > This example > <https://github.com/zareenc/haproxy-lua-examples/blob/f5853013087642c0ed34ed47bea4c6efbf96dd29/lua_scripts/background_thread.lua> > was > linked on the arpalert.org website but it seems to use *"io.* methods in > runtime"*. > > To demonstrate this, I used the example you provided (lua task that uses > io.* methods), > and I modified it to make sure IO calls were blocking (ie: trying to read > from /dev/zero): > https://gist.github.com/Darlelet/5768628e40c0d960fce8c20649877e12 > > It does not take long for the watchdog to trigger and crash the process. > > Thus, if you were to use io.* from Lua task, you could be fine for some > time, but as long > as some io takes too long due to io.* blocking nature (and it will > happen), you > would cause the whole process to fail. > > I would highly recommend sticking to core.add_acl/core.del_acl for acl > management from lua task. > Maybe you could also leverage haproxy stats socket directly from the task > using core.tcp facility if > core.add_acl/core.del_acl is not what you're looking for. > > Regards, > Aurelien >
