Hi, I'm talking only about performance ways)
About socket. I use UDP for sending, there are no reasons for delays. However, my bad - I misunderstood some FDs in "lsof". It is not related to that UDP-sending, that is OK. About file system. I open file from disk for GeoIP, but finally it cached in memory. I don't think that there will be difference if I move it to ramdisk. Example for Redis (works same as for GeoIP): 1) luarocks install redis-lua 2) At the top in lua-file (or inside core.register_init()): redisClient = pcall(redis.connect,'unix:///run/redis/redis.sock') 3) For cfg just "lua-load" param should be enough. For each reload "lsof" would show one additional unix-socket on master and worker. As I understand: - The LUA initialization executes for master and creates FD. - During reload "re-executed" master-process keeps old FD and gets one more by new lua-init. - Workers inherit everything from master. > Hi, > First, using: > - fd for file system access > - fd for tcp/udp connections throught Lua Socket > are a bad ideas because these actions block HAProxy. While the > fd/socket is waiting for data, HAProxy does nothing, and it not process any > other connections. > With other words: If you tcp connection reach a timeout of 30s, > HAProxy will be blocked during 30s, and all the incomming or > established connections wille be block during 30s. > - For filesystem access, you can use a ramdisk FS (like /dev/shm > on most distro), or preload files. > - For tcp connections, you can user core.tcp() > - For udp connection, I don’t have workaround. > 1) > When you reload haproxy, the old processes are destroyed, and news > process are created, so the FD are closed with the process. If the > main process own FD, childrens inherit these socket, but, the main > process can’t open more sockets, so you can’t reach limits. It is strange. > Do you have simple conf+lua code which reproduce the problem ? > 2) > As I wrote, using socket with HAProxy + Lua is not a good idea. > I think that it is a bad idea to increase limits for preventing > something linked to usage of a dangerous method. > The mindset of HAProxy is performance and resources saving first ! > BR, > Thierry >> On 16 Jan 2019, at 08:59, Wert <accp...@gmail.com> wrote: >> >>> CC'ing Thierry: as this has come on this discourse, can we have your >>> opinion about the FD's in LUA and howto best handle ulimit? >> >> >>> Apologies for the duplicate mail. >> >> >>> Thanks, >>> Lukas >> >> 1. FD >> I don't know your architecture too much. From user-side I just see no >> reasons to keep FD that created in LUA. >> For cases when I make redis-connection, open GEO-DB file or some socket to >> send, there is no reason to keep such FDs for new instance. >> It might be option or default policy to completely stop transferring LUA-FDs >> to new master. >> >> If it is difficult, probably there could be some ways to make ability for >> checking and cleaning it manually with CLI. >> >> 2. Ulimit >> It is impossible to know - how many FDs would be used by LUA even after >> fixing infinite grow. >> I use "ulimit-n 10000000". Of cause it looks like dirty thing but I can't >> imagine the case when it can make real harm, while low limit can a lot. >> If there is some harm, at least you may adjust current auto-calculated limit >> with "+100","*2" or some similar modifier and it will cover many real cases. >> >> --- >> Wert >>
