On Tue, 1 Dec 2015 18:58:13 +0100 joris dedieu <[email protected]> wrote:
> 2015-12-01 10:57 GMT+01:00 Thierry FOURNIER <[email protected]>: > > On Mon, 30 Nov 2015 18:03:00 +0100 > > joris dedieu <[email protected]> wrote: > > > >> Thanks Thierry, for your answers. > >> > >> > >> 2015-11-30 16:53 GMT+01:00 Thierry FOURNIER > >> <[email protected]>: > >> > On Mon, 30 Nov 2015 08:37:00 +0100 > >> > joris dedieu <[email protected]> wrote: > >> > > >> >> Hi all, > >> >> > >> >> I started to drive into haproxy's lua interface. I produced a few code > >> >> that allows dnsbl lookup and it seems to work. > >> >> > >> >> First I have a C wrapper against the libc resolver.. > >> >> > >> >> #include <netdb.h> > >> >> #include <sys/types.h> > >> >> #include <sys/socket.h> > >> >> #include <netinet/in.h> > >> >> #include <arpa/inet.h> > >> >> > >> >> #include <lua53/lua.h> > >> >> #include <lua53/lauxlib.h> > >> >> > >> >> static int gethostbyname_wrapper(lua_State *L) > >> >> { > >> >> const char* query = luaL_checkstring(L, 1); > >> >> struct hostent *he; > >> >> if ((he = gethostbyname(query)) != NULL) { > >> >> const char *first_addr = > >> >> inet_ntoa(*(struct in_addr*)he->h_addr_list[0]); > >> >> lua_pushstring(L, first_addr); > >> >> return 1; > >> >> } > >> >> return 0; > >> >> } > >> >> > >> >> static const luaL_Reg sysdb_methods[] = { > >> >> {"gethostbyname", gethostbyname_wrapper}, > >> >> {NULL, NULL} > >> >> }; > >> >> > >> >> LUALIB_API int luaopen_sysdb(lua_State *L) { > >> >> luaL_newlib(L, sysdb_methods); > >> >> return 1; > >> >> } > >> >> > >> >> I have some doubts on the asyncness of libc operations but in other > >> >> side I don't want to reinvent the wheel. Should I prefer a resolver > >> >> implementation that uses lua socket ? As far as I tested libc seems to > >> >> do the job. > >> > > >> > Hello, > >> > > >> > I confirm your doubts: gethostbyname is synchronous and it is a > >> > blocking call. If your hostname resolution is in the /etc/hosts file, > >> > it blocks while reading file. It it is from DNS server, it blocks > >> > waiting for the server response (or worse: wainting for the timeout). > >> > > >> > So, this code seems to run, but your HAProxy will be not efficient > >> > because the entire haproxy process will be blocked during each > >> > resolution. For example: if your DNS fails after 3s timeout, during 3s, > >> > HAProxy doesn't process any data. > >> > > >> > Otherwise, your code is the good way to perform fast Lua/C libraries. > >> > > >> > There are no way to simulate blocking access out of the HAProxy core, > >> > all the functions writed for Lua must be non block. > >> > >> Ok, I will check for a non blocking solution (maybe lua socket + pack > >> / unpack in C) . > > > > > > This is an potential solution because the Lua Sockets (implemented in > > HAProxy) are non blocking. You cannot exploit the Lua socket from the C > > part, because the non blocking system can't throught 2 level of Lua/C. > > > > Maybe you can use the DNS protocol with and TCP/53 DNS server. With > > this solution, the DNS serveur already exists, and the TCP socket are > > already non block compliant. > > I started to port lua-resty-dns (pure lua). TCP is not a problem as > it's known by all major resolvers and as far as I know require by the > standard. Great, iy you will share the code, it become a great and famous DNS module for haproxy ;) For information, we try to find an existing service (like reddit) for collecting HAProxy useful Lua script and sharing ti between users. Actually, I store my own scripts on my personnal web page "http://ww.arpalert.org/haproxy-scripts.html";. > > Note that HAProxy supports now the native DNS, maybe in the version 1.7 > > the DNS will be repported in le Lua core. > > It would be great. IMHO unless your are too hurry, DNS is a nice way > to distribute informations across a network. I already see tow usages > : dnsbl and googlebot^w crawler validation. But there should be more > usage. I agree, an other example is to use the TXT field for storing user defined data. Thierry > Joris > > > > > >> > > >> > > >> >> Then the lua code > >> >> > >> >> local sysdb = require("sysdb") > >> >> > >> >> core.register_fetches("rbl", function(txn, rbl, ip) > >> >> if (not ip) then > >> >> ip = txn.sf:src() > >> >> end > >> >> if (not rbl) then > >> >> rbl = "zen.spamhaus.org" > >> >> end > >> >> local query = rbl > >> >> for x in string.gmatch(ip, "[^%.]+") do > >> >> query = x .. '.' .. query > >> >> end > >> >> if(sysdb.gethostbyname(query)) then > >> >> return 1 > >> >> else > >> >> return 0 > >> >> end > >> >> end) > >> >> > >> >> I want to use a sticky table as a local cache so my second question : > >> >> is there a way to set a gpt0 value from lua ? > >> > > >> > > >> > You can use the samples fetches mapper and use the sc_set_gpt0. The > >> > syntax is like this: > >> > > >> > For the read access: > >> > > >> > txn.sf:sc_set_gpt0() > >> > txn.sc:table_gpc0() > >> > > >> > For the write access, I don't have direct solution. You must use an Lua > >> > sample fetch and the following configuration directive: > >> > > >> > http-request sc-set-gpt0 lua.my_sample_fetch > >> > >> Yes that's an option. > >> > >> > > >> > Maybe it will be a good idea to implement the stick table access in Lua. > >> > > >> > If you want a other maneer to store shared data inhaproxy, you can use > >> > maps. The maps are shared by all the HAProxy process including Lua with > >> > a special API (see Map class) > >> > >> I thought on Maps but I didn't find a write access in lua according > >> to http://www.arpalert.org/src/haproxy-lua-api/1.6/index.html#map-class > >> and some of my experiments > > > > > > You're absolutely right. I'm afraid that I forgot to implement the > > write acces. Maybe I had a good reason to not implement this, but I'm > > not remember. > > > > I will look for this. > > > > > > Thierry > > > > > >> Thanks > >> Joris > >> > > >> > Thierry
