Re: Configure Haproxy to dynamically set backend server
Hello, Thanks for all your help. Any rough estimate on when the patch for doing DNS resolutions during runtime with asynchronous methods might be out? -Mrunmayi On Saturday, May 30, 2015 2:47 AM, Thierry FOURNIER tfourn...@haproxy.com wrote: On Sat, 30 May 2015 00:25:59 + (UTC) Mrunmayi Dhume mrunmayi.dh...@yahoo.com wrote: Hello Thierry, This seems to be what we are looking for, however it doesn't seem to work as expected. When we use your example as is, it seems to fail with a 500 error. When we switch to using a IP address, it works, so it seems like DNS resolution is a problem? Yes, haproxy can not execute DNS résolutions while is running. It does DNS resolution only during the configuration parsing. Its because the standard DNS resolution are a synchronous process, and the HAProxy architecture does not accept synchronous processes. Note that a patch is currently in development for doing DNS resolutions during the runtime with asynchronous methods. As per this doc - http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#4-option%20http_proxy http-proxy mode does not accept hostname, instead it only accepts IP. The below example works: core.register_fetches(choose_backend, function(txn) if txn.sf.req_fhdr(Host) == 'example.test.com' and txn.sf.req_path == '/test' then return 1.1.1.1 else if [...] end return default_backend end); In the haproxy configuration file, you must load the lua file, and usethe new declared fetch in your frontend: global [...] lua-load your-lua-file.lua [...] listen your_frt [...] option http_proxy http-request set-uri http://%[lua.choose_backend]%[url] We don’t want to have to specify the IP as that means that we will have to perform some sort of dns resolution in lua. Can you suggest an alternative? Actually, you can use maps to do this. A map is a file containing a name and his correspondance. You can put a hostname and the associated ip. This map can be updated throught the HAProxy's socket (without restarting HAProxy). This is not a real DNS Resolution, you must kown the full list of your domains. Thierry Thanks! -Mrunmayi On Wednesday, May 27, 2015 1:50 AM, Thierry FOURNIER tfourn...@haproxy.com wrote: On Tue, 26 May 2015 21:39:23 + (UTC) Mrunmayi Dhume mrunmayi.dh...@yahoo.com wrote: Thanks for your detailed reply Thierry. While this approach would solve the aspect of choosing the backend dynamically we still need to explicitly define each backend server separately in the haproxy config file. Our use-case involves having 100+ backends and we would prefer not to complicate the config file by defining each backend server as it is not easy to maintain. We prefer to set it based on incoming http request information like path or host header and keep haproxy config file simple with just listen and FE directives and of course single default backend server Hi, HAProxy does not permit to choose the destination IP and PORT. But, maybe I have an ugly solution to your problem. You can try to deal with the option option http_proxy. This option understand the base proxy requests. You create a listen section which use Lua for rewriting the path of the HTTP request like this: core.register_fetches(choose_backend, function(txn) if txn.sf.req_fhdr(Host) == 'example.test.com' and txn.sf.req_path == '/test' then return test1. example.com:8081 else if [...] end return default_backend end); In the haproxy configuration file, you must load the lua file, and use the new declared fetch inyour frontend: global [...] lua-load your-lua-file.lua [...] listen your_frt [...] option http_proxy http-request set-uri http://%[lua.choose_backend]%[url] [...] Thierry if (Host == aaa.example.com’ and path == ‘/test’) thenbackend server == test1. example.com:8081 (Origin port changed) elsif (Host == aaa.example.com’ and path == ‘/test2’) backend server == test2. example.com:8080 (Origin port changed) elsif (Host == aaa.example.com’ and path == ‘/test3’) backend server == xxx.example.com: ((Origin host and port changed) looking for something like the Traffic server lua api - https://docs.trafficserver.apache.org/en/latest/reference/plugins/ts_lua.en.html#ts-client-request-set-url-host Thanks, Mrunmayi On Tuesday, May 26, 2015 3:15 AM, Thierry FOURNIER tfourn...@haproxy.com wrote: On Fri, 22 May 2015 19:06:59 + (UTC) Mrunmayi Dhume mrunmayi.dh...@yahoo.com wrote: Hello, I am using haproxy-1.6 with Lua. I have a use-case where I want to set the destination (backend server) very dynamically, based on certain layer 7 information (I am trying to avoid updating haproxy configuration
Re: Configure Haproxy to dynamically set backend server
Hello Thierry, This seems to be what we are looking for, however it doesn't seem to work as expected. When we use your example as is, it seems to fail with a 500 error. When we switch to using a IP address, it works, so it seems like DNS resolution is a problem? As per this doc - http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#4-option%20http_proxy http-proxy mode does not accept hostname, instead it only accepts IP. The below example works: core.register_fetches(choose_backend, function(txn) if txn.sf.req_fhdr(Host) == 'example.test.com' and txn.sf.req_path == '/test' then return 1.1.1.1 else if [...] end return default_backend end); In the haproxy configuration file, you must load the lua file, and usethe new declared fetch in your frontend: global [...] lua-load your-lua-file.lua [...] listen your_frt [...] option http_proxy http-request set-uri http://%[lua.choose_backend]%[url] We don’t want to have to specify the IP as that means that we will have to perform some sort of dns resolution in lua. Can you suggest an alternative? Thanks! -Mrunmayi On Wednesday, May 27, 2015 1:50 AM, Thierry FOURNIER tfourn...@haproxy.com wrote: On Tue, 26 May 2015 21:39:23 + (UTC) Mrunmayi Dhume mrunmayi.dh...@yahoo.com wrote: Thanks for your detailed reply Thierry. While this approach would solve the aspect of choosing the backend dynamically we still need to explicitly define each backend server separately in the haproxy config file. Our use-case involves having 100+ backends and we would prefer not to complicate the config file by defining each backend server as it is not easy to maintain. We prefer to set it based on incoming http request information like path or host header and keep haproxy config file simple with just listen and FE directives and of course single default backend server Hi, HAProxy does not permit to choose the destination IP and PORT. But, maybe I have an ugly solution to your problem. You can try to deal with the option option http_proxy. This option understand the base proxy requests. You create a listen section which use Lua for rewriting the path of the HTTP request like this: core.register_fetches(choose_backend, function(txn) if txn.sf.req_fhdr(Host) == 'example.test.com' and txn.sf.req_path == '/test' then return test1. example.com:8081 else if [...] end return default_backend end); In the haproxy configuration file, you must load the lua file, and use the new declared fetch inyour frontend: global [...] lua-load your-lua-file.lua [...] listen your_frt [...] option http_proxy http-request set-uri http://%[lua.choose_backend]%[url] [...] Thierry if (Host == aaa.example.com’ and path == ‘/test’) thenbackend server == test1. example.com:8081 (Origin port changed) elsif (Host == aaa.example.com’ and path == ‘/test2’) backend server == test2. example.com:8080 (Origin port changed) elsif (Host == aaa.example.com’ and path == ‘/test3’) backend server == xxx.example.com: ((Origin host and port changed) looking for something like the Traffic server lua api - https://docs.trafficserver.apache.org/en/latest/reference/plugins/ts_lua.en.html#ts-client-request-set-url-host Thanks, Mrunmayi On Tuesday, May 26, 2015 3:15 AM, Thierry FOURNIER tfourn...@haproxy.com wrote: On Fri, 22 May 2015 19:06:59 + (UTC) Mrunmayi Dhume mrunmayi.dh...@yahoo.com wrote: Hello, I am using haproxy-1.6 with Lua. I have a use-case where I want to set the destination (backend server) very dynamically, based on certain layer 7 information (I am trying to avoid updating haproxy configuration and make it complicated with a ~100 domain names and their corresponding backend server name and do a map lookup, I want lua to do it ) eg: from lua, based on certain layer 7 information I want to change the backend server (set destination) it should connect to.. if Host is ‘example.test.com’; then backend is test1.server.com elsif Host is ‘example2.test.com; then backend is test2.server.com I want to keep the haproxy config simple and have only the Listen directive listed and take more control of the request flow from the LUA code. The ATS traffic server does provide such functionality through the LUA API hook to override backend server dynamically from Lua end if ts.client_request.header[Host] == 'example.test.com' then ts.client_request.set_url_host( test1.server.comts.client_request.set_url_port(80)elsif ts.client_request.header[Host] == 'example2.test.com' then ts.client_request.set_url_host( test2.server.com ) ts.client_request.set_url_port(80)end Is it possible to do the same using Haproxy and Lua? Hi, Yes, you can, but the way is a little bit
Re: Configure Haproxy to dynamically set backend server
Thanks for your detailed reply Thierry. While this approach would solve the aspect of choosing the backend dynamically we still need to explicitly define each backend server separately in the haproxy config file. Our use-case involves having 100+ backends and we would prefer not to complicate the config file by defining each backend server as it is not easy to maintain. We prefer to set it based on incoming http request information like path or host header and keep haproxy config file simple with just listen and FE directives and of course single default backend server if (Host == aaa.example.com’ and path == ‘/test’) thenbackend server == test1. example.com:8081 (Origin port changed) elsif (Host == aaa.example.com’ and path == ‘/test2’) backend server == test2. example.com:8080 (Origin port changed) elsif (Host == aaa.example.com’ and path == ‘/test3’) backend server == xxx.example.com: ((Origin host and port changed) looking for something like the Traffic server lua api - https://docs.trafficserver.apache.org/en/latest/reference/plugins/ts_lua.en.html#ts-client-request-set-url-host Thanks, Mrunmayi On Tuesday, May 26, 2015 3:15 AM, Thierry FOURNIER tfourn...@haproxy.com wrote: On Fri, 22 May 2015 19:06:59 + (UTC) Mrunmayi Dhume mrunmayi.dh...@yahoo.com wrote: Hello, I am using haproxy-1.6 with Lua. I have a use-case where I want to set the destination (backend server) very dynamically, based on certain layer 7 information (I am trying to avoid updating haproxy configuration and make it complicated with a ~100 domain names and their corresponding backend server name and do a map lookup, I want lua to do it ) eg: from lua, based on certain layer 7 information I want to change the backend server (set destination) it should connect to.. if Host is ‘example.test.com’; then backend is test1.server.com elsif Host is ‘example2.test.com; then backend is test2.server.com I want to keep the haproxy config simple and have only the Listen directive listed and take more control of the request flow from the LUA code. The ATS traffic server does provide such functionality through the LUA API hook to override backend server dynamically from Lua end if ts.client_request.header[Host] == 'example.test.com' then ts.client_request.set_url_host( test1.server.comts.client_request.set_url_port(80)elsif ts.client_request.header[Host] == 'example2.test.com' then ts.client_request.set_url_host( test2.server.com ) ts.client_request.set_url_port(80)end Is it possible to do the same using Haproxy and Lua? Hi, Yes, you can, but the way is a little bit different. You must create a new sample fetch in Lua. This fetch executes the analysis of your HTTP resuest and returns the backend name. The following code (untested) must set in a lua file: core.register_fetches(choose_backend, function(txn) if txn.sf.req_fhdr(Host) == 'example.test.com' then return backend1 else if txn.sf.req_fhdr(Host) == 'other.domain.com' then return backend2 [...] end return default_backend end); In the haproxy configuration file, you must load the lua file, and use the new declared fetch inyour frontend: global [...] lua-load your-lua-file.lua [...] frontend your_frt [...] use_backend %[lua.choose_backend] [...] Note that your example code seem to choose a backend regarding only the header host. In this case, the lua is useless, you can use the maps to choose your backend. The configuration looks like this: frontend yur_frt [...] use_backend %[req.fhdr(host),tolower,map(host_to_bck.map,default)] [...] The default argument is the name of the default backend. And the file host_to_bck.map contains a mapping between the hostnames (in lower case) and the backend names: example.test.com backend1 other.domain.com backend2 etc... Thierry Thank you, Mrunmayi
