Not sure if my mails to haproxy mailing lists are being blocked.
Hi Admin, Need help here , not sure if my mails to the mailing lists are being blocked. Can you kindly check. regards badari
Info required regarding health check in http mode.
Hi , I am using haproxy as L7 load balancer and in my configuration have enabled L4 level health checks to back end nodes. During testing for long duration saw intermittent healthcheck errors. Have few queries: 1) is it advisable to use L4 health checks for L7 load balancer ? 2) if backend nodes have some internal issue and respond to request with 500 internal server error and close the TCP sessions for duration say 2 mins, would this impact the L4 health checks ? Server tough is capable of receiving new requests. For this test I am using default values for tcp health check timeouts. Thanks Badari
Re: How to allow Client Requests at a given rate
Hi, Thank you for the response. I will try to configure 1 for 10 seconds and check the outcome. regards, badari On Thu, Apr 25, 2019 at 12:00 AM PiBa-NL wrote: > Hi Badari, > > Op 24-4-2019 om 8:13 schreef Badari Prasad: > > Hi > Any inputs on this issue mentioned in earlier mail. Rate limiting is > not kicking in properly for about 60 secs with the config > > frontend apiGateWay2 > bind 0.0.0.0:11002 > > mode http > option forwardfor > stick-table type string size 1m expire 1m store http_req_rate(1m) > http-request set-var(req.rate_limit) > path,map_sub(/etc/haproxy/maps/apiGateWay2_rates.map) > http-request set-var(req.asname) > path,map_sub(/etc/haproxy/maps/apiGateWay2_path2as.map) > > http-request set-var(req.request_rate) > var(req.asname),table_http_req_rate(apiGateWay2) > acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0 > http-request deny deny_status 429 if rate_abuse > http-request track-sc0 var(req.asname) > use_backend nodes > > thanks > badari > > On Mon, Apr 22, 2019 at 8:47 PM Badari Prasad wrote: > >> Hi Igor, >> I am using the configuration mentioned in this mail thread for rate >> limiting. For some reason the rate limiting is not applied properly for >> say 60 to 120 seconds... >> My configuration is as follows >> >> frontend apiGateWay2 >> bind 0.0.0.0:11002 >> >> mode http >> option forwardfor >> stick-table type string size 1m expire 1m store http_req_rate(1m) >> http-request set-var(req.rate_limit) >> path,map_sub(/etc/haproxy/maps/apiGateWay2_rates.map) >> http-request set-var(req.asname) >> path,map_sub(/etc/haproxy/maps/apiGateWay2_path2as.map) >> >> http-request set-var(req.request_rate) >> var(req.asname),table_http_req_rate(apiGateWay2) >> acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0 >> http-request deny deny_status 429 if rate_abuse >> http-request track-sc0 var(req.asname) >> use_backend nodes >> >> And contents of tile apiGateWay2_rates.map are : I would want to limit >> 100,000 request per minute on uri containing AS0002 or A01 >> /AS2/ 10 >> /A01/ 10 >> >> >> And contents of apiGateWay2_path2as.map file are: >> /A01/ A01 >> /AS2/ AS2 >> >> >> And stats from haproxy sticky tables : >> >>>>> load with url containing A1 and then with AS2 >>>> >> >> root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat >> unix:/var/lib/haproxy/stats stdio >> # table: api_gateway, type: string, size:1048576, used:2 >> 0x14c8090: key=A01 use=0 exp=56494 http_req_rate(6)=48583 >> 0x14f6fb0: key=AS2 use=0 exp=59998 http_req_rate(6)=38 >> >> root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat >> unix:/var/lib/haproxy/stats stdio >> # table: api_gateway, type: string, size:1048576, used:2 >> 0x14c8090: key=A01 use=0 exp=7 http_req_rate(6)=48583 >> 0x14f6fb0: key=AS2 use=0 exp=6 http_req_rate(6)=2807 >> >> root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat >> unix:/var/lib/haproxy/stats stdio >> # table: api_gateway, type: string, size:1048576, used:2 >> 0x14c8090: key=A01 use=0 exp=52736 http_req_rate(6)=48583 >> 0x14f6fb0: key=AS2 use=2 exp=6 http_req_rate(6)=27815 >> >> >> After 60 secs when load with AS2 is running >> root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat >> unix:/var/lib/haproxy/stats stdio >> # table: api_gateway, type: string, size:1048576, used:1 >> 0x14f6fb0: key=AS2 use=3 exp=6 http_req_rate(6)=11 >> >> >> Rate of HTTP request received at back end node which is just a HTTP echo >> server absolutely no processing done here: >> >> start of test >>> >> E0422 10:59:10.406466 18653 EchoServer.cpp:117] >> > current rate : 1 >> E0422 10:59:11.406616 18653 EchoServer.cpp:117] >> > current rate : 2742 >> E0422 10:59:12.406698 18653 EchoServer.cpp:117] >> > current rate : 6330 >> E0422 10:59:13.406762 18653 EchoServer.cpp:117] >> > current rate : 8729 >> E0422 10:59:14.406828 18653 EchoServer.cpp:117] >> =
Re: How to allow Client Requests at a given rate
Hi Any inputs on this issue mentioned in earlier mail. Rate limiting is not kicking in properly for about 60 secs with the config frontend apiGateWay2 bind 0.0.0.0:11002 mode http option forwardfor stick-table type string size 1m expire 1m store http_req_rate(1m) http-request set-var(req.rate_limit) path,map_sub(/etc/haproxy/maps/apiGateWay2_rates.map) http-request set-var(req.asname) path,map_sub(/etc/haproxy/maps/apiGateWay2_path2as.map) http-request set-var(req.request_rate) var(req.asname),table_http_req_rate(apiGateWay2) acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0 http-request deny deny_status 429 if rate_abuse http-request track-sc0 var(req.asname) use_backend nodes thanks badari On Mon, Apr 22, 2019 at 8:47 PM Badari Prasad wrote: > Hi Igor, > I am using the configuration mentioned in this mail thread for rate > limiting. For some reason the rate limiting is not applied properly for > say 60 to 120 seconds... > My configuration is as follows > > frontend apiGateWay2 > bind 0.0.0.0:11002 > > mode http > option forwardfor > stick-table type string size 1m expire 1m store http_req_rate(1m) > http-request set-var(req.rate_limit) > path,map_sub(/etc/haproxy/maps/apiGateWay2_rates.map) > http-request set-var(req.asname) > path,map_sub(/etc/haproxy/maps/apiGateWay2_path2as.map) > > http-request set-var(req.request_rate) > var(req.asname),table_http_req_rate(apiGateWay2) > acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0 > http-request deny deny_status 429 if rate_abuse > http-request track-sc0 var(req.asname) > use_backend nodes > > And contents of tile apiGateWay2_rates.map are : I would want to limit > 100,000 request per minute on uri containing AS0002 or A01 > /AS2/ 10 > /A01/ 10 > > > And contents of apiGateWay2_path2as.map file are: > /A01/ A01 > /AS2/ AS2 > > > And stats from haproxy sticky tables : > >>>>> load with url containing A1 and then with AS2 >>>> > > root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat > unix:/var/lib/haproxy/stats stdio > # table: api_gateway, type: string, size:1048576, used:2 > 0x14c8090: key=A01 use=0 exp=56494 http_req_rate(6)=48583 > 0x14f6fb0: key=AS2 use=0 exp=59998 http_req_rate(6)=38 > > root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat > unix:/var/lib/haproxy/stats stdio > # table: api_gateway, type: string, size:1048576, used:2 > 0x14c8090: key=A01 use=0 exp=7 http_req_rate(6)=48583 > 0x14f6fb0: key=AS2 use=0 exp=6 http_req_rate(6)=2807 > > root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat > unix:/var/lib/haproxy/stats stdio > # table: api_gateway, type: string, size:1048576, used:2 > 0x14c8090: key=A01 use=0 exp=52736 http_req_rate(6)=48583 > 0x14f6fb0: key=AS2 use=2 exp=6 http_req_rate(6)=27815 > > >> After 60 secs when load with AS2 is running > root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat > unix:/var/lib/haproxy/stats stdio > # table: api_gateway, type: string, size:1048576, used:1 > 0x14f6fb0: key=AS2 use=3 exp=6 http_req_rate(6)=11 > > > Rate of HTTP request received at back end node which is just a HTTP echo > server absolutely no processing done here: > >> start of test >>> > E0422 10:59:10.406466 18653 EchoServer.cpp:117] > > current rate : 1 > E0422 10:59:11.406616 18653 EchoServer.cpp:117] > > current rate : 2742 > E0422 10:59:12.406698 18653 EchoServer.cpp:117] > > current rate : 6330 > E0422 10:59:13.406762 18653 EchoServer.cpp:117] > > current rate : 8729 > E0422 10:59:14.406828 18653 EchoServer.cpp:117] > > current rate : 11832 > E0422 10:59:15.407163 18653 EchoServer.cpp:117] > > current rate : 12323 > E0422 10:59:16.407294 18653 EchoServer.cpp:117] > > current rate : 12556 > E0422 10:59:17.408223 18653 EchoServer.cpp:117] > > current rate : 12962 > E0422 10:59:18.408849 18653 EchoServer.cpp:117] > > current rate : 13815 > E0422 10:59:19.408854 18653 EchoServer.cpp:117] > > current rate : 16224 > E0422 10
Re: How to allow Client Requests at a given rate
Hi Igor, I am using the configuration mentioned in this mail thread for rate limiting. For some reason the rate limiting is not applied properly for say 60 to 120 seconds... My configuration is as follows frontend apiGateWay2 bind 0.0.0.0:11002 mode http option forwardfor stick-table type string size 1m expire 1m store http_req_rate(1m) http-request set-var(req.rate_limit) path,map_sub(/etc/haproxy/maps/apiGateWay2_rates.map) http-request set-var(req.asname) path,map_sub(/etc/haproxy/maps/apiGateWay2_path2as.map) http-request set-var(req.request_rate) var(req.asname),table_http_req_rate(apiGateWay2) acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0 http-request deny deny_status 429 if rate_abuse http-request track-sc0 var(req.asname) use_backend nodes And contents of tile apiGateWay2_rates.map are : I would want to limit 100,000 request per minute on uri containing AS0002 or A01 /AS2/ 10 /A01/ 10 And contents of apiGateWay2_path2as.map file are: /A01/ A01 /AS2/ AS2 And stats from haproxy sticky tables : > load with url containing A1 and then with AS2 root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat unix:/var/lib/haproxy/stats stdio # table: api_gateway, type: string, size:1048576, used:2 0x14c8090: key=A01 use=0 exp=56494 http_req_rate(6)=48583 0x14f6fb0: key=AS2 use=0 exp=59998 http_req_rate(6)=38 root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat unix:/var/lib/haproxy/stats stdio # table: api_gateway, type: string, size:1048576, used:2 0x14c8090: key=A01 use=0 exp=7 http_req_rate(6)=48583 0x14f6fb0: key=AS2 use=0 exp=6 http_req_rate(6)=2807 root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat unix:/var/lib/haproxy/stats stdio # table: api_gateway, type: string, size:1048576, used:2 0x14c8090: key=A01 use=0 exp=52736 http_req_rate(6)=48583 0x14f6fb0: key=AS2 use=2 exp=6 http_req_rate(6)=27815 >> After 60 secs when load with AS2 is running root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat unix:/var/lib/haproxy/stats stdio # table: api_gateway, type: string, size:1048576, used:1 0x14f6fb0: key=AS2 use=3 exp=6 http_req_rate(6)=11 Rate of HTTP request received at back end node which is just a HTTP echo server absolutely no processing done here: >> start of test >>> E0422 10:59:10.406466 18653 EchoServer.cpp:117] > current rate : 1 E0422 10:59:11.406616 18653 EchoServer.cpp:117] > current rate : 2742 E0422 10:59:12.406698 18653 EchoServer.cpp:117] > current rate : 6330 E0422 10:59:13.406762 18653 EchoServer.cpp:117] > current rate : 8729 E0422 10:59:14.406828 18653 EchoServer.cpp:117] > current rate : 11832 E0422 10:59:15.407163 18653 EchoServer.cpp:117] > current rate : 12323 E0422 10:59:16.407294 18653 EchoServer.cpp:117] > current rate : 12556 E0422 10:59:17.408223 18653 EchoServer.cpp:117] > current rate : 12962 E0422 10:59:18.408849 18653 EchoServer.cpp:117] > current rate : 13815 E0422 10:59:19.408854 18653 EchoServer.cpp:117] > current rate : 16224 E0422 10:59:22.603286 18653 EchoServer.cpp:117] > current rate : 2488 >>> until almost 60 no http request are received to back ends >> this time gap varies with every run ... >>> after 60 secs rate limits are applied properly E0422 11:00:07.690192 18653 EchoServer.cpp:117] > current rate : 1 E0422 11:00:10.411736 18653 EchoServer.cpp:117] > current rate : 1 E0422 11:00:11.412317 18653 EchoServer.cpp:117] > current rate : 1679 E0422 11:00:12.412369 18653 EchoServer.cpp:117] > current rate : 1667 E0422 11:00:13.451706 18653 EchoServer.cpp:117] > current rate : 1668 E0422 11:00:14.453778 18653 EchoServer.cpp:117] > current rate : 1668 E0422 11:00:15.457597 18653 EchoServer.cpp:117] > current rate : 1645 E0422 11:00:16.458938 18653 EchoServer.cpp:117] > current rate : 1762 E0422 11:00:17.470010 18653 EchoServer.cpp:117] > current rate : 1598 Can I get some info on the issue, is this know issue or am I missing some config for rate limiting to be applied
Re: Issue with systemd haproxy.service on RHEL 7.4
Patrick, William, systemctl daemon-reload did the trick, initially i did copy haproxy.service.in and repalced it with proper haproxy.service. systemctl daemon-reload reloaded the latest file and issue is resolved now. Thanks badari On Thu, Mar 7, 2019 at 10:45 PM William Lallemand wrote: > On Thu, Mar 07, 2019 at 09:38:01PM +0530, Badari Prasad wrote: > > Hi > > RHEL 7.4 comes with haproxy 1.5.18, I wanted use latest version of > > haproxy 1.9.4. So source code comes with haproxy.service.in [ > > > https://github.com/haproxy/haproxy/blob/master/contrib/systemd/haproxy.service.in > ] > > . > > Executing make in the dir contrib/systemd/ creates haproxy.service. I > tried > > to copy this generarted file in : > /usr/lib/systemd/system/haproxy.service . > > With this I see lots of errors : > > #systemctl status haproxy.service > > ● haproxy.service - HAProxy Load Balancer > >Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled; > > vendor preset: disabled) > >Active: inactive (dead) > > > > Mar 07 03:13:47 local systemd[1]: > > [/usr/lib/systemd/system/haproxy.service:9] Executable path is not > > absolute, ignoring: @SBINDIR@/haproxy -f $CONFIG -c -q > > Mar 07 03:13:47 local systemd[1]: haproxy.service lacks both ExecStart= > and > > ExecStop= setting. Refusing. > > Mar 07 03:13:47 local systemd[1]: > > [/usr/lib/systemd/system/haproxy.service:3] Failed to add dependency on > > =syslog.target, ignoring: Invalid argument > > Mar 07 03:13:47 local systemd[1]: > > [/usr/lib/systemd/system/haproxy.service:7] Executable path is not > > absolute, ignoring: @SBINDIR@/haproxy -f $CONFIG -c -q > > Mar 07 03:13:47 local systemd[1]: > > [/usr/lib/systemd/system/haproxy.service:8] Executable path is not > > absolute, ignoring: @SBINDIR@/haproxy -Ws -f $CONFIG -p $PIDFILE > > Mar 07 03:13:47 local systemd[1]: > > [/usr/lib/systemd/system/haproxy.service:9] Executable path is not > > absolute, ignoring: @SBINDIR@/haproxy -f $CONFIG -c -q > > Mar 07 03:13:47 local systemd[1]: haproxy.service lacks both ExecStart= > and > > ExecStop= setting. Refusing. > > Mar 07 03:29:51 local systemd[1]: Unit haproxy.service cannot be reloaded > > because it is inactive. > > Mar 07 09:28:57 local systemd[1]: Unit haproxy.service cannot be reloaded > > because it is inactive. > > Mar 07 09:35:45 local systemd[1]: Unit haproxy.service cannot be reloaded > > because it is inactive. > > > > > > Have attached haproxy.service for reference. Can I get some pointers to > > resolve this issue. > > > > Thanks > > badari > > Looks like you copied the haproxy.service.in instead of the > haproxy.service generated. > > "@SBINDIR@" is a macro that is present in the .in but replaced in the > .service. > > -- > William Lallemand >
Issue with systemd haproxy.service on RHEL 7.4
Hi RHEL 7.4 comes with haproxy 1.5.18, I wanted use latest version of haproxy 1.9.4. So source code comes with haproxy.service.in [ https://github.com/haproxy/haproxy/blob/master/contrib/systemd/haproxy.service.in] . Executing make in the dir contrib/systemd/ creates haproxy.service. I tried to copy this generarted file in : /usr/lib/systemd/system/haproxy.service . With this I see lots of errors : #systemctl status haproxy.service ● haproxy.service - HAProxy Load Balancer Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled; vendor preset: disabled) Active: inactive (dead) Mar 07 03:13:47 local systemd[1]: [/usr/lib/systemd/system/haproxy.service:9] Executable path is not absolute, ignoring: @SBINDIR@/haproxy -f $CONFIG -c -q Mar 07 03:13:47 local systemd[1]: haproxy.service lacks both ExecStart= and ExecStop= setting. Refusing. Mar 07 03:13:47 local systemd[1]: [/usr/lib/systemd/system/haproxy.service:3] Failed to add dependency on =syslog.target, ignoring: Invalid argument Mar 07 03:13:47 local systemd[1]: [/usr/lib/systemd/system/haproxy.service:7] Executable path is not absolute, ignoring: @SBINDIR@/haproxy -f $CONFIG -c -q Mar 07 03:13:47 local systemd[1]: [/usr/lib/systemd/system/haproxy.service:8] Executable path is not absolute, ignoring: @SBINDIR@/haproxy -Ws -f $CONFIG -p $PIDFILE Mar 07 03:13:47 local systemd[1]: [/usr/lib/systemd/system/haproxy.service:9] Executable path is not absolute, ignoring: @SBINDIR@/haproxy -f $CONFIG -c -q Mar 07 03:13:47 local systemd[1]: haproxy.service lacks both ExecStart= and ExecStop= setting. Refusing. Mar 07 03:29:51 local systemd[1]: Unit haproxy.service cannot be reloaded because it is inactive. Mar 07 09:28:57 local systemd[1]: Unit haproxy.service cannot be reloaded because it is inactive. Mar 07 09:35:45 local systemd[1]: Unit haproxy.service cannot be reloaded because it is inactive. Have attached haproxy.service for reference. Can I get some pointers to resolve this issue. Thanks badari haproxy.service Description: Binary data
Re: Require info on ACL for rate limiting on per URL basis.
Thanks for response and information. And I came across this link : https://www.haproxy.com/blog/introduction-to-haproxy-maps/#rate-limiting-by-url-path/ for url based rate-limit. With this exact config (copied exact config to haproxy.cfg and also created maps/rates.map file) I made one observation that the when I start a load towards haproxy, and if the rate for a url is say 100 requests per second. haproxy does send only 100 requests to backend server and no more requests are sent to backend and rest of requests are getting rejected with 429 ! And from admin socket : root@VM-Ubuntu:/etc/haproxy# echo "show table api_gateway" | socat unix:/var/lib/haproxy/stats stdio # table: api_gateway, type: binary, size:1048576, used:1 0x1bbb600: key=42EE33090A25091E use=0 exp=1 http_req_rate(1)=17351 Any info on whats going on with this config ? Thanks badari On Fri, Feb 22, 2019 at 1:03 PM Jarno Huuskonen wrote: > Hi, > > On Thu, Feb 21, Badari Prasad wrote: > > But by replacing 'src' with 'path', rate-limiting did not work. My > current > > config after the change is : > > > > backend st_src_as2_monte > > stick-table type string len 64 size 1m expire 1s store > http_req_rate(1s) > > (for testing it helps to use longer expire eg. 60s and longer rate > (60s). Then it's easier to use admin socket to view stick table values > to see if the stick table is updated etc). > > > frontend scef > > bind 0.0.0.0:80 > > bind 0.0.0.0:443 ssl crt /etc/ssl/private/as1.pem > > mode http > > option forwardfor > > > > http-request track-sc1 path table st_src_as2_monte > > You're using sc1 here. > > > acl monte_as2_api_url path_beg /api/v1/monitoring-event/A02/ > > #500 requests per second. > > acl monte_as1_exceeds_limit sc0_http_req_rate(st_src_as1_monte) gt > 500 > > And sc0 here, change this to sc1 (or use track-sc1). > > -Jarno > > > http-request deny deny_status 429 if monte_as2_api_url > > monte_as2_exceeds_limit > > use_backend nodes > > Appreciate the response on this, and going further I will have to extend > > the rate limiting to multiple url's . > > > > > > Thanks > > badari > > > > > > > > On Wed, Feb 20, 2019 at 11:13 PM Jarno Huuskonen > > > wrote: > > > > > Hi, > > > > > > On Wed, Feb 20, Badari Prasad wrote: > > > > Thank you for responding. Came up with based on the inputs: > > > > > > > > #printf "as2monte" | mkpasswd --stdin --method=md5 > > > > userlist AuthUsers_MONTE_AS2 > > > > user appuser_as2 password $1$t25fZ7Oe$bjthsMcXgbCt2EJvQo8r0/ > > > > > > > > backend st_src_as2_monte > > > > stick-table type string len 64 size 1000 expire 1s store > > > > http_req_rate(1s) > > > > > > > > frontend scef > > > > bind 0.0.0.0:80 > > > > bind 0.0.0.0:443 ssl crt /etc/ssl/private/as1.pem > > > > mode http > > > > #option httpclose > > > > option forwardfor > > > > > > > > acl monte_as2_api_url url_beg /api/v1/monitoring-event/A02/ > > > > #500 requests per second. > > > > acl monte_as2_exceeds_limit src_http_req_rate(st_src_as2_monte) > gt > > > 500 > > > > http-request track-sc1 src table st_src_as2_monte unless > > > > monte_as2_exceeds_limit > > > > http-request deny deny_status 429 if monte_as2_api_url > > > > monte_as2_exceeds_limit > > > > > > I'm confused :) what your requirements are but I think with > > > this configuration each src address can have rate 500 to > > > /api/v1/monitoring-event/A02/. (so with 10 different src addresses > > > you can have 5000 rate to /api/v1/monitoring-event/A02/). > > > > > > (And you're using type string stick table, type ip or ipv6 is better > > > fit for tracking src). > > > > > > But if it fits your requirements then I'm glad you found a working > > > solution. > > > > > > -Jarno > > > > > > > http-request auth realm basicauth if monte_as2_api_url > > > > !authorized_monte_as2 > > > > > > > > use_backend nodes > > > > > > > > With this config I was able to rate limit per url basis. > > > > > > > > Thanks > > > > badari > > > > > > > > > > > > > > > > On Tue, Feb 19, 201
Re: Require info on ACL for rate limiting on per URL basis.
Hi, Thank you for response, I would want to have rate-limiting on url no matter what src ip is. So one difference I noticed is : http-request track-sc1 src table st_src_as2_monte unless monte_as2_exceeds_limit >From your example I see: http-request track-sc0 path table test_be But by replacing 'src' with 'path', rate-limiting did not work. My current config after the change is : backend st_src_as2_monte stick-table type string len 64 size 1m expire 1s store http_req_rate(1s) frontend scef bind 0.0.0.0:80 bind 0.0.0.0:443 ssl crt /etc/ssl/private/as1.pem mode http option forwardfor http-request track-sc1 path table st_src_as2_monte acl monte_as2_api_url path_beg /api/v1/monitoring-event/A02/ #500 requests per second. acl monte_as1_exceeds_limit sc0_http_req_rate(st_src_as1_monte) gt 500 http-request deny deny_status 429 if monte_as2_api_url monte_as2_exceeds_limit use_backend nodes Appreciate the response on this, and going further I will have to extend the rate limiting to multiple url's . Thanks badari On Wed, Feb 20, 2019 at 11:13 PM Jarno Huuskonen wrote: > Hi, > > On Wed, Feb 20, Badari Prasad wrote: > > Thank you for responding. Came up with based on the inputs: > > > > #printf "as2monte" | mkpasswd --stdin --method=md5 > > userlist AuthUsers_MONTE_AS2 > > user appuser_as2 password $1$t25fZ7Oe$bjthsMcXgbCt2EJvQo8r0/ > > > > backend st_src_as2_monte > > stick-table type string len 64 size 1000 expire 1s store > > http_req_rate(1s) > > > > frontend scef > > bind 0.0.0.0:80 > > bind 0.0.0.0:443 ssl crt /etc/ssl/private/as1.pem > > mode http > > #option httpclose > > option forwardfor > > > > acl monte_as2_api_url url_beg /api/v1/monitoring-event/A02/ > > #500 requests per second. > > acl monte_as2_exceeds_limit src_http_req_rate(st_src_as2_monte) gt > 500 > > http-request track-sc1 src table st_src_as2_monte unless > > monte_as2_exceeds_limit > > http-request deny deny_status 429 if monte_as2_api_url > > monte_as2_exceeds_limit > > I'm confused :) what your requirements are but I think with > this configuration each src address can have rate 500 to > /api/v1/monitoring-event/A02/. (so with 10 different src addresses > you can have 5000 rate to /api/v1/monitoring-event/A02/). > > (And you're using type string stick table, type ip or ipv6 is better > fit for tracking src). > > But if it fits your requirements then I'm glad you found a working > solution. > > -Jarno > > > http-request auth realm basicauth if monte_as2_api_url > > !authorized_monte_as2 > > > > use_backend nodes > > > > With this config I was able to rate limit per url basis. > > > > Thanks > > badari > > > > > > > > On Tue, Feb 19, 2019 at 10:01 PM Jarno Huuskonen > > > wrote: > > > > > Hi, > > > > > > On Mon, Feb 11, Badari Prasad wrote: > > > >I want to rate limit based on url > > > > [/api/v1/monitoring-event/A01, /api/v1/client1/transfer_data, > > > > /api/v1/client2/transfer_data ] no matter what the source ip > address > > > is. > > > > > > Something like this might help you. Unfortunately at the moment > > > I don't have time to create a better example. > > > > > > acl api_a1 path_beg /a1 > > > acl api_b1 path_beg /b1 > > > acl rate_5 sc0_http_req_rate(test_be) gt 5 > > > acl rate_15 sc0_http_req_rate(test_be) gt 15 > > > > > > # You might want to add acl so you'll only track paths you're > > > # interested in. > > > http-request track-sc0 path table test_be > > > # if you want to track only /a1 /b1 part of path > > > # you can use for example field converter: > > > #http-request track-sc0 path,field(1,/,2) table test_be > > > #http-request set-header X-Rate %[sc0_http_req_rate(test_be)] > > > > > > http-request deny deny_status 429 if api_a1 rate_5 > > > http-request deny deny_status 403 if api_b1 rate_15 > > > > > > # adjust len and size etc. to your needs > > > backend test_be > > > stick-table type string len 40 size 20 expire 180s store > > > http_req_rate(60s) > > > > > > -Jarno > > > > > > > On Mon, Feb 11, 2019 at 7:34 PM Jarno Huuskonen < > jarno.huusko...@uef.fi> > > > > wrote: > > > > > > > &
Re: Require info on ACL for rate limiting on per URL basis.
Hi Jarno and Marco,
Thank you for responding. Came up with based on the inputs:
#printf "as2monte" | mkpasswd --stdin --method=md5
userlist AuthUsers_MONTE_AS2
user appuser_as2 password $1$t25fZ7Oe$bjthsMcXgbCt2EJvQo8r0/
backend st_src_as2_monte
stick-table type string len 64 size 1000 expire 1s store
http_req_rate(1s)
frontend scef
bind 0.0.0.0:80
bind 0.0.0.0:443 ssl crt /etc/ssl/private/as1.pem
mode http
#option httpclose
option forwardfor
acl monte_as2_api_url url_beg /api/v1/monitoring-event/A02/
#500 requests per second.
acl monte_as2_exceeds_limit src_http_req_rate(st_src_as2_monte) gt 500
http-request track-sc1 src table st_src_as2_monte unless
monte_as2_exceeds_limit
http-request deny deny_status 429 if monte_as2_api_url
monte_as2_exceeds_limit
http-request auth realm basicauth if monte_as2_api_url
!authorized_monte_as2
use_backend nodes
With this config I was able to rate limit per url basis.
Thanks
badari
On Tue, Feb 19, 2019 at 10:01 PM Jarno Huuskonen
wrote:
> Hi,
>
> On Mon, Feb 11, Badari Prasad wrote:
> >I want to rate limit based on url
> > [/api/v1/monitoring-event/A01, /api/v1/client1/transfer_data,
> > /api/v1/client2/transfer_data ] no matter what the source ip address
> is.
>
> Something like this might help you. Unfortunately at the moment
> I don't have time to create a better example.
>
> acl api_a1 path_beg /a1
> acl api_b1 path_beg /b1
> acl rate_5 sc0_http_req_rate(test_be) gt 5
> acl rate_15 sc0_http_req_rate(test_be) gt 15
>
> # You might want to add acl so you'll only track paths you're
> # interested in.
> http-request track-sc0 path table test_be
> # if you want to track only /a1 /b1 part of path
> # you can use for example field converter:
> #http-request track-sc0 path,field(1,/,2) table test_be
> #http-request set-header X-Rate %[sc0_http_req_rate(test_be)]
>
> http-request deny deny_status 429 if api_a1 rate_5
> http-request deny deny_status 403 if api_b1 rate_15
>
> # adjust len and size etc. to your needs
> backend test_be
> stick-table type string len 40 size 20 expire 180s store
> http_req_rate(60s)
>
> -Jarno
>
> > On Mon, Feb 11, 2019 at 7:34 PM Jarno Huuskonen
> > wrote:
> >
> > > Hi,
> > >
> > > On Mon, Feb 11, Badari Prasad wrote:
> > > > Thank you for the response. I came up with my own haproxy cfg,
> where
> > > i
> > > > would want to rate limit based on event name and client id in url.
> > > > URL ex : /api/v1//
> > > >
> > > > Have attached a file for my haproxy cfg. But it does not seems to be
> > > rate
> > > > limiting the incoming requests.
> > >
> > > > backend st_src_monte
> > > > stick-table type string size 1m expire 10s store
> http_req_rate(10s)
> > > > ...
> > > >
> > > >acl monte_as1_exceeds_limit src_http_req_rate(st_src_as1_monte)
> gt 990
> > > >acl monte_in_limit src_http_req_rate(st_src_as1_monte) lt 1000
> > > >http-request track-sc0 src table st_src_as1_monte
> > >
> > > There's no st_src_as1_monte table in your example config, there's
> > > st_src_monte table.
> > >
> > > >http-request deny deny_status 429 if { path_beg
> > > /api/v1/monitoring-event/A01 AND monte_as1_exceeds_limit }
> > >
> > > You're tracking connections with src, but the stick table is type
> string,
> > > have you checked from admin socket that the stick table has entries,
> > > something like:
> > > echo 'show table st_src_monte' | nc -U /var/lib/haproxy/stats
> > > (insted of nc -U, socat stdio /var/lib/haproxy/stats should also work).
> > >
> > > If you want to track src ip, then stick-table type ip or ipv6 is
> > > probably better.
> > >
> > > >> I would want to configure 1000 tps for url
> > > /api/v1/client1/transfer_data or
> > > >> 500 tps for /api/v1/client2/user_data and so on
> > >
> > > Do you mean that only 1000 tps goes to
> > > /api/v1/client1/transfer_data (no matter what the source ip addresses
> > > are) or each source ip can send 1000 tps to
> /api/v1/client1/transfer_data ?
>
> --
> Jarno Huuskonen
>
Re: Require info on ACL for rate limiting on per URL basis.
Hi Jarno ,
I want to rate limit based on url
[/api/v1/monitoring-event/A01, /api/v1/client1/transfer_data,
/api/v1/client2/transfer_data ] no matter what the source ip address is.
thanks
badari
On Mon, Feb 11, 2019 at 7:34 PM Jarno Huuskonen
wrote:
> Hi,
>
> On Mon, Feb 11, Badari Prasad wrote:
> > Thank you for the response. I came up with my own haproxy cfg, where
> i
> > would want to rate limit based on event name and client id in url.
> > URL ex : /api/v1//
> >
> > Have attached a file for my haproxy cfg. But it does not seems to be
> rate
> > limiting the incoming requests.
>
> > backend st_src_monte
> > stick-table type string size 1m expire 10s store http_req_rate(10s)
> > ...
> >
> >acl monte_as1_exceeds_limit src_http_req_rate(st_src_as1_monte) gt 990
> >acl monte_in_limit src_http_req_rate(st_src_as1_monte) lt 1000
> >http-request track-sc0 src table st_src_as1_monte
>
> There's no st_src_as1_monte table in your example config, there's
> st_src_monte table.
>
> >http-request deny deny_status 429 if { path_beg
> /api/v1/monitoring-event/A01 AND monte_as1_exceeds_limit }
>
> You're tracking connections with src, but the stick table is type string,
> have you checked from admin socket that the stick table has entries,
> something like:
> echo 'show table st_src_monte' | nc -U /var/lib/haproxy/stats
> (insted of nc -U, socat stdio /var/lib/haproxy/stats should also work).
>
> If you want to track src ip, then stick-table type ip or ipv6 is
> probably better.
>
> >> I would want to configure 1000 tps for url
> /api/v1/client1/transfer_data or
> >> 500 tps for /api/v1/client2/user_data and so on
>
> Do you mean that only 1000 tps goes to
> /api/v1/client1/transfer_data (no matter what the source ip addresses
> are) or each source ip can send 1000 tps to /api/v1/client1/transfer_data ?
>
> -Jarno
>
> --
> Jarno Huuskonen
>
Re: Require info on ACL for rate limiting on per URL basis.
Ok Thank you will check this.
On Mon, Feb 11, 2019 at 6:22 PM Marco Corte wrote:
> Il 2019-02-11 6:36 Badari Prasad ha scritto:
> > Hi Marco
> > Thank you for the response. I came up with my own haproxy cfg,
> > where i would want to rate limit based on event name and client id in
> > url.
> > URL ex : /api/v1//
> >
> > Have attached a file for my haproxy cfg. But it does not seems to be
> > rate limiting the incoming requests.
> >
>
> Hi
>
> I think that
>
>http-request deny deny_status 429 if { path_beg
> /api/v1/monitoring-event/A01 AND monte_as1_exceeds_limit }
>
> is incorrect. I guess that the rule should be
>
>http-request deny deny_status 429 if { path_beg
> /api/v1/monitoring-event/A01 } monte_as1_exceeds_limit
>
>
> But... since I always do a mess in mixing ACL formats, I would rewrite
> the rule as
>
> acl api_url path_beg /api/v1/monitoring-event/A01
> http-request deny deny_status 429 if api_url monte_as1_exceeds_limit
>
>
> Ciao!
>
> .marcoc
>
>
Re: Require info on ACL for rate limiting on per URL basis.
Hi Marco Thank you for the response. I came up with my own haproxy cfg, where i would want to rate limit based on event name and client id in url. URL ex : /api/v1// Have attached a file for my haproxy cfg. But it does not seems to be rate limiting the incoming requests. Regards, badari On Fri, Feb 8, 2019 at 8:28 PM Marco Corte wrote: > Il 2019-02-08 14:46 Badari Prasad ha scritto: > > > Can I get some reference for a url based rate limiting, so that I can > > build on this > > Hi! > > I found there two posts very valuable > > https://www.haproxy.com/blog/introduction-to-haproxy-stick-tables/ > > https://www.haproxy.com/blog/application-layer-ddos-attack-protection-with-haproxy/ > > Ciao! > > .marcoc > > haproxy_test.cfg Description: Binary data
Require info on ACL for rate limiting on per URL basis.
Hi ,
I am a novice for HAProxy, was checking if HAProxy can support rate
limiting per url basis.
I did check some examples and documentation, amount of info is
overwhelming.
My back end server exposes url's say
1) /api/v1/{client_name}/transfer_data
Ex: /api/v1/client1/transfer_data or /api/v1/client2/transfer_data
2) /api/v1/{client_name}/user_data
Ex: /api/v1/client1/user_data or /api/v1/client2/user_data
where client1 and client2 are client identifiers which are known ahead at
haproxy.
I would want to configure 1000 tps for url /api/v1/client1/transfer_data or
500 tps for /api/v1/client2/user_data and so on
I did try out some config but did not help much ( based on this link:
https://jve.linuxwall.info/ressources/taf/haproxy-aws/#id28 )
Can I get some reference for a url based rate limiting, so that I can build
on this
Thanks in advance.
Badari
