Hi
Any inputs on this issue mentioned in earlier mail. Rate limiting is
not kicking in properly for about 60 secs with the config
frontend apiGateWay2
bind 0.0.0.0:11002
mode http
option forwardfor
stick-table type string size 1m expire 1m store http_req_rate(1m)
http-request set-var(req.rate_limit)
path,map_sub(/etc/haproxy/maps/apiGateWay2_rates.map)
http-request set-var(req.asname)
path,map_sub(/etc/haproxy/maps/apiGateWay2_path2as.map)
http-request set-var(req.request_rate)
var(req.asname),table_http_req_rate(apiGateWay2)
acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0
http-request deny deny_status 429 if rate_abuse
http-request track-sc0 var(req.asname)
use_backend nodes
thanks
badari
On Mon, Apr 22, 2019 at 8:47 PM Badari Prasad <badari...@gmail.com> wrote:
> Hi Igor,
> I am using the configuration mentioned in this mail thread for rate
> limiting. For some reason the rate limiting is not applied properly for
> say 60 to 120 seconds...
> My configuration is as follows
>
> frontend apiGateWay2
> bind 0.0.0.0:11002
>
> mode http
> option forwardfor
> stick-table type string size 1m expire 1m store http_req_rate(1m)
> http-request set-var(req.rate_limit)
> path,map_sub(/etc/haproxy/maps/apiGateWay2_rates.map)
> http-request set-var(req.asname)
> path,map_sub(/etc/haproxy/maps/apiGateWay2_path2as.map)
>
> http-request set-var(req.request_rate)
> var(req.asname),table_http_req_rate(apiGateWay2)
> acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0
> http-request deny deny_status 429 if rate_abuse
> http-request track-sc0 var(req.asname)
> use_backend nodes
>
> And contents of tile apiGateWay2_rates.map are : I would want to limit
> 100,000 request per minute on uri containing AS0002 or A000001
> /AS00002/ 100000
> /A000001/ 100000
>
>
> And contents of apiGateWay2_path2as.map file are:
> /A000001/ A000001
> /AS00002/ AS00002
>
>
> And stats from haproxy sticky tables :
> >>>>> load with url containing A00001 and then with AS00002 >>>>
>
> root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat
> unix:/var/lib/haproxy/stats stdio
> # table: api_gateway, type: string, size:1048576, used:2
> 0x14c8090: key=A000001 use=0 exp=56494 http_req_rate(60000)=48583
> 0x14f6fb0: key=AS00002 use=0 exp=59998 http_req_rate(60000)=38
>
> root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat
> unix:/var/lib/haproxy/stats stdio
> # table: api_gateway, type: string, size:1048576, used:2
> 0x14c8090: key=A000001 use=0 exp=55557 http_req_rate(60000)=48583
> 0x14f6fb0: key=AS00002 use=0 exp=60000 http_req_rate(60000)=2807
>
> root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat
> unix:/var/lib/haproxy/stats stdio
> # table: api_gateway, type: string, size:1048576, used:2
> 0x14c8090: key=A000001 use=0 exp=52736 http_req_rate(60000)=48583
> 0x14f6fb0: key=AS00002 use=2 exp=60000 http_req_rate(60000)=27815
>
> >> After 60 secs when load with AS00002 is running
> root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat
> unix:/var/lib/haproxy/stats stdio
> # table: api_gateway, type: string, size:1048576, used:1
> 0x14f6fb0: key=AS00002 use=3 exp=60000 http_req_rate(60000)=100001
>
>
> Rate of HTTP request received at back end node which is just a HTTP echo
> server absolutely no processing done here:
> >> start of test >>>
> E0422 10:59:10.406466 18653 EchoServer.cpp:117]
> ========================================> current rate : 1
> E0422 10:59:11.406616 18653 EchoServer.cpp:117]
> ========================================> current rate : 2742
> E0422 10:59:12.406698 18653 EchoServer.cpp:117]
> ========================================> current rate : 6330
> E0422 10:59:13.406762 18653 EchoServer.cpp:117]
> ========================================> current rate : 8729
> E0422 10:59:14.406828 18653 EchoServer.cpp:117]
> ========================================> current rate : 11832
> E0422 10:59:15.407163 18653 EchoServer.cpp:117]
> ========================================> current rate : 12323
> E0422 10:59:16.407294 18653 EchoServer.cpp:117]
> ========================================> current rate : 12556
> E0422 10:59:17.408223 18653 EchoServer.cpp:117]
> ========================================> current rate : 12962
> E0422 10:59:18.408849 18653 EchoServer.cpp:117]
> ========================================> current rate : 13815
> E0422 10:59:19.408854 18653 EchoServer.cpp:117]
> ========================================> current rate : 16224
> E0422 10:59:22.603286 18653 EchoServer.cpp:117]
> ========================================> current rate : 2488
>
> >>> until almost 60 no http request are received to back ends >> this time
> gap varies with every run ...
> >>> after 60 secs rate limits are applied properly >>>>
> E0422 11:00:07.690192 18653 EchoServer.cpp:117]
> ========================================> current rate : 1
> E0422 11:00:10.411736 18653 EchoServer.cpp:117]
> ========================================> current rate : 1
> E0422 11:00:11.412317 18653 EchoServer.cpp:117]
> ========================================> current rate : 1679
> E0422 11:00:12.412369 18653 EchoServer.cpp:117]
> ========================================> current rate : 1667
> E0422 11:00:13.451706 18653 EchoServer.cpp:117]
> ========================================> current rate : 1668
> E0422 11:00:14.453778 18653 EchoServer.cpp:117]
> ========================================> current rate : 1668
> E0422 11:00:15.457597 18653 EchoServer.cpp:117]
> ========================================> current rate : 1645
> E0422 11:00:16.458938 18653 EchoServer.cpp:117]
> ========================================> current rate : 1762
> E0422 11:00:17.470010 18653 EchoServer.cpp:117]
> ========================================> current rate : 1598
>
>
> Can I get some info on the issue, is this know issue or am I missing some
> config for rate limiting to be applied properly ?
>
> Thanks in advance,
> Badari
>
>
> On Sat, Feb 23, 2019 at 8:48 PM Igor Cicimov <
> ig...@encompasscorporation.com> wrote:
>
>> On Sat, 23 Feb 2019 3:09 pm Santos Das <santos....@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I have a requirement where I need to allow only certain request rate for
>>> a given URL.
>>>
>>> Say /login can be accessed at the rate of 10 RPS. If I get 100 RPS, then
>>> 10 should be allowed and 90 should be denied.
>>>
>>> Any help on how this can be achieved ?
>>>
>>> *I tried to use the sticky table, but once it blocks it blocks for ever.
>>> Please advise.*
>>>
>>>
>>> frontend api_gateway
>>> bind 0.0.0.0:80 <http://0.0.0.0/>
>>> mode http
>>> option forwardfor
>>>
>>> default_backend nodes
>>>
>>> # Set up stick table to track request rates
>>> stick-table type binary len 8 size 1m expire 10s store
>>> http_req_rate(10s)
>>>
>>> # Track client by base32+src (Host header + URL path + src IP)
>>> http-request track-sc0 base32+src
>>>
>>> # Check map file to get rate limit for path
>>> http-request set-var(req.rate_limit)
>>> path,map_beg(/etc/hapee-1.8/maps/rates.map)
>>>
>>> # Client's request rate is tracked
>>> http-request set-var(req.request_rate)
>>> base32+src,table_http_req_rate(api_gateway)
>>>
>>> # Subtract the current request rate from the limit
>>> # If less than zero, set rate_abuse to true
>>> acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0
>>>
>>
>> Shouldn't this be:
>> acl rate_abuse var(req.rate_limit),sub(var(req.request_rate)) lt 0
>>
>>
>>> # Deny if rate abuse
>>> http-request deny deny_status 429 if rate_abuse
>>>
>>> backend nodes
>>> mode http
>>> balance roundrobin
>>> server echoprgm 10.37.9.30:11001 check
>>>
>>>
>>>
>>>
