Hi Igor,
I am using the configuration mentioned in this mail thread for rate
limiting. For some reason the rate limiting is not applied properly for
say 60 to 120 seconds...
My configuration is as follows
frontend apiGateWay2
bind 0.0.0.0:11002
mode http
option forwardfor
stick-table type string size 1m expire 1m store http_req_rate(1m)
http-request set-var(req.rate_limit)
path,map_sub(/etc/haproxy/maps/apiGateWay2_rates.map)
http-request set-var(req.asname)
path,map_sub(/etc/haproxy/maps/apiGateWay2_path2as.map)
http-request set-var(req.request_rate)
var(req.asname),table_http_req_rate(apiGateWay2)
acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0
http-request deny deny_status 429 if rate_abuse
http-request track-sc0 var(req.asname)
use_backend nodes
And contents of tile apiGateWay2_rates.map are : I would want to limit
100,000 request per minute on uri containing AS0002 or A000001
/AS00002/ 100000
/A000001/ 100000
And contents of apiGateWay2_path2as.map file are:
/A000001/ A000001
/AS00002/ AS00002
And stats from haproxy sticky tables :
>>>>> load with url containing A00001 and then with AS00002 >>>>
root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat
unix:/var/lib/haproxy/stats stdio
# table: api_gateway, type: string, size:1048576, used:2
0x14c8090: key=A000001 use=0 exp=56494 http_req_rate(60000)=48583
0x14f6fb0: key=AS00002 use=0 exp=59998 http_req_rate(60000)=38
root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat
unix:/var/lib/haproxy/stats stdio
# table: api_gateway, type: string, size:1048576, used:2
0x14c8090: key=A000001 use=0 exp=55557 http_req_rate(60000)=48583
0x14f6fb0: key=AS00002 use=0 exp=60000 http_req_rate(60000)=2807
root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat
unix:/var/lib/haproxy/stats stdio
# table: api_gateway, type: string, size:1048576, used:2
0x14c8090: key=A000001 use=0 exp=52736 http_req_rate(60000)=48583
0x14f6fb0: key=AS00002 use=2 exp=60000 http_req_rate(60000)=27815
>> After 60 secs when load with AS00002 is running
root@VM-Ubuntu-VM:/etc/bind# echo "show table api_gateway" | socat
unix:/var/lib/haproxy/stats stdio
# table: api_gateway, type: string, size:1048576, used:1
0x14f6fb0: key=AS00002 use=3 exp=60000 http_req_rate(60000)=100001
Rate of HTTP request received at back end node which is just a HTTP echo
server absolutely no processing done here:
>> start of test >>>
E0422 10:59:10.406466 18653 EchoServer.cpp:117]
========================================> current rate : 1
E0422 10:59:11.406616 18653 EchoServer.cpp:117]
========================================> current rate : 2742
E0422 10:59:12.406698 18653 EchoServer.cpp:117]
========================================> current rate : 6330
E0422 10:59:13.406762 18653 EchoServer.cpp:117]
========================================> current rate : 8729
E0422 10:59:14.406828 18653 EchoServer.cpp:117]
========================================> current rate : 11832
E0422 10:59:15.407163 18653 EchoServer.cpp:117]
========================================> current rate : 12323
E0422 10:59:16.407294 18653 EchoServer.cpp:117]
========================================> current rate : 12556
E0422 10:59:17.408223 18653 EchoServer.cpp:117]
========================================> current rate : 12962
E0422 10:59:18.408849 18653 EchoServer.cpp:117]
========================================> current rate : 13815
E0422 10:59:19.408854 18653 EchoServer.cpp:117]
========================================> current rate : 16224
E0422 10:59:22.603286 18653 EchoServer.cpp:117]
========================================> current rate : 2488
>>> until almost 60 no http request are received to back ends >> this time
gap varies with every run ...
>>> after 60 secs rate limits are applied properly >>>>
E0422 11:00:07.690192 18653 EchoServer.cpp:117]
========================================> current rate : 1
E0422 11:00:10.411736 18653 EchoServer.cpp:117]
========================================> current rate : 1
E0422 11:00:11.412317 18653 EchoServer.cpp:117]
========================================> current rate : 1679
E0422 11:00:12.412369 18653 EchoServer.cpp:117]
========================================> current rate : 1667
E0422 11:00:13.451706 18653 EchoServer.cpp:117]
========================================> current rate : 1668
E0422 11:00:14.453778 18653 EchoServer.cpp:117]
========================================> current rate : 1668
E0422 11:00:15.457597 18653 EchoServer.cpp:117]
========================================> current rate : 1645
E0422 11:00:16.458938 18653 EchoServer.cpp:117]
========================================> current rate : 1762
E0422 11:00:17.470010 18653 EchoServer.cpp:117]
========================================> current rate : 1598
Can I get some info on the issue, is this know issue or am I missing some
config for rate limiting to be applied properly ?
Thanks in advance,
Badari
On Sat, Feb 23, 2019 at 8:48 PM Igor Cicimov <[email protected]>
wrote:
> On Sat, 23 Feb 2019 3:09 pm Santos Das <[email protected]> wrote:
>
>> Hi,
>>
>> I have a requirement where I need to allow only certain request rate for
>> a given URL.
>>
>> Say /login can be accessed at the rate of 10 RPS. If I get 100 RPS, then
>> 10 should be allowed and 90 should be denied.
>>
>> Any help on how this can be achieved ?
>>
>> *I tried to use the sticky table, but once it blocks it blocks for ever.
>> Please advise.*
>>
>>
>> frontend api_gateway
>> bind 0.0.0.0:80 <http://0.0.0.0/>
>> mode http
>> option forwardfor
>>
>> default_backend nodes
>>
>> # Set up stick table to track request rates
>> stick-table type binary len 8 size 1m expire 10s store
>> http_req_rate(10s)
>>
>> # Track client by base32+src (Host header + URL path + src IP)
>> http-request track-sc0 base32+src
>>
>> # Check map file to get rate limit for path
>> http-request set-var(req.rate_limit)
>> path,map_beg(/etc/hapee-1.8/maps/rates.map)
>>
>> # Client's request rate is tracked
>> http-request set-var(req.request_rate)
>> base32+src,table_http_req_rate(api_gateway)
>>
>> # Subtract the current request rate from the limit
>> # If less than zero, set rate_abuse to true
>> acl rate_abuse var(req.rate_limit),sub(req.request_rate) lt 0
>>
>
> Shouldn't this be:
> acl rate_abuse var(req.rate_limit),sub(var(req.request_rate)) lt 0
>
>
>> # Deny if rate abuse
>> http-request deny deny_status 429 if rate_abuse
>>
>> backend nodes
>> mode http
>> balance roundrobin
>> server echoprgm 10.37.9.30:11001 check
>>
>>
>>
>>
