Re: Rate Limit a specific HTML request
Hi,
On Tue, 2022-11-22 at 20:57 +, Branitsky, Norman wrote:
> I have the following "generic" rate limit defined - 150 requests in 10s
> from the same IP address:
> stick-table type ip size 100k expire 30s store http_req_rate(10s)
> http-request track-sc0 src unless { src -f
> /etc/CONFIG/haproxy/cidr.lst }
> http-request deny deny_status 429 if { sc_http_req_rate(0) gt 150 }
>
> Is it possible to rate limit a specific "computationally expensive" HTML
> request from the same IP address to a much smaller number?
Untested, but try using sc1 for the search url:
http-request track-sc1 src table search_table if
acl_matching_datamart_searchbyname !acl_exclude_cidr_lst
http-request deny deny_status 429 if { sc1_http_req_cnt(search_table) gt 5 }
backend search_table
stick-table type ... store http_req_cnt,http_req_rate...
-Jarno
--
Jarno Huuskonen
RE: Rate Limit a specific HTML request
The Public search call can take between 5s and 30s to respond depending on the
specificity of the request.
When I see 50 requests in 1 minute from the same IP address, for example, I
know this is someone abusing the system - it is clearly not a human being
interacting normally with the service.
In this case I want to limit 5 requests in 1 minute from the same IP address to
the following URL:
https:///datamart/searchByName.do
Norman Branitsky
Senior Cloud Architect
P: 416-916-1752
-Original Message-
From: Aleksandar Lazic
Sent: Tuesday, November 22, 2022 7:44 PM
To: Branitsky, Norman
Cc: HAProxy
Subject: Re: Rate Limit a specific HTML request
Hi.
On 22.11.22 23:19, Branitsky, Norman wrote:
> A "computationally expensive" request is a request sent to our Public
> Search service - no login required so it seems to be the target of abuse.
> For example:
> https:///datamart/searchByName.do?anchor=169a72e.0
Okay, let me rephrase your question.
How can be a IP blocked which creates a request which takes $too_much_time to
response.
Where could be the $too_much_time defined?
Could it be the "timeout server ..." config parameter?
Could the "%Tr" or "%TR" be used from logformat for that?
https://urldefense.com/v3/__https://docs.haproxy.org/2.6/configuration.html*8.2.6__;Iw!!A69Ausm6DtA!b3LTpCREJRN2Hj4NoBfjNEPkQJOByCMuzvxqRf8ovSBUZcwltPnqTF65sAGIyVpGWJLD55QNtJo9nh_zUwuWiz39dFlj$
or the request get a 504 for internal state.
Idea:
backend block_bad_client
stick-table type ip size 100k expire 30s store http_req_rate(10s)
http-request track-sc0 src unless { $too_much_time }
and call the table block_bad_client in the frontend config.
Is this what you would like to do?
I'm not sure if this is possible with HAProxy.
Regards
Alex
> Norman Branitsky
> Senior Cloud Architect
> P: 416-916-1752
>
> -Original Message-
> From: Aleksandar Lazic
> Sent: Tuesday, November 22, 2022 4:27 PM
> To: Branitsky, Norman
> Cc: HAProxy
> Subject: Re: Rate Limit a specific HTML request
>
> Hi.
>
> On 22.11.22 21:57, Branitsky, Norman wrote:
>> I have the following "generic" rate limit defined - 150 requests in
>> 10s from the same IP address:
>>
>> stick-table type ip size 100k expire 30s store
>> http_req_rate(10s)
>> http-request track-sc0 src unless { src -f
>> /etc/CONFIG/haproxy/cidr.lst }
>> http-request deny deny_status 429 if { sc_http_req_rate(0) gt
>> 150 }
>>
>> Is it possible to rate limit a specific "computationally expensive"
>> HTML request from the same IP address to a much smaller number?
>
> What do you define as a "computationally expensive" request?
>
> Maybe you could draw a bigger Picture and tell us what version of HAProxy do
> you use.
>
> In the upcoming 2.7 is also a "Bandwidth limitation", maybe this could help
> to solve your issue.
> https://urldefense.com/v3/__https://docs.haproxy.org/dev/configuration
> .html*9.7__;Iw!!A69Ausm6DtA!cXofLVgdVtpc37THsFRU0XMLkddQpViT0iPILErgEs
> XJ5Ij0hkHgjayqKAMX3sQrCOK74wbouLMjDkb0ZJe5a08n2NK9$
>
> HTML is a Description Language therefore I think you want to restrict HTTP
> Request/Response, isn't it?
>
> https://urldefense.com/v3/__https://www.rfc-editor.org/rfc/rfc1866__;!
> !A69Ausm6DtA!cXofLVgdVtpc37THsFRU0XMLkddQpViT0iPILErgEsXJ5Ij0hkHgjayqK
> AMX3sQrCOK74wbouLMjDkb0ZJe5a55k2_bp$
>
>> *Norman Branitsky*
>> Senior Cloud Architect
>> Tyler Technologies, Inc.
>
> Regards
> Alex
>
>> P: 416-916-1752
>> C: 416.843.0670
>> http://www.tylertech.com
>> Tyler Technologies
>
Re: Rate Limit a specific HTML request
Hi.
On 22.11.22 23:19, Branitsky, Norman wrote:
A "computationally expensive" request is a request sent to our Public Search
service - no login required so it seems to be the target of abuse.
For example:
https:///datamart/searchByName.do?anchor=169a72e.0
Okay, let me rephrase your question.
How can be a IP blocked which creates a request which takes
$too_much_time to response.
Where could be the $too_much_time defined?
Could it be the "timeout server ..." config parameter?
Could the "%Tr" or "%TR" be used from logformat for that?
https://docs.haproxy.org/2.6/configuration.html#8.2.6
or the request get a 504 for internal state.
Idea:
backend block_bad_client
stick-table type ip size 100k expire 30s store http_req_rate(10s)
http-request track-sc0 src unless { $too_much_time }
and call the table block_bad_client in the frontend config.
Is this what you would like to do?
I'm not sure if this is possible with HAProxy.
Regards
Alex
Norman Branitsky
Senior Cloud Architect
P: 416-916-1752
-Original Message-
From: Aleksandar Lazic
Sent: Tuesday, November 22, 2022 4:27 PM
To: Branitsky, Norman
Cc: HAProxy
Subject: Re: Rate Limit a specific HTML request
Hi.
On 22.11.22 21:57, Branitsky, Norman wrote:
I have the following "generic" rate limit defined - 150 requests in
10s from the same IP address:
stick-table type ip size 100k expire 30s store
http_req_rate(10s)
http-request track-sc0 src unless { src -f
/etc/CONFIG/haproxy/cidr.lst }
http-request deny deny_status 429 if { sc_http_req_rate(0) gt 150
}
Is it possible to rate limit a specific "computationally expensive"
HTML request from the same IP address to a much smaller number?
What do you define as a "computationally expensive" request?
Maybe you could draw a bigger Picture and tell us what version of HAProxy do
you use.
In the upcoming 2.7 is also a "Bandwidth limitation", maybe this could help to
solve your issue.
https://urldefense.com/v3/__https://docs.haproxy.org/dev/configuration.html*9.7__;Iw!!A69Ausm6DtA!cXofLVgdVtpc37THsFRU0XMLkddQpViT0iPILErgEsXJ5Ij0hkHgjayqKAMX3sQrCOK74wbouLMjDkb0ZJe5a08n2NK9$
HTML is a Description Language therefore I think you want to restrict HTTP
Request/Response, isn't it?
https://urldefense.com/v3/__https://www.rfc-editor.org/rfc/rfc1866__;!!A69Ausm6DtA!cXofLVgdVtpc37THsFRU0XMLkddQpViT0iPILErgEsXJ5Ij0hkHgjayqKAMX3sQrCOK74wbouLMjDkb0ZJe5a55k2_bp$
*Norman Branitsky*
Senior Cloud Architect
Tyler Technologies, Inc.
Regards
Alex
P: 416-916-1752
C: 416.843.0670
http://www.tylertech.com
Tyler Technologies
RE: Rate Limit a specific HTML request
A "computationally expensive" request is a request sent to our Public Search
service -
no login required so it seems to be the target of abuse.
For example:
https:///datamart/searchByName.do?anchor=169a72e.0
Norman Branitsky
Senior Cloud Architect
P: 416-916-1752
-Original Message-
From: Aleksandar Lazic
Sent: Tuesday, November 22, 2022 4:27 PM
To: Branitsky, Norman
Cc: HAProxy
Subject: Re: Rate Limit a specific HTML request
Hi.
On 22.11.22 21:57, Branitsky, Norman wrote:
> I have the following "generic" rate limit defined - 150 requests in
> 10s from the same IP address:
>
> stick-table type ip size 100k expire 30s store
> http_req_rate(10s)
> http-request track-sc0 src unless { src -f
> /etc/CONFIG/haproxy/cidr.lst }
> http-request deny deny_status 429 if { sc_http_req_rate(0) gt 150
> }
>
> Is it possible to rate limit a specific "computationally expensive"
> HTML request from the same IP address to a much smaller number?
What do you define as a "computationally expensive" request?
Maybe you could draw a bigger Picture and tell us what version of HAProxy do
you use.
In the upcoming 2.7 is also a "Bandwidth limitation", maybe this could help to
solve your issue.
https://urldefense.com/v3/__https://docs.haproxy.org/dev/configuration.html*9.7__;Iw!!A69Ausm6DtA!cXofLVgdVtpc37THsFRU0XMLkddQpViT0iPILErgEsXJ5Ij0hkHgjayqKAMX3sQrCOK74wbouLMjDkb0ZJe5a08n2NK9$
HTML is a Description Language therefore I think you want to restrict HTTP
Request/Response, isn't it?
https://urldefense.com/v3/__https://www.rfc-editor.org/rfc/rfc1866__;!!A69Ausm6DtA!cXofLVgdVtpc37THsFRU0XMLkddQpViT0iPILErgEsXJ5Ij0hkHgjayqKAMX3sQrCOK74wbouLMjDkb0ZJe5a55k2_bp$
> *Norman Branitsky*
> Senior Cloud Architect
> Tyler Technologies, Inc.
Regards
Alex
> P: 416-916-1752
> C: 416.843.0670
> http://www.tylertech.com
> Tyler Technologies
Re: Rate Limit a specific HTML request
Hi.
On 22.11.22 21:57, Branitsky, Norman wrote:
I have the following "generic" rate limit defined - 150 requests in 10s
from the same IP address:
stick-table type ip size 100k expire 30s store http_req_rate(10s)
http-request track-sc0 src unless { src -f
/etc/CONFIG/haproxy/cidr.lst }
http-request deny deny_status 429 if { sc_http_req_rate(0) gt 150 }
Is it possible to rate limit a specific "computationally expensive" HTML
request from the same IP address to a much smaller number?
What do you define as a "computationally expensive" request?
Maybe you could draw a bigger Picture and tell us what version of
HAProxy do you use.
In the upcoming 2.7 is also a "Bandwidth limitation", maybe this could
help to solve your issue.
https://docs.haproxy.org/dev/configuration.html#9.7
HTML is a Description Language therefore I think you want to restrict
HTTP Request/Response, isn't it?
https://www.rfc-editor.org/rfc/rfc1866
*Norman Branitsky*
Senior Cloud Architect
Tyler Technologies, Inc.
Regards
Alex
P: 416-916-1752
C: 416.843.0670
www.tylertech.com
Tyler Technologies
Rate Limit a specific HTML request
I have the following "generic" rate limit defined - 150 requests in 10s from
the same IP address:
stick-table type ip size 100k expire 30s store http_req_rate(10s)
http-request track-sc0 src unless { src -f /etc/CONFIG/haproxy/cidr.lst }
http-request deny deny_status 429 if { sc_http_req_rate(0) gt 150 }
Is it possible to rate limit a specific "computationally expensive" HTML
request from the same IP address to a much smaller number?
Norman Branitsky
Senior Cloud Architect
Tyler Technologies, Inc.
P: 416-916-1752
C: 416.843.0670
www.tylertech.com
[Tyler Technologies]
