Re: Updating a stick table from the HTTP response
Hi Holger,
"tcp-response content track- / http-response track-" would be a nice
feature, don't know if this is on the roadmap.
For the moment i can only imagine the following (needs HAProxy 1.6):
http-response lua script.lua
Within this Lua function, you check the http response code and
update/set a stick-table entry per haproxy socket ("set table ...").
(I don't know if there is a native lua function for this case (without
using the socket))
I didn't say it's an elegant solution ... :)
---
Best Regards / Mit freundlichen Grüßen
Bjoern
2015-06-30 12:37 GMT+02:00 Holger Just :
> Hello all,
>
> Unfortunately, I have not received any feedback on my earlier email so
> I'll try again.
>
> I am still struggling trying to implement a throttling mechanism based
> on prior HTTP responses of the same client.
>
> Basically, I have some requests (using Basic Auth or other mechanisms)
> that might result in HTTP 401 responses from the backend server. I want
> to throttle further requests by the same client if the number of such
> responses reaches a certain threshold (responses / second). In that
> case, I want to first delay and finally block new requests completely
> until the error rate goes down.
>
> In order to implement this, it would be great if we would have the
> possibility to update stick entries based on the response and not only
> the request, e.g.
>
> tcp-response content track-sc2 src if { status 401 }
>
> Is something like this feasible, maybe under the restriction that the
> user has to make sure on their own that the data required to update the
> stick table entry is still available?
>
> Thank you for your feedback.
>
> --Holger
>
>
> Holger Just wrote:
>> Hello all,
>>
>> with HAProxy 1.5.11, we have implemented rate limiting based on some
>> aspects of the request (Host header, path, ...). In our implementation,
>> we delay limited requests by forcing a WAIT_END in order to prevent
>> brute-force attacks against e.g. passwords or login tokens:
>>
>>
>> acl bruteforce_slowdown sc2_http_req_rate gt 20
>> acl limited_path path_beg /sensitive/stuff
>>
>> stick-table type ip size 100k expire 30m store http_req_rate(300s)
>> tcp-request content track-sc2 src if METH_POST limited_path
>>
>> # Delay the request for 10 seconds if we have too many requests
>> tcp-request inspect-delay 10s
>> tcp-request content accept unless bruteforce_slowdown limited_path
>> tcp-request content accept if WAIT_END
>>
>>
>> As you can see above, we track only certain requests to sensitive
>> resources and delay further requests after 20 req / 300 s without taking
>> the actual response into account. This is good enough for e.g. a web
>> form to login or change a password.
>>
>> Now, unfortunately we have some endpoints which are protected with Basic
>> Auth which is validated by the application. If the password is
>> incorrect, we return an HTTP 401.
>>
>> In order to prevent brute-forcing of passwords against these endpoints,
>> we would like to employ a similar delay mechanism. Unfortunately, we
>> can't detect from the request headers alone if we have a bad request but
>> have to inspect the response and increase the sc2 counter only of we
>> have seen a 401.
>>
>> In the end, I would like to use a fetch similar to sc1_http_err_rate but
>> reduced to only specific cases, i.e. 401 responses on certain paths or
>> Host names.
>>
>> Now the problem is that we apparently can't manipulate the stick table
>> from a HTTP response, or more precisely: I have not found a way to do it.
>>
>> We would like to do something like
>>
>>
>> tcp-response content track-sc2 src if { status 401 }
>>
>>
>> which would allow us to track these error-responses similar to the first
>> approach and handle the next requests the same way as above.
>>
>> Now my questions are:
>>
>> * Is something like this possible/feasible right now?
>> * Is there some other way to implement rate limiting based on certain
>> server responses?
>> * If this is not possible right now, would it be feasible to implement
>> the possibility to track responses similar to what is possible with
>> requests right now?
>>
>> Thank you for your feedback,
>> Holger Just
>>
>
Re: Updating a stick table from the HTTP response
Hello all,
Unfortunately, I have not received any feedback on my earlier email so
I'll try again.
I am still struggling trying to implement a throttling mechanism based
on prior HTTP responses of the same client.
Basically, I have some requests (using Basic Auth or other mechanisms)
that might result in HTTP 401 responses from the backend server. I want
to throttle further requests by the same client if the number of such
responses reaches a certain threshold (responses / second). In that
case, I want to first delay and finally block new requests completely
until the error rate goes down.
In order to implement this, it would be great if we would have the
possibility to update stick entries based on the response and not only
the request, e.g.
tcp-response content track-sc2 src if { status 401 }
Is something like this feasible, maybe under the restriction that the
user has to make sure on their own that the data required to update the
stick table entry is still available?
Thank you for your feedback.
--Holger
Holger Just wrote:
> Hello all,
>
> with HAProxy 1.5.11, we have implemented rate limiting based on some
> aspects of the request (Host header, path, ...). In our implementation,
> we delay limited requests by forcing a WAIT_END in order to prevent
> brute-force attacks against e.g. passwords or login tokens:
>
>
> acl bruteforce_slowdown sc2_http_req_rate gt 20
> acl limited_path path_beg /sensitive/stuff
>
> stick-table type ip size 100k expire 30m store http_req_rate(300s)
> tcp-request content track-sc2 src if METH_POST limited_path
>
> # Delay the request for 10 seconds if we have too many requests
> tcp-request inspect-delay 10s
> tcp-request content accept unless bruteforce_slowdown limited_path
> tcp-request content accept if WAIT_END
>
>
> As you can see above, we track only certain requests to sensitive
> resources and delay further requests after 20 req / 300 s without taking
> the actual response into account. This is good enough for e.g. a web
> form to login or change a password.
>
> Now, unfortunately we have some endpoints which are protected with Basic
> Auth which is validated by the application. If the password is
> incorrect, we return an HTTP 401.
>
> In order to prevent brute-forcing of passwords against these endpoints,
> we would like to employ a similar delay mechanism. Unfortunately, we
> can't detect from the request headers alone if we have a bad request but
> have to inspect the response and increase the sc2 counter only of we
> have seen a 401.
>
> In the end, I would like to use a fetch similar to sc1_http_err_rate but
> reduced to only specific cases, i.e. 401 responses on certain paths or
> Host names.
>
> Now the problem is that we apparently can't manipulate the stick table
> from a HTTP response, or more precisely: I have not found a way to do it.
>
> We would like to do something like
>
>
> tcp-response content track-sc2 src if { status 401 }
>
>
> which would allow us to track these error-responses similar to the first
> approach and handle the next requests the same way as above.
>
> Now my questions are:
>
> * Is something like this possible/feasible right now?
> * Is there some other way to implement rate limiting based on certain
> server responses?
> * If this is not possible right now, would it be feasible to implement
> the possibility to track responses similar to what is possible with
> requests right now?
>
> Thank you for your feedback,
> Holger Just
>
Updating a stick table from the HTTP response
Hello all,
with HAProxy 1.5.11, we have implemented rate limiting based on some
aspects of the request (Host header, path, ...). In our implementation,
we delay limited requests by forcing a WAIT_END in order to prevent
brute-force attacks against e.g. passwords or login tokens:
acl bruteforce_slowdown sc2_http_req_rate gt 20
acl limited_path path_beg /sensitive/stuff
stick-table type ip size 100k expire 30m store http_req_rate(300s)
tcp-request content track-sc2 src if METH_POST limited_path
# Delay the request for 10 seconds if we have too many requests
tcp-request inspect-delay 10s
tcp-request content accept unless bruteforce_slowdown limited_path
tcp-request content accept if WAIT_END
As you can see above, we track only certain requests to sensitive
resources and delay further requests after 20 req / 300 s without taking
the actual response into account. This is good enough for e.g. a web
form to login or change a password.
Now, unfortunately we have some endpoints which are protected with Basic
Auth which is validated by the application. If the password is
incorrect, we return an HTTP 401.
In order to prevent brute-forcing of passwords against these endpoints,
we would like to employ a similar delay mechanism. Unfortunately, we
can't detect from the request headers alone if we have a bad request but
have to inspect the response and increase the sc2 counter only of we
have seen a 401.
In the end, I would like to use a fetch similar to sc1_http_err_rate but
reduced to only specific cases, i.e. 401 responses on certain paths or
Host names.
Now the problem is that we apparently can't manipulate the stick table
from a HTTP response, or more precisely: I have not found a way to do it.
We would like to do something like
tcp-request content track-sc2 src if { status 401 }
which would allow us to track these error-responses similar to the first
approach and handle the next requests the same way as above.
Now my questions are:
* Is something like this possible/feasible right now?
* Is there some other way to implement rate limiting based on certain
server responses?
* If this is not possible right now, would it be feasible to implement
the possibility to track responses similar to what is possible with
requests right now?
Thank you for your feedback,
Holger Just
