Re: urls in stick-table, any timeline?

[Baptiste]

of course not.
It is in the dev18 release, which seems to be quite stable!

Baptiste

On Sat, Apr 27, 2013 at 2:58 AM, S Ahmed  wrote:
> Is this in the latest stable release?
>
>
> On Thu, Apr 25, 2013 at 11:38 AM, Baptiste  wrote:
>>
>> Hi,
>>
>> So basically, you want to rate limit on the URL including the query
>> string.
>> something like:
>>
>> frontend webservice
>> [...]
>>  acl url_to_protect path /something/object /something_else/whatever
>>  stick-table type string len 128 size 1m expire 10m store gpc0
>>  tcp-request content track-sc1 url if url_to_protect
>>  tcp-request content reject if { sc1_get_gpc0 gt 0 }
>> [...]
>>
>> backend webservice
>> [...]
>>  acl url_to_protect path /something/object /something_else/whatever
>>  stick-table type string len 128 size 1m expire 10m store
>> http_req_rate(1m)
>>  tcp-request content track-sc2 url if url_to_protect
>>  acl abuser sc2_http_req_rate gt 100
>>  acl mark_as_abuser sc1_inc_gpc0 gt 0
>>  tcp-request content reject if abuser mark_as_abuser
>> [...]
>>
>> Basically, you're going to track only the URLs which matches the
>> url_to_protect acl.
>> On the frontend, you just store the URLs + a counter (gpc0) which will
>> be used to track the banned URLs.
>> In the backend, you store the URLs + the HTTP request rate over 1
>> minute associated to each of them. A couple of ACLs monitor the
>> request rate and increments gpc0 from frontend table if the req rate
>> is over a limit (100 in this case).
>>
>> Note that it can be easy to forge a URL, so you could drop all the
>> URLs which does not look like regular.
>> I mean /other?id=123 and /other?id=123&foo=bar will be 2 URLs for the same
>> user.
>> Otherwise, instead of tracking on 'url' you can track a single url
>> parameter like using 'urlp(id)' .
>>
>> I have not tested the configuration above, I made it out of my head,
>> any issues, please let me know.
>>
>> And please let me know if it works in your case.
>>
>> Baptiste
>>
>>
>>
>> On Thu, Apr 25, 2013 at 4:49 PM, S Ahmed  wrote:
>> > Each client (might be upto 100K of them) will have a unique URL, let me
>> > clarify the url:
>> >
>> > client#123
>> > api.example.com/some_thing/other?clientId=123
>> >
>> > client#123
>> > api.example.com/some_thing/other?clientId=124
>> >
>> > etc.
>> >
>> > So each client has a unique URL, but the source IP of the client might
>> > be
>> > different as they will be connecting to this service from multiple
>> > servers.
>> >
>> > I want to rate limit each client individually, giving them e.g. 100
>> > requests
>> > per minute, if they go over, just drop the connection as fast as
>> > possible.
>> >
>> > Also, is it possible to rate limit each client with a different rate
>> > limit?
>> > e.g. some may be 50 requests per minute, while others maybe be 100.
>> >
>> >
>> > On Thu, Apr 25, 2013 at 1:24 AM, Baptiste  wrote:
>> >>
>> >> Hi,
>> >>
>> >> Last question: Will you have one URL per client? I mean will the query
>> >> string change with each client?
>> >> Then do you want to rate limit each client individually or do you want
>> >> to rate limit the number of call to the script named "other" as a
>> >> whole in your example?
>> >>
>> >> Baptiste
>> >>
>> >> On Wed, Apr 24, 2013 at 7:49 PM, S Ahmed  wrote:
>> >> > Nice!
>> >> >
>> >> > Is this in the latest 1.4 release or a dev release?
>> >> >
>> >> > I need to rate limit on a URL (that includes query string values)
>> >> > like:
>> >> >
>> >> > api.example.com/some_thing/other?id=asdf234asdf234&id2=asdf234234
>> >> >
>> >> > Multiple sources are possible, but I don't care of the source IP I
>> >> > just
>> >> > want
>> >> > to rate limit on the URL (customers will be using this endpoint from
>> >> > their
>> >> > servers, but I want to rate limit to x number of requests per minute,
>> >> > if
>> >> > they go over, just drop the request until the rate limit has
>> >> > expired).
>> >> >
>> >> >
>> >> > On Tue, Apr 23, 2013 at 1:39 AM, Baptiste  wrote:
>> >> >>
>> >> >> Hi Ahmed,
>> >> >>
>> >> >> Yes, it has been implemented.
>> >> >> You can store a URL and rate limited on it.
>> >> >>
>> >> >> Baptiste
>> >> >>
>> >> >> On Mon, Apr 22, 2013 at 11:15 PM, S Ahmed 
>> >> >> wrote:
>> >> >> > Hello,
>> >> >> >
>> >> >> > Has this feature been released yet by any change? :)
>> >> >> >
>> >> >> > Again my initial request was to do:
>> >> >> >
>> >> >> > I was told that soon you will be able to store a URL in a
>> >> >> > stick-table,
>> >> >> > so I
>> >> >> > could block a particular url and then remove the block by making a
>> >> >> > request.
>> >> >> >
>> >> >> > I situation is I will be blocking up to 250K urls (for rate
>> >> >> > limiting,
>> >> >> > hard
>> >> >> > drop any connection.for a given period of time, then I need to
>> >> >> > un-block
>> >> >> > them).
>> >> >> >
>> >> >> > Any rough timelines on when this might be released?
>> >> >> >
>> >> >> >>Soon, you'll be able to store an URL in a stick-table, so you'll
>> >> >> >> be
>> >> >> >>abl

Re: urls in stick-table, any timeline?

[S Ahmed]

Is this in the latest stable release?


On Thu, Apr 25, 2013 at 11:38 AM, Baptiste  wrote:

> Hi,
>
> So basically, you want to rate limit on the URL including the query string.
> something like:
>
> frontend webservice
> [...]
>  acl url_to_protect path /something/object /something_else/whatever
>  stick-table type string len 128 size 1m expire 10m store gpc0
>  tcp-request content track-sc1 url if url_to_protect
>  tcp-request content reject if { sc1_get_gpc0 gt 0 }
> [...]
>
> backend webservice
> [...]
>  acl url_to_protect path /something/object /something_else/whatever
>  stick-table type string len 128 size 1m expire 10m store http_req_rate(1m)
>  tcp-request content track-sc2 url if url_to_protect
>  acl abuser sc2_http_req_rate gt 100
>  acl mark_as_abuser sc1_inc_gpc0 gt 0
>  tcp-request content reject if abuser mark_as_abuser
> [...]
>
> Basically, you're going to track only the URLs which matches the
> url_to_protect acl.
> On the frontend, you just store the URLs + a counter (gpc0) which will
> be used to track the banned URLs.
> In the backend, you store the URLs + the HTTP request rate over 1
> minute associated to each of them. A couple of ACLs monitor the
> request rate and increments gpc0 from frontend table if the req rate
> is over a limit (100 in this case).
>
> Note that it can be easy to forge a URL, so you could drop all the
> URLs which does not look like regular.
> I mean /other?id=123 and /other?id=123&foo=bar will be 2 URLs for the same
> user.
> Otherwise, instead of tracking on 'url' you can track a single url
> parameter like using 'urlp(id)' .
>
> I have not tested the configuration above, I made it out of my head,
> any issues, please let me know.
>
> And please let me know if it works in your case.
>
> Baptiste
>
>
>
> On Thu, Apr 25, 2013 at 4:49 PM, S Ahmed  wrote:
> > Each client (might be upto 100K of them) will have a unique URL, let me
> > clarify the url:
> >
> > client#123
> > api.example.com/some_thing/other?clientId=123
> >
> > client#123
> > api.example.com/some_thing/other?clientId=124
> >
> > etc.
> >
> > So each client has a unique URL, but the source IP of the client might be
> > different as they will be connecting to this service from multiple
> servers.
> >
> > I want to rate limit each client individually, giving them e.g. 100
> requests
> > per minute, if they go over, just drop the connection as fast as
> possible.
> >
> > Also, is it possible to rate limit each client with a different rate
> limit?
> > e.g. some may be 50 requests per minute, while others maybe be 100.
> >
> >
> > On Thu, Apr 25, 2013 at 1:24 AM, Baptiste  wrote:
> >>
> >> Hi,
> >>
> >> Last question: Will you have one URL per client? I mean will the query
> >> string change with each client?
> >> Then do you want to rate limit each client individually or do you want
> >> to rate limit the number of call to the script named "other" as a
> >> whole in your example?
> >>
> >> Baptiste
> >>
> >> On Wed, Apr 24, 2013 at 7:49 PM, S Ahmed  wrote:
> >> > Nice!
> >> >
> >> > Is this in the latest 1.4 release or a dev release?
> >> >
> >> > I need to rate limit on a URL (that includes query string values)
> like:
> >> >
> >> > api.example.com/some_thing/other?id=asdf234asdf234&id2=asdf234234
> >> >
> >> > Multiple sources are possible, but I don't care of the source IP I
> just
> >> > want
> >> > to rate limit on the URL (customers will be using this endpoint from
> >> > their
> >> > servers, but I want to rate limit to x number of requests per minute,
> if
> >> > they go over, just drop the request until the rate limit has expired).
> >> >
> >> >
> >> > On Tue, Apr 23, 2013 at 1:39 AM, Baptiste  wrote:
> >> >>
> >> >> Hi Ahmed,
> >> >>
> >> >> Yes, it has been implemented.
> >> >> You can store a URL and rate limited on it.
> >> >>
> >> >> Baptiste
> >> >>
> >> >> On Mon, Apr 22, 2013 at 11:15 PM, S Ahmed 
> wrote:
> >> >> > Hello,
> >> >> >
> >> >> > Has this feature been released yet by any change? :)
> >> >> >
> >> >> > Again my initial request was to do:
> >> >> >
> >> >> > I was told that soon you will be able to store a URL in a
> >> >> > stick-table,
> >> >> > so I
> >> >> > could block a particular url and then remove the block by making a
> >> >> > request.
> >> >> >
> >> >> > I situation is I will be blocking up to 250K urls (for rate
> limiting,
> >> >> > hard
> >> >> > drop any connection.for a given period of time, then I need to
> >> >> > un-block
> >> >> > them).
> >> >> >
> >> >> > Any rough timelines on when this might be released?
> >> >> >
> >> >> >>Soon, you'll be able to store an URL in a stick-table, so you'll be
> >> >> >>able to update a gpc counter by setting up a particular header on
> the
> >> >> >>server side which tells HAProxy to block this request.
> >> >> >>For the cancellation of this blocking system, you could request the
> >> >> >>URL with a particular header to unblock it.
> >> >> >
> >> >> >
> >> >> > On Tue, Aug 21, 2012 at 2:04 PM, Baptiste 
> wrote:
> >

Re: urls in stick-table, any timeline?

[Baptiste]

Hi,

So basically, you want to rate limit on the URL including the query string.
something like:

frontend webservice
[...]
 acl url_to_protect path /something/object /something_else/whatever
 stick-table type string len 128 size 1m expire 10m store gpc0
 tcp-request content track-sc1 url if url_to_protect
 tcp-request content reject if { sc1_get_gpc0 gt 0 }
[...]

backend webservice
[...]
 acl url_to_protect path /something/object /something_else/whatever
 stick-table type string len 128 size 1m expire 10m store http_req_rate(1m)
 tcp-request content track-sc2 url if url_to_protect
 acl abuser sc2_http_req_rate gt 100
 acl mark_as_abuser sc1_inc_gpc0 gt 0
 tcp-request content reject if abuser mark_as_abuser
[...]

Basically, you're going to track only the URLs which matches the
url_to_protect acl.
On the frontend, you just store the URLs + a counter (gpc0) which will
be used to track the banned URLs.
In the backend, you store the URLs + the HTTP request rate over 1
minute associated to each of them. A couple of ACLs monitor the
request rate and increments gpc0 from frontend table if the req rate
is over a limit (100 in this case).

Note that it can be easy to forge a URL, so you could drop all the
URLs which does not look like regular.
I mean /other?id=123 and /other?id=123&foo=bar will be 2 URLs for the same user.
Otherwise, instead of tracking on 'url' you can track a single url
parameter like using 'urlp(id)' .

I have not tested the configuration above, I made it out of my head,
any issues, please let me know.

And please let me know if it works in your case.

Baptiste



On Thu, Apr 25, 2013 at 4:49 PM, S Ahmed  wrote:
> Each client (might be upto 100K of them) will have a unique URL, let me
> clarify the url:
>
> client#123
> api.example.com/some_thing/other?clientId=123
>
> client#123
> api.example.com/some_thing/other?clientId=124
>
> etc.
>
> So each client has a unique URL, but the source IP of the client might be
> different as they will be connecting to this service from multiple servers.
>
> I want to rate limit each client individually, giving them e.g. 100 requests
> per minute, if they go over, just drop the connection as fast as possible.
>
> Also, is it possible to rate limit each client with a different rate limit?
> e.g. some may be 50 requests per minute, while others maybe be 100.
>
>
> On Thu, Apr 25, 2013 at 1:24 AM, Baptiste  wrote:
>>
>> Hi,
>>
>> Last question: Will you have one URL per client? I mean will the query
>> string change with each client?
>> Then do you want to rate limit each client individually or do you want
>> to rate limit the number of call to the script named "other" as a
>> whole in your example?
>>
>> Baptiste
>>
>> On Wed, Apr 24, 2013 at 7:49 PM, S Ahmed  wrote:
>> > Nice!
>> >
>> > Is this in the latest 1.4 release or a dev release?
>> >
>> > I need to rate limit on a URL (that includes query string values) like:
>> >
>> > api.example.com/some_thing/other?id=asdf234asdf234&id2=asdf234234
>> >
>> > Multiple sources are possible, but I don't care of the source IP I just
>> > want
>> > to rate limit on the URL (customers will be using this endpoint from
>> > their
>> > servers, but I want to rate limit to x number of requests per minute, if
>> > they go over, just drop the request until the rate limit has expired).
>> >
>> >
>> > On Tue, Apr 23, 2013 at 1:39 AM, Baptiste  wrote:
>> >>
>> >> Hi Ahmed,
>> >>
>> >> Yes, it has been implemented.
>> >> You can store a URL and rate limited on it.
>> >>
>> >> Baptiste
>> >>
>> >> On Mon, Apr 22, 2013 at 11:15 PM, S Ahmed  wrote:
>> >> > Hello,
>> >> >
>> >> > Has this feature been released yet by any change? :)
>> >> >
>> >> > Again my initial request was to do:
>> >> >
>> >> > I was told that soon you will be able to store a URL in a
>> >> > stick-table,
>> >> > so I
>> >> > could block a particular url and then remove the block by making a
>> >> > request.
>> >> >
>> >> > I situation is I will be blocking up to 250K urls (for rate limiting,
>> >> > hard
>> >> > drop any connection.for a given period of time, then I need to
>> >> > un-block
>> >> > them).
>> >> >
>> >> > Any rough timelines on when this might be released?
>> >> >
>> >> >>Soon, you'll be able to store an URL in a stick-table, so you'll be
>> >> >>able to update a gpc counter by setting up a particular header on the
>> >> >>server side which tells HAProxy to block this request.
>> >> >>For the cancellation of this blocking system, you could request the
>> >> >>URL with a particular header to unblock it.
>> >> >
>> >> >
>> >> > On Tue, Aug 21, 2012 at 2:04 PM, Baptiste  wrote:
>> >> >>
>> >> >> Hey,
>> >> >>
>> >> >> Nothing coming right now.
>> >> >> Maybe for Christmas :)
>> >> >>
>> >> >> cheers
>> >> >>
>> >> >> On Tue, Aug 21, 2012 at 5:19 PM, S Ahmed 
>> >> >> wrote:
>> >> >> > Hello,
>> >> >> >
>> >> >> > Any updates or guestimates on if sticky-table feature will be
>> >> >> > released?
>> >> >> >
>> >> >> > Just haven't been watching this lis

Re: urls in stick-table, any timeline?

[S Ahmed]

Each client (might be upto 100K of them) will have a unique URL, let me
clarify the url:

client#123
api.example.com/some_thing/other?
clientId=123

client#123
api.example.com/some_thing/other?
clientId=124

etc.

So each client has a unique URL, but the source IP of the client might be
different as they will be connecting to this service from multiple servers.

I want to rate limit each client individually, giving them e.g. 100
requests per minute, if they go over, just drop the connection as fast as
possible.

Also, is it possible to rate limit each client with a different rate limit?
e.g. some may be 50 requests per minute, while others maybe be 100.


On Thu, Apr 25, 2013 at 1:24 AM, Baptiste  wrote:

> Hi,
>
> Last question: Will you have one URL per client? I mean will the query
> string change with each client?
> Then do you want to rate limit each client individually or do you want
> to rate limit the number of call to the script named "other" as a
> whole in your example?
>
> Baptiste
>
> On Wed, Apr 24, 2013 at 7:49 PM, S Ahmed  wrote:
> > Nice!
> >
> > Is this in the latest 1.4 release or a dev release?
> >
> > I need to rate limit on a URL (that includes query string values) like:
> >
> > api.example.com/some_thing/other?id=asdf234asdf234&id2=asdf234234
> >
> > Multiple sources are possible, but I don't care of the source IP I just
> want
> > to rate limit on the URL (customers will be using this endpoint from
> their
> > servers, but I want to rate limit to x number of requests per minute, if
> > they go over, just drop the request until the rate limit has expired).
> >
> >
> > On Tue, Apr 23, 2013 at 1:39 AM, Baptiste  wrote:
> >>
> >> Hi Ahmed,
> >>
> >> Yes, it has been implemented.
> >> You can store a URL and rate limited on it.
> >>
> >> Baptiste
> >>
> >> On Mon, Apr 22, 2013 at 11:15 PM, S Ahmed  wrote:
> >> > Hello,
> >> >
> >> > Has this feature been released yet by any change? :)
> >> >
> >> > Again my initial request was to do:
> >> >
> >> > I was told that soon you will be able to store a URL in a stick-table,
> >> > so I
> >> > could block a particular url and then remove the block by making a
> >> > request.
> >> >
> >> > I situation is I will be blocking up to 250K urls (for rate limiting,
> >> > hard
> >> > drop any connection.for a given period of time, then I need to
> un-block
> >> > them).
> >> >
> >> > Any rough timelines on when this might be released?
> >> >
> >> >>Soon, you'll be able to store an URL in a stick-table, so you'll be
> >> >>able to update a gpc counter by setting up a particular header on the
> >> >>server side which tells HAProxy to block this request.
> >> >>For the cancellation of this blocking system, you could request the
> >> >>URL with a particular header to unblock it.
> >> >
> >> >
> >> > On Tue, Aug 21, 2012 at 2:04 PM, Baptiste  wrote:
> >> >>
> >> >> Hey,
> >> >>
> >> >> Nothing coming right now.
> >> >> Maybe for Christmas :)
> >> >>
> >> >> cheers
> >> >>
> >> >> On Tue, Aug 21, 2012 at 5:19 PM, S Ahmed 
> wrote:
> >> >> > Hello,
> >> >> >
> >> >> > Any updates or guestimates on if sticky-table feature will be
> >> >> > released?
> >> >> >
> >> >> > Just haven't been watching this list for a while and curious if
> there
> >> >> > has
> >> >> > been any progress.
> >> >> >
> >> >> > Appreciate it!
> >> >> >
> >> >> > On Sun, Jun 24, 2012 at 1:28 AM, Willy Tarreau  wrote:
> >> >> >>
> >> >> >> Hi,
> >> >> >>
> >> >> >> On Thu, Jun 21, 2012 at 05:16:22PM -0400, S Ahmed wrote:
> >> >> >> > I was told that soon you will be able to store a URL in a
> >> >> >> > stick-table,
> >> >> >> > so I
> >> >> >> > could block a particular url and then remove the block by
> making a
> >> >> >> > request.
> >> >> >> >
> >> >> >> > I situation is I will be blocking up to 250K urls (for rate
> >> >> >> > limiting,
> >> >> >> > hard
> >> >> >> > drop any connection.for a given period of time, then I need to
> >> >> >> > un-block
> >> >> >> > them).
> >> >> >> >
> >> >> >> > Any rough timelines on when this might be released?
> >> >> >>
> >> >> >> Unfortunately, no, there is no timeline. Several subjects are
> being
> >> >> >> addressed
> >> >> >> at the same time so it's a matter of priority. Right now we have
> to
> >> >> >> rework
> >> >> >> all
> >> >> >> the low-level connection management to ensure proper integration
> of
> >> >> >> SSL,
> >> >> >> so we
> >> >> >> will see the stick tables after that.
> >> >> >>
> >> >> >> Regards,
> >> >> >> Willy
> >> >> >>
> >> >> >
> >> >
> >> >
> >
> >
>

Re: urls in stick-table, any timeline?

[Baptiste]

Hi,

Last question: Will you have one URL per client? I mean will the query
string change with each client?
Then do you want to rate limit each client individually or do you want
to rate limit the number of call to the script named "other" as a
whole in your example?

Baptiste

On Wed, Apr 24, 2013 at 7:49 PM, S Ahmed  wrote:
> Nice!
>
> Is this in the latest 1.4 release or a dev release?
>
> I need to rate limit on a URL (that includes query string values) like:
>
> api.example.com/some_thing/other?id=asdf234asdf234&id2=asdf234234
>
> Multiple sources are possible, but I don't care of the source IP I just want
> to rate limit on the URL (customers will be using this endpoint from their
> servers, but I want to rate limit to x number of requests per minute, if
> they go over, just drop the request until the rate limit has expired).
>
>
> On Tue, Apr 23, 2013 at 1:39 AM, Baptiste  wrote:
>>
>> Hi Ahmed,
>>
>> Yes, it has been implemented.
>> You can store a URL and rate limited on it.
>>
>> Baptiste
>>
>> On Mon, Apr 22, 2013 at 11:15 PM, S Ahmed  wrote:
>> > Hello,
>> >
>> > Has this feature been released yet by any change? :)
>> >
>> > Again my initial request was to do:
>> >
>> > I was told that soon you will be able to store a URL in a stick-table,
>> > so I
>> > could block a particular url and then remove the block by making a
>> > request.
>> >
>> > I situation is I will be blocking up to 250K urls (for rate limiting,
>> > hard
>> > drop any connection.for a given period of time, then I need to un-block
>> > them).
>> >
>> > Any rough timelines on when this might be released?
>> >
>> >>Soon, you'll be able to store an URL in a stick-table, so you'll be
>> >>able to update a gpc counter by setting up a particular header on the
>> >>server side which tells HAProxy to block this request.
>> >>For the cancellation of this blocking system, you could request the
>> >>URL with a particular header to unblock it.
>> >
>> >
>> > On Tue, Aug 21, 2012 at 2:04 PM, Baptiste  wrote:
>> >>
>> >> Hey,
>> >>
>> >> Nothing coming right now.
>> >> Maybe for Christmas :)
>> >>
>> >> cheers
>> >>
>> >> On Tue, Aug 21, 2012 at 5:19 PM, S Ahmed  wrote:
>> >> > Hello,
>> >> >
>> >> > Any updates or guestimates on if sticky-table feature will be
>> >> > released?
>> >> >
>> >> > Just haven't been watching this list for a while and curious if there
>> >> > has
>> >> > been any progress.
>> >> >
>> >> > Appreciate it!
>> >> >
>> >> > On Sun, Jun 24, 2012 at 1:28 AM, Willy Tarreau  wrote:
>> >> >>
>> >> >> Hi,
>> >> >>
>> >> >> On Thu, Jun 21, 2012 at 05:16:22PM -0400, S Ahmed wrote:
>> >> >> > I was told that soon you will be able to store a URL in a
>> >> >> > stick-table,
>> >> >> > so I
>> >> >> > could block a particular url and then remove the block by making a
>> >> >> > request.
>> >> >> >
>> >> >> > I situation is I will be blocking up to 250K urls (for rate
>> >> >> > limiting,
>> >> >> > hard
>> >> >> > drop any connection.for a given period of time, then I need to
>> >> >> > un-block
>> >> >> > them).
>> >> >> >
>> >> >> > Any rough timelines on when this might be released?
>> >> >>
>> >> >> Unfortunately, no, there is no timeline. Several subjects are being
>> >> >> addressed
>> >> >> at the same time so it's a matter of priority. Right now we have to
>> >> >> rework
>> >> >> all
>> >> >> the low-level connection management to ensure proper integration of
>> >> >> SSL,
>> >> >> so we
>> >> >> will see the stick tables after that.
>> >> >>
>> >> >> Regards,
>> >> >> Willy
>> >> >>
>> >> >
>> >
>> >
>
>

Re: urls in stick-table, any timeline?

[S Ahmed]

Nice!

Is this in the latest 1.4 release or a dev release?

I need to rate limit on a URL (that includes query string values) like:

api.example.com/some_thing/other?id=asdf234asdf234&id2=asdf234234

Multiple sources are possible, but I don't care of the source IP I just
want to rate limit on the URL (customers will be using this endpoint from
their servers, but I want to rate limit to x number of requests per minute,
if they go over, just drop the request until the rate limit has expired).


On Tue, Apr 23, 2013 at 1:39 AM, Baptiste  wrote:

> Hi Ahmed,
>
> Yes, it has been implemented.
> You can store a URL and rate limited on it.
>
> Baptiste
>
> On Mon, Apr 22, 2013 at 11:15 PM, S Ahmed  wrote:
> > Hello,
> >
> > Has this feature been released yet by any change? :)
> >
> > Again my initial request was to do:
> >
> > I was told that soon you will be able to store a URL in a stick-table,
> so I
> > could block a particular url and then remove the block by making a
> request.
> >
> > I situation is I will be blocking up to 250K urls (for rate limiting,
> hard
> > drop any connection.for a given period of time, then I need to un-block
> > them).
> >
> > Any rough timelines on when this might be released?
> >
> >>Soon, you'll be able to store an URL in a stick-table, so you'll be
> >>able to update a gpc counter by setting up a particular header on the
> >>server side which tells HAProxy to block this request.
> >>For the cancellation of this blocking system, you could request the
> >>URL with a particular header to unblock it.
> >
> >
> > On Tue, Aug 21, 2012 at 2:04 PM, Baptiste  wrote:
> >>
> >> Hey,
> >>
> >> Nothing coming right now.
> >> Maybe for Christmas :)
> >>
> >> cheers
> >>
> >> On Tue, Aug 21, 2012 at 5:19 PM, S Ahmed  wrote:
> >> > Hello,
> >> >
> >> > Any updates or guestimates on if sticky-table feature will be
> released?
> >> >
> >> > Just haven't been watching this list for a while and curious if there
> >> > has
> >> > been any progress.
> >> >
> >> > Appreciate it!
> >> >
> >> > On Sun, Jun 24, 2012 at 1:28 AM, Willy Tarreau  wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> On Thu, Jun 21, 2012 at 05:16:22PM -0400, S Ahmed wrote:
> >> >> > I was told that soon you will be able to store a URL in a
> >> >> > stick-table,
> >> >> > so I
> >> >> > could block a particular url and then remove the block by making a
> >> >> > request.
> >> >> >
> >> >> > I situation is I will be blocking up to 250K urls (for rate
> limiting,
> >> >> > hard
> >> >> > drop any connection.for a given period of time, then I need to
> >> >> > un-block
> >> >> > them).
> >> >> >
> >> >> > Any rough timelines on when this might be released?
> >> >>
> >> >> Unfortunately, no, there is no timeline. Several subjects are being
> >> >> addressed
> >> >> at the same time so it's a matter of priority. Right now we have to
> >> >> rework
> >> >> all
> >> >> the low-level connection management to ensure proper integration of
> >> >> SSL,
> >> >> so we
> >> >> will see the stick tables after that.
> >> >>
> >> >> Regards,
> >> >> Willy
> >> >>
> >> >
> >
> >
>

Re: urls in stick-table, any timeline?

[Baptiste]

Hi Ahmed,

Yes, it has been implemented.
You can store a URL and rate limited on it.

Baptiste

On Mon, Apr 22, 2013 at 11:15 PM, S Ahmed  wrote:
> Hello,
>
> Has this feature been released yet by any change? :)
>
> Again my initial request was to do:
>
> I was told that soon you will be able to store a URL in a stick-table, so I
> could block a particular url and then remove the block by making a request.
>
> I situation is I will be blocking up to 250K urls (for rate limiting, hard
> drop any connection.for a given period of time, then I need to un-block
> them).
>
> Any rough timelines on when this might be released?
>
>>Soon, you'll be able to store an URL in a stick-table, so you'll be
>>able to update a gpc counter by setting up a particular header on the
>>server side which tells HAProxy to block this request.
>>For the cancellation of this blocking system, you could request the
>>URL with a particular header to unblock it.
>
>
> On Tue, Aug 21, 2012 at 2:04 PM, Baptiste  wrote:
>>
>> Hey,
>>
>> Nothing coming right now.
>> Maybe for Christmas :)
>>
>> cheers
>>
>> On Tue, Aug 21, 2012 at 5:19 PM, S Ahmed  wrote:
>> > Hello,
>> >
>> > Any updates or guestimates on if sticky-table feature will be released?
>> >
>> > Just haven't been watching this list for a while and curious if there
>> > has
>> > been any progress.
>> >
>> > Appreciate it!
>> >
>> > On Sun, Jun 24, 2012 at 1:28 AM, Willy Tarreau  wrote:
>> >>
>> >> Hi,
>> >>
>> >> On Thu, Jun 21, 2012 at 05:16:22PM -0400, S Ahmed wrote:
>> >> > I was told that soon you will be able to store a URL in a
>> >> > stick-table,
>> >> > so I
>> >> > could block a particular url and then remove the block by making a
>> >> > request.
>> >> >
>> >> > I situation is I will be blocking up to 250K urls (for rate limiting,
>> >> > hard
>> >> > drop any connection.for a given period of time, then I need to
>> >> > un-block
>> >> > them).
>> >> >
>> >> > Any rough timelines on when this might be released?
>> >>
>> >> Unfortunately, no, there is no timeline. Several subjects are being
>> >> addressed
>> >> at the same time so it's a matter of priority. Right now we have to
>> >> rework
>> >> all
>> >> the low-level connection management to ensure proper integration of
>> >> SSL,
>> >> so we
>> >> will see the stick tables after that.
>> >>
>> >> Regards,
>> >> Willy
>> >>
>> >
>
>

Re: urls in stick-table, any timeline?

[S Ahmed]

Hello,

Has this feature been released yet by any change? :)

Again my initial request was to do:

I was told that soon you will be able to store a URL in a stick-table, so I
could block a particular url and then remove the block by making a request.

I situation is I will be blocking up to 250K urls (for rate limiting, hard
drop any connection.for a given period of time, then I need to un-block
them).

Any rough timelines on when this might be released?

>Soon, you'll be able to store an URL in a stick-table, so you'll be
>able to update a gpc counter by setting up a particular header on the
>server side which tells HAProxy to block this request.
>For the cancellation of this blocking system, you could request the
>URL with a particular header to unblock it.


On Tue, Aug 21, 2012 at 2:04 PM, Baptiste  wrote:

> Hey,
>
> Nothing coming right now.
> Maybe for Christmas :)
>
> cheers
>
> On Tue, Aug 21, 2012 at 5:19 PM, S Ahmed  wrote:
> > Hello,
> >
> > Any updates or guestimates on if sticky-table feature will be released?
> >
> > Just haven't been watching this list for a while and curious if there has
> > been any progress.
> >
> > Appreciate it!
> >
> > On Sun, Jun 24, 2012 at 1:28 AM, Willy Tarreau  wrote:
> >>
> >> Hi,
> >>
> >> On Thu, Jun 21, 2012 at 05:16:22PM -0400, S Ahmed wrote:
> >> > I was told that soon you will be able to store a URL in a stick-table,
> >> > so I
> >> > could block a particular url and then remove the block by making a
> >> > request.
> >> >
> >> > I situation is I will be blocking up to 250K urls (for rate limiting,
> >> > hard
> >> > drop any connection.for a given period of time, then I need to
> un-block
> >> > them).
> >> >
> >> > Any rough timelines on when this might be released?
> >>
> >> Unfortunately, no, there is no timeline. Several subjects are being
> >> addressed
> >> at the same time so it's a matter of priority. Right now we have to
> rework
> >> all
> >> the low-level connection management to ensure proper integration of SSL,
> >> so we
> >> will see the stick tables after that.
> >>
> >> Regards,
> >> Willy
> >>
> >
>

Re: urls in stick-table, any timeline?

[Willy Tarreau]

On Tue, Aug 21, 2012 at 11:19:57AM -0400, S Ahmed wrote:
> Hello,
> 
> Any updates or guestimates on if sticky-table feature will be released?
> 
> Just haven't been watching this list for a while and curious if there has
> been any progress.

Just thinking out loud, until the "track url" feature is implemented, I
believe you could chain two instances of haproxy and achieve what you need :

  listen first-layer
 bind :80
 stick-table type string len 100 size 250k
 stick-match url
 balance round-robin
 server limit 127.0.0.1: weight 0 id 1 maxconn 1000
 server srv1 192.168.1.1:80
 server srv2 192.168.1.2:80
 server srv3 192.168.1.3:80

  listen second-layer
 bind 127.0.0.1:
 rate-limit 100
 balance round-robin
 server srv1 192.168.1.1:80
 server srv2 192.168.1.2:80
 server srv3 192.168.1.3:80

Above, the first layer receives requests. If the URL is found in the stick
table, it uses the server stored there, otherwise it uses the normal servers
(srv1..srv3) which are the only ones with a non-zero weight. The idea is to
only assign server "limit" to the URLs which match the stick table so that
this server is selected only for URLs you want.

When this second server is selected, the traffic is forwarded to the second
proxy (second-layer) which uses the same servers (for instance) and applies
a rate-limit of 100 req/s. Since the first layer's maxconn also has a maxconn,
excess requests will be queued.

In order to force the ID, I see two possibilities :
  - use a "store-request url if XXX" statement where XXX is a condition
on the request which also ensures that the requested server will be
used, for instance based on a cookie or source address. For instance :

  stick store-request url if { src 192.168.0.0/24 }
  use-server limit if { src 192.168.0.0/24 }

That way entries will only be added when the traffic is sent to that
server. However I don't see how to remove them.

  - manually feed these entries on the CLI :

  echo "set table first-layer key /a/b/c" | socat stdio 
/var/run/haproxy.sock

The advantage of the last method is that you can add/remove/list table
entries from the same control point, which is much better in my opinion.

Hoping this helps,
Willy

Re: urls in stick-table, any timeline?

[Baptiste]

Hey,

Nothing coming right now.
Maybe for Christmas :)

cheers

On Tue, Aug 21, 2012 at 5:19 PM, S Ahmed  wrote:
> Hello,
>
> Any updates or guestimates on if sticky-table feature will be released?
>
> Just haven't been watching this list for a while and curious if there has
> been any progress.
>
> Appreciate it!
>
> On Sun, Jun 24, 2012 at 1:28 AM, Willy Tarreau  wrote:
>>
>> Hi,
>>
>> On Thu, Jun 21, 2012 at 05:16:22PM -0400, S Ahmed wrote:
>> > I was told that soon you will be able to store a URL in a stick-table,
>> > so I
>> > could block a particular url and then remove the block by making a
>> > request.
>> >
>> > I situation is I will be blocking up to 250K urls (for rate limiting,
>> > hard
>> > drop any connection.for a given period of time, then I need to un-block
>> > them).
>> >
>> > Any rough timelines on when this might be released?
>>
>> Unfortunately, no, there is no timeline. Several subjects are being
>> addressed
>> at the same time so it's a matter of priority. Right now we have to rework
>> all
>> the low-level connection management to ensure proper integration of SSL,
>> so we
>> will see the stick tables after that.
>>
>> Regards,
>> Willy
>>
>

Re: urls in stick-table, any timeline?

[S Ahmed]

Hello,

Any updates or guestimates on if sticky-table feature will be released?

Just haven't been watching this list for a while and curious if there has
been any progress.

Appreciate it!

On Sun, Jun 24, 2012 at 1:28 AM, Willy Tarreau  wrote:

> Hi,
>
> On Thu, Jun 21, 2012 at 05:16:22PM -0400, S Ahmed wrote:
> > I was told that soon you will be able to store a URL in a stick-table,
> so I
> > could block a particular url and then remove the block by making a
> request.
> >
> > I situation is I will be blocking up to 250K urls (for rate limiting,
> hard
> > drop any connection.for a given period of time, then I need to un-block
> > them).
> >
> > Any rough timelines on when this might be released?
>
> Unfortunately, no, there is no timeline. Several subjects are being
> addressed
> at the same time so it's a matter of priority. Right now we have to rework
> all
> the low-level connection management to ensure proper integration of SSL,
> so we
> will see the stick tables after that.
>
> Regards,
> Willy
>
>

Re: urls in stick-table, any timeline?

[S Ahmed]

So would you say weeks away, or months away, or a year away? (rough
estimate)

Appreciate your time!

On Sun, Jun 24, 2012 at 1:28 AM, Willy Tarreau  wrote:

> Hi,
>
> On Thu, Jun 21, 2012 at 05:16:22PM -0400, S Ahmed wrote:
> > I was told that soon you will be able to store a URL in a stick-table,
> so I
> > could block a particular url and then remove the block by making a
> request.
> >
> > I situation is I will be blocking up to 250K urls (for rate limiting,
> hard
> > drop any connection.for a given period of time, then I need to un-block
> > them).
> >
> > Any rough timelines on when this might be released?
>
> Unfortunately, no, there is no timeline. Several subjects are being
> addressed
> at the same time so it's a matter of priority. Right now we have to rework
> all
> the low-level connection management to ensure proper integration of SSL,
> so we
> will see the stick tables after that.
>
> Regards,
> Willy
>
>

Re: urls in stick-table, any timeline?

[Willy Tarreau]

Hi,

On Thu, Jun 21, 2012 at 05:16:22PM -0400, S Ahmed wrote:
> I was told that soon you will be able to store a URL in a stick-table, so I
> could block a particular url and then remove the block by making a request.
> 
> I situation is I will be blocking up to 250K urls (for rate limiting, hard
> drop any connection.for a given period of time, then I need to un-block
> them).
> 
> Any rough timelines on when this might be released?

Unfortunately, no, there is no timeline. Several subjects are being addressed
at the same time so it's a matter of priority. Right now we have to rework all
the low-level connection management to ensure proper integration of SSL, so we
will see the stick tables after that.

Regards,
Willy