Re: could haproxy call redis for a result?
Baptiste, Whenever this feature will be implemented, will it work for a specific url like: subdomain1.example.com What about by query string? like: www.example.com/customer/12345 or www.example.com/some/path?customerId=12345 Will it work for all the above? On Tue, May 8, 2012 at 9:38 PM, S Ahmed sahmed1...@gmail.com wrote: Yes it is the lookup that I am worried about. On Tue, May 8, 2012 at 5:46 PM, Baptiste bed...@gmail.com wrote: Hi, Willy has just released 1.5-dev9, but unfortunately the track functions can't yet track strings (and so URLs). I'll let you know once a nightly snapshot could do it and we could work on a proof of concept configuration. Concerning 250K URLs, that should not be an issue at all to store them. Maybe looking for one URL could have a performance impact, we'll see. cheers On Tue, May 8, 2012 at 10:00 PM, S Ahmed sahmed1...@gmail.com wrote: Great. So any ideas how many urls one can story in these sticky tables before it becomes a problem? Would 250K be something of a concern? On Tue, May 8, 2012 at 11:26 AM, Baptiste bed...@gmail.com wrote: On Tue, May 8, 2012 at 3:25 PM, S Ahmed sahmed1...@gmail.com wrote: Ok that sounds awesome, how will that work though? i.e. from say java, how will I do that? From what your saying it sounds like I will just have to modify the response add and a particular header. And on the flip side, if I want to unblock I'll make a http request with something in the header that will unblock it? That's it. You'll have to track these headers with ACLs in HAProxy and to update the stick table accordingly. Then based on the value setup in the stick table, HAProxy can decide whether it will allow or reject the request. When do you think this will go live? In an other mail, Willy said he will release 1.5-dev9 today. So I guess it won't be too long now. Worste case would be later in the week or next week. cheers
Re: could haproxy call redis for a result?
Hi, I'm jut guessing, but to me it can work for URLs only, so in your case, it will match /, /customer/12345, and /some/path?customerId=12345. For now, the string table can't have a concatenated string of 2 information, Host header and URL in your case. But who knows, maybe this feature will arrive soon too :) cheers On Thu, May 24, 2012 at 6:33 PM, S Ahmed sahmed1...@gmail.com wrote: Baptiste, Whenever this feature will be implemented, will it work for a specific url like: subdomain1.example.com What about by query string? like: www.example.com/customer/12345 or www.example.com/some/path?customerId=12345 Will it work for all the above? On Tue, May 8, 2012 at 9:38 PM, S Ahmed sahmed1...@gmail.com wrote: Yes it is the lookup that I am worried about. On Tue, May 8, 2012 at 5:46 PM, Baptiste bed...@gmail.com wrote: Hi, Willy has just released 1.5-dev9, but unfortunately the track functions can't yet track strings (and so URLs). I'll let you know once a nightly snapshot could do it and we could work on a proof of concept configuration. Concerning 250K URLs, that should not be an issue at all to store them. Maybe looking for one URL could have a performance impact, we'll see. cheers On Tue, May 8, 2012 at 10:00 PM, S Ahmed sahmed1...@gmail.com wrote: Great. So any ideas how many urls one can story in these sticky tables before it becomes a problem? Would 250K be something of a concern? On Tue, May 8, 2012 at 11:26 AM, Baptiste bed...@gmail.com wrote: On Tue, May 8, 2012 at 3:25 PM, S Ahmed sahmed1...@gmail.com wrote: Ok that sounds awesome, how will that work though? i.e. from say java, how will I do that? From what your saying it sounds like I will just have to modify the response add and a particular header. And on the flip side, if I want to unblock I'll make a http request with something in the header that will unblock it? That's it. You'll have to track these headers with ACLs in HAProxy and to update the stick table accordingly. Then based on the value setup in the stick table, HAProxy can decide whether it will allow or reject the request. When do you think this will go live? In an other mail, Willy said he will release 1.5-dev9 today. So I guess it won't be too long now. Worste case would be later in the week or next week. cheers
Re: could haproxy call redis for a result?
nginx would be more suitable for something like this. It even has a redis plugin: http://wiki.nginx.org/HttpRedis Perhaps you can achieve your functionality with the redis_next_upstream parameter. Sergej On Tue, May 8, 2012 at 4:39 AM, S Ahmed sahmed1...@gmail.com wrote: I agree it will add overheard for each call. Well would there a way for me to somehow tell haproxy from my application to block a particular url, and then send another api call to allow traffic from that url? That would be really cool to have an API where I could do this from. I know haproxy has rate limiting as per: http://blog.serverfault.com/2010/08/26/1016491873/ But wondering if one could have more control over it, like say you have multiple haproxy servers and you want to synch them, or simply the application layer needs to decide when to drop a url connection or when to accept. On Mon, May 7, 2012 at 7:39 PM, Baptiste bed...@gmail.com wrote: On Tue, May 8, 2012 at 12:26 AM, S Ahmed sahmed1...@gmail.com wrote: I'm sure this isn't possible but it would be cool if it is. My backend services write to redis, and if a client reaches a certain threshold, I want to hard drop all further requests until x minutes have passed. Would it be possible, for each request, haproxy performs a lookup in redis, and if a 0 is returned, drop the request completly (hard drop), if it is 1, continue processing. It would introduce latency in the request processing. Why would you need such way of serving your request? By the way, this is not doable with HAProxy. Well, at least, not out of the box :) Depending on your needs, you could hack some dirty scripts which can sync your redis DB with HAProxy server status through the stats socket. cheers
Re: could haproxy call redis for a result?
On Tue, May 8, 2012 at 4:39 AM, S Ahmed sahmed1...@gmail.com wrote: I agree it will add overheard for each call. Well would there a way for me to somehow tell haproxy from my application to block a particular url, and then send another api call to allow traffic from that url? This is different. Soon, you'll be able to store an URL in a stick-table, so you'll be able to update a gpc counter by setting up a particular header on the server side which tells HAProxy to block this request. For the cancellation of this blocking system, you could request the URL with a particular header to unblock it. It might be doable with HAProxy nightly snapshot, but you should definitely wait for Willy to provide the 1.5-dev9 which allows strings in stick-tables.
Re: could haproxy call redis for a result?
Ok that sounds awesome, how will that work though? i.e. from say java, how will I do that? From what your saying it sounds like I will just have to modify the response add and a particular header. And on the flip side, if I want to unblock I'll make a http request with something in the header that will unblock it? When do you think this will go live? On Tue, May 8, 2012 at 4:26 AM, Baptiste bed...@gmail.com wrote: On Tue, May 8, 2012 at 4:39 AM, S Ahmed sahmed1...@gmail.com wrote: I agree it will add overheard for each call. Well would there a way for me to somehow tell haproxy from my application to block a particular url, and then send another api call to allow traffic from that url? This is different. Soon, you'll be able to store an URL in a stick-table, so you'll be able to update a gpc counter by setting up a particular header on the server side which tells HAProxy to block this request. For the cancellation of this blocking system, you could request the URL with a particular header to unblock it. It might be doable with HAProxy nightly snapshot, but you should definitely wait for Willy to provide the 1.5-dev9 which allows strings in stick-tables.
Re: could haproxy call redis for a result?
On Tue, May 8, 2012 at 3:25 PM, S Ahmed sahmed1...@gmail.com wrote: Ok that sounds awesome, how will that work though? i.e. from say java, how will I do that? From what your saying it sounds like I will just have to modify the response add and a particular header. And on the flip side, if I want to unblock I'll make a http request with something in the header that will unblock it? That's it. You'll have to track these headers with ACLs in HAProxy and to update the stick table accordingly. Then based on the value setup in the stick table, HAProxy can decide whether it will allow or reject the request. When do you think this will go live? In an other mail, Willy said he will release 1.5-dev9 today. So I guess it won't be too long now. Worste case would be later in the week or next week. cheers
Re: could haproxy call redis for a result?
Great. So any ideas how many urls one can story in these sticky tables before it becomes a problem? Would 250K be something of a concern? On Tue, May 8, 2012 at 11:26 AM, Baptiste bed...@gmail.com wrote: On Tue, May 8, 2012 at 3:25 PM, S Ahmed sahmed1...@gmail.com wrote: Ok that sounds awesome, how will that work though? i.e. from say java, how will I do that? From what your saying it sounds like I will just have to modify the response add and a particular header. And on the flip side, if I want to unblock I'll make a http request with something in the header that will unblock it? That's it. You'll have to track these headers with ACLs in HAProxy and to update the stick table accordingly. Then based on the value setup in the stick table, HAProxy can decide whether it will allow or reject the request. When do you think this will go live? In an other mail, Willy said he will release 1.5-dev9 today. So I guess it won't be too long now. Worste case would be later in the week or next week. cheers
Re: could haproxy call redis for a result?
Hi, Willy has just released 1.5-dev9, but unfortunately the track functions can't yet track strings (and so URLs). I'll let you know once a nightly snapshot could do it and we could work on a proof of concept configuration. Concerning 250K URLs, that should not be an issue at all to store them. Maybe looking for one URL could have a performance impact, we'll see. cheers On Tue, May 8, 2012 at 10:00 PM, S Ahmed sahmed1...@gmail.com wrote: Great. So any ideas how many urls one can story in these sticky tables before it becomes a problem? Would 250K be something of a concern? On Tue, May 8, 2012 at 11:26 AM, Baptiste bed...@gmail.com wrote: On Tue, May 8, 2012 at 3:25 PM, S Ahmed sahmed1...@gmail.com wrote: Ok that sounds awesome, how will that work though? i.e. from say java, how will I do that? From what your saying it sounds like I will just have to modify the response add and a particular header. And on the flip side, if I want to unblock I'll make a http request with something in the header that will unblock it? That's it. You'll have to track these headers with ACLs in HAProxy and to update the stick table accordingly. Then based on the value setup in the stick table, HAProxy can decide whether it will allow or reject the request. When do you think this will go live? In an other mail, Willy said he will release 1.5-dev9 today. So I guess it won't be too long now. Worste case would be later in the week or next week. cheers
Re: could haproxy call redis for a result?
Yes it is the lookup that I am worried about. On Tue, May 8, 2012 at 5:46 PM, Baptiste bed...@gmail.com wrote: Hi, Willy has just released 1.5-dev9, but unfortunately the track functions can't yet track strings (and so URLs). I'll let you know once a nightly snapshot could do it and we could work on a proof of concept configuration. Concerning 250K URLs, that should not be an issue at all to store them. Maybe looking for one URL could have a performance impact, we'll see. cheers On Tue, May 8, 2012 at 10:00 PM, S Ahmed sahmed1...@gmail.com wrote: Great. So any ideas how many urls one can story in these sticky tables before it becomes a problem? Would 250K be something of a concern? On Tue, May 8, 2012 at 11:26 AM, Baptiste bed...@gmail.com wrote: On Tue, May 8, 2012 at 3:25 PM, S Ahmed sahmed1...@gmail.com wrote: Ok that sounds awesome, how will that work though? i.e. from say java, how will I do that? From what your saying it sounds like I will just have to modify the response add and a particular header. And on the flip side, if I want to unblock I'll make a http request with something in the header that will unblock it? That's it. You'll have to track these headers with ACLs in HAProxy and to update the stick table accordingly. Then based on the value setup in the stick table, HAProxy can decide whether it will allow or reject the request. When do you think this will go live? In an other mail, Willy said he will release 1.5-dev9 today. So I guess it won't be too long now. Worste case would be later in the week or next week. cheers
Re: could haproxy call redis for a result?
On Tue, May 8, 2012 at 12:26 AM, S Ahmed sahmed1...@gmail.com wrote: I'm sure this isn't possible but it would be cool if it is. My backend services write to redis, and if a client reaches a certain threshold, I want to hard drop all further requests until x minutes have passed. Would it be possible, for each request, haproxy performs a lookup in redis, and if a 0 is returned, drop the request completly (hard drop), if it is 1, continue processing. It would introduce latency in the request processing. Why would you need such way of serving your request? By the way, this is not doable with HAProxy. Well, at least, not out of the box :) Depending on your needs, you could hack some dirty scripts which can sync your redis DB with HAProxy server status through the stats socket. cheers
Re: could haproxy call redis for a result?
I agree it will add overheard for each call. Well would there a way for me to somehow tell haproxy from my application to block a particular url, and then send another api call to allow traffic from that url? That would be really cool to have an API where I could do this from. I know haproxy has rate limiting as per: http://blog.serverfault.com/2010/08/26/1016491873/ But wondering if one could have more control over it, like say you have multiple haproxy servers and you want to synch them, or simply the application layer needs to decide when to drop a url connection or when to accept. On Mon, May 7, 2012 at 7:39 PM, Baptiste bed...@gmail.com wrote: On Tue, May 8, 2012 at 12:26 AM, S Ahmed sahmed1...@gmail.com wrote: I'm sure this isn't possible but it would be cool if it is. My backend services write to redis, and if a client reaches a certain threshold, I want to hard drop all further requests until x minutes have passed. Would it be possible, for each request, haproxy performs a lookup in redis, and if a 0 is returned, drop the request completly (hard drop), if it is 1, continue processing. It would introduce latency in the request processing. Why would you need such way of serving your request? By the way, this is not doable with HAProxy. Well, at least, not out of the box :) Depending on your needs, you could hack some dirty scripts which can sync your redis DB with HAProxy server status through the stats socket. cheers
