A "different kind of rate limiting"
Hey guys, I'm trying to find a solution to a problem I'm having This might be a unique use case, but the "why" is a bit complicated so I'll just leave that out of the picture for now. I would like to make a maximum of 50 requests per second to my backend (or, optionally, one request per second per each of the 50 backend servers). This can't be accomplished with normal session rate limiting because of a catch: I don't want HAProxy to reject the request. Is there any way to have HAproxy accept and queue the requests, but throttle the backend requests to 50/sec or 1/server/sec? The goal is to make efficient use of 50 requests per second. If not, can anyone think of any creative ways to accomplish this? Maybe via PF/IPFW? Any help would be appreciated :) Thanks, Will
Re: Weird problem with stats_uri
On Sun, Apr 5, 2009 at 1:06 PM, Will Buckner wrote: > Hey guys, > > I'm seeing an interesting thing with 1.3.x in my production environment... I > have my stats_uri defined as /proxy_http_stats, requiring a username and > password. When I access this URL, about half the time I'm taken to a 404 > page on my destination servers (haproxy never catches the request for stats) > and about half of the time I'll get the stats page. Any ideas on where I can > start looking? > start by first eliminating the question of caches from your equation. What do u mean by "haproxy never catches the request for stats"? Does it at least come out in the haproxy logs (ie. haproxy *did* see the request - but just didnt interpret it as a stats_uri)? -jf -- In the meantime, here is your PSA: "It's so hard to write a graphics driver that open-sourcing it would not help." -- Andrew Fear, Software Product Manager, NVIDIA Corporation http://kerneltrap.org/node/7228
Weird problem with stats_uri
Hey guys, I'm seeing an interesting thing with 1.3.x in my production environment... I have my stats_uri defined as /proxy_http_stats, requiring a username and password. When I access this URL, about half the time I'm taken to a 404 page on my destination servers (haproxy never catches the request for stats) and about half of the time I'll get the stats page. Any ideas on where I can start looking? Thanks, Will
Re: Delay incoming tcp connections
Hi, On Sat, Apr 04, 2009 at 07:46:28PM +0400, Alexey wrote: > Hi, > > I saw post about delaying incoming smtp connections via haproxy. Looks like I > need transparent proxy for saving source ip addresses, but it requires TPROXY > in linux kernel. yes it does. > I need to patch kernel + iptables for make it working? Yes. Malcolm Turnbull posted a howto on the subject. > What > difference between squid and haproxy transparenting (squid requires only -j > REDIRECT support in kernel) ? - Squid does only HTTP, not TCP - the "-j REDIRECT" method only works for destination, but does not help binding to the client's source address. > Is there any simpiest ways to delay incoming tcp > connections without changin source address? Not that I'm aware of. This is also called "delayed binding" and at least requires an equipment which is able to translate TCP sequence numbers. Doing that in a proxy is the simplest and most reliable method to do this, but this requires a very recent linux kernel (>= 2.6.28) or to apply the TProxy patch. Regards, Willy
Delay incoming tcp connections
Hi, I saw post about delaying incoming smtp connections via haproxy. Looks like I need transparent proxy for saving source ip addresses, but it requires TPROXY in linux kernel. I need to patch kernel + iptables for make it working? What difference between squid and haproxy transparenting (squid requires only -j REDIRECT support in kernel) ? Is there any simpiest ways to delay incoming tcp connections without changin source address? Alexey.
Re: tcp proxy
On Sat, Apr 04, 2009 at 11:43:38AM -0300, Nicolas Cohen wrote: > Hi Willy, > > It seems right to implement it. > I'll review this with the team and let you know once we have an > available patch. Nice, thanks! Willy
Re: tcp proxy
Hi Willy, It seems right to implement it. I'll review this with the team and let you know once we have an available patch. regards, n. On 04/04/2009, at 03:49, Willy Tarreau wrote: Hi Nicolas, On Fri, Apr 03, 2009 at 10:29:32PM -0300, Nicolas Cohen wrote: hi, i want to use haproxy to load balance a virtual world app we are developing in java, the app server benefits if most connected users in one particular machine are in the same regions of the virtual world (less objects need to be referrenced and caching strategies are more efficient) i thought of making the login process map the region of the world where the user's avatar is placed to one of a set of subdomains (for instance: region1.domain.com, regions2.domain.com). i would then make the DNS assign a different ip for each subdomain. all ips would be configured for the same haproxy machine, which would try to put all connections to the same ip in the same machine, and then fallback to leastconn if a server is too loaded. this is basically the same algorithm than *source* but for destination. is there any way to achieve this in haproxy? It depends on how many regions you have. If you only have a small set of regions, you can already achieve this using ACLs on the destination address, using which you will select a backend. But this will quickly become very boring when you have more than 4-5 regions. Another possibility is to implement the "balance dstip" algo which should be pretty trivial. If you're deploying a new site, I think it makes a lot of sense to adapt the tools to do the job right. Do you feel OK with adding this feature yourself ? I could then merge it back into next release, as I think it will not be an invasive change at all. Regards, Willy
Re: patch: nested acl evaluation
On Sat, Apr 4, 2009 at 3:30 PM, Willy Tarreau wrote: > On Sat, Apr 04, 2009 at 10:20:23AM +0800, Jeffrey 'jf' Lim wrote: >> > OK maybe "use" is OK in fact, considering the alternatives. >> > >> >> :) some proposals for the keywords: >> >> for/use >> condition/use >> cond/use >> >> (cond/use seems the best compromise - short, but understandable enough) > > what would you think about "do/use" ? > I actually prefer cond/use (more intuitive with the idea of an "if" block), but hey, you're the man... :P >> > If we extend the system, we'll have to associate parameters >> > with a condition. >> > >> > But since the entry point is the switching rule here, maybe we'll end up >> > with something very close to what you have implemented in your patch, in >> > the end, and it's just that we want to make it more generic to use the >> > two conditions layers in any ruleset. >> > >> >> I would guess so! Even the redirect rules and 'block' rules look >> pretty similar... :) > > yes, and maybe this is what we should try to improve : converge towards > a generic condition matching/execution system, to which we pass the > action and the args. That way, we just have to run it over redirect_rules > or switching_rules always using the same code. > :) right... :D Does this sound like fun? lol -jf
Re: patch: nested acl evaluation
On Sat, Apr 04, 2009 at 10:20:23AM +0800, Jeffrey 'jf' Lim wrote: > > OK maybe "use" is OK in fact, considering the alternatives. > > > > :) some proposals for the keywords: > > for/use > condition/use > cond/use > > (cond/use seems the best compromise - short, but understandable enough) what would you think about "do/use" ? > > If we extend the system, we'll have to associate parameters > > with a condition. > > > > But since the entry point is the switching rule here, maybe we'll end up > > with something very close to what you have implemented in your patch, in > > the end, and it's just that we want to make it more generic to use the > > two conditions layers in any ruleset. > > > > I would guess so! Even the redirect rules and 'block' rules look > pretty similar... :) yes, and maybe this is what we should try to improve : converge towards a generic condition matching/execution system, to which we pass the action and the args. That way, we just have to run it over redirect_rules or switching_rules always using the same code. Willy
