Re: [PATCH] MEDIUM: stats: Add `show events` command.
Hi Aman, On Mon, Apr 02, 2012 at 03:34:08PM -0700, Aman Gupta wrote: OK just to be sure, you should *really* make your changes on 1.5-dev, not 1.4. 1.4 is in deep maintenance mode and I don't intend to merge such changes there, as every time I did I caused some regressions. I'm in process of porting my patch to 1.5. I don't particularly want to run 1.5 in production though, and it occurs to me that my patch is quite involved for what I'm trying to accomplish. My goal is essentially to have the remote ip of the upstream client available to my application (sitting behind haproxy). For http backends, I can use forwardfor and it works great. However, this specific service is raw tcp. I know I can use tproxy, but would prefer to avoid upgrading my kernel and setting up additional iptables rules. The idea with this patch was that the application could connect to haproxy to get events and use that data to figure out the upstream client's ip. This adds a lot of complexity though, so I'm trying to come up with alternatives. The only other obvious solution is to have haproxy prepend something to the tcp stream. In my case this will work fine, since I can modify my application to extract this from the stream before consuming the client data. This is the principle of the PROXY protocol. The send-proxy server option was added to 1.5-dev, but it's not in 1.4 though I have the patch to make this possible. It requires very minor changes to the application and I know that some people running FTP servers and SSL servers have modified them to parse this line. The protocol was also already adopted by Stud and Stunnel. The protocol is described here : http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt Please let me know whether you're interested, then I'll check where I left the patch :-) Willy
Re: [PATCH] MEDIUM: stats: Add `show events` command.
On Mon, Apr 2, 2012 at 11:21 PM, Willy Tarreau w...@1wt.eu wrote: Hi Aman, On Mon, Apr 02, 2012 at 03:34:08PM -0700, Aman Gupta wrote: OK just to be sure, you should *really* make your changes on 1.5-dev, not 1.4. 1.4 is in deep maintenance mode and I don't intend to merge such changes there, as every time I did I caused some regressions. I'm in process of porting my patch to 1.5. I don't particularly want to run 1.5 in production though, and it occurs to me that my patch is quite involved for what I'm trying to accomplish. My goal is essentially to have the remote ip of the upstream client available to my application (sitting behind haproxy). For http backends, I can use forwardfor and it works great. However, this specific service is raw tcp. I know I can use tproxy, but would prefer to avoid upgrading my kernel and setting up additional iptables rules. The idea with this patch was that the application could connect to haproxy to get events and use that data to figure out the upstream client's ip. This adds a lot of complexity though, so I'm trying to come up with alternatives. The only other obvious solution is to have haproxy prepend something to the tcp stream. In my case this will work fine, since I can modify my application to extract this from the stream before consuming the client data. This is the principle of the PROXY protocol. The send-proxy server option was added to 1.5-dev, but it's not in 1.4 though I have the patch to make this possible. It requires very minor changes to the application and I know that some people running FTP servers and SSL servers have modified them to parse this line. The protocol was also already adopted by Stud and Stunnel. The protocol is described here : http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt Please let me know whether you're interested, then I'll check where I left the patch :-) This is exactly what I am looking for. I am very much interested in a 1.4 backport, and happy to help with the porting and testing process. Aman Willy
Re: [PATCH] MEDIUM: stats: Add `show events` command.
On Tue, Apr 3, 2012 at 12:10 AM, Brane F. Gračnar brane.grac...@tsmedia.si wrote: On 04/03/2012 08:21 AM, Willy Tarreau wrote: The protocol is described here : http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt Please let me know whether you're interested, then I'll check where I left the patch :-) Here ya go :) Thanks, but this appears to be the accept-proxy side of the implementation. I don't see any send-proxy code. Aman Best regards, Brane
acl re-using in another acl ?
Hello all, I am starting to hit some limitations in haproxy's configuration options ! I have decided to use this product to sit in front of a cloud environment hosting several websites as I have used it many times for several years and have alays found it very, very efficient. At the moment I have over 20 hosted websites (much more might come in). So basically, at least as much ACL's for this (and actually much, much more). I do NOT want a default_backend. On the contrary, I need (security reasons) to block access excepted if the user comes with the correct http header (virtual hosting). I also have to grant access to the site to a restricted list of IP's (developer agencies) before an official go-live is set up for specific sites, or for some admin areas of those websites. So, in other words, I have one HUGE line with is a : block unless is_website_01 || is_website02 || is_website_02 || is_allowed_agency_01 || is_allowed_agency_02 ... Then follow the use_backend website_01 if is_website_01 (and potentially also if_allowed_agency_01) And... I have faced a limit of over 2047 chracters in one line, now ! I tried to rewrite the ACL's list in this way : acl is_website_01 www.website01.com (already existing) acl is_website_02 www.website02.com (already existing) acl is_all_allowed_websites is_website_01 acl is_all_allowed_websites is_website_02 acl is_allowed_agency_01 x.x.x.x (already existing) acl is_allowed_agency_02 y.y.y.y (already existing) acl is_all_allowed_agencies is_allowed_agency_01 acl is_all_allowed_agencies is_allowed_agency_02 block unless is_all_allowed_websites || is_all_allowed_agencies But apparently this approach is not possible in haproxy ? That leaves me with the only option of re-writing all ACL's twice, then... acl is_website_01 www.website01.com (already existing) acl is_website_02 www.website02.com (already existing) acl is_all_allowed_websites www.website01.com acl is_all_allowed_websites www.website02.com acl is_allowed_agency_01 x.x.x.x (already existing) acl is_allowed_agency_02 y.y.y.y (already existing) acl is_all_allowed_agencies x.x.x.x acl is_all_allowed_agencies y.y.y.y block unless is_all_allowed_websites || is_all_allowed_agencies Is there no way to have the preferred option of re-using ACL's into another ACL's possible ? Many thanks !! Guillaume Demillecamps
Re: balance uri archaic options
Le (On) Tue, Apr 03, 2012 at 07:45:03AM +0200, Baptiste ecrivit (wrote): Hi, May I ask you why you want absolutely not hash this part? I clearly don't understand, and I don't see any disadvantage of hashing a full URI. The full URI includes informations (like a key for instance) that shouldn't be hashed since I want to maintain a specific cache affinity based on a certain part of the URI. I understand this is not possible to do easily, and I just find it weird since haproxy is full of powerful regexps, but that's fine, I can hash according to the first 3 characters of the URI, it's just not elegant. Cheers, On Mon, Apr 2, 2012 at 8:28 PM, Sameh Ghane sa...@anthologeek.net wrote: Le (On) Mon, Apr 02, 2012 at 06:32:31PM +0100, Jonathan Matthews ecrivit (wrote): On 2 April 2012 17:25, Sameh Ghane sameh+s...@anthologeek.net wrote: Is there anything I missed ? Like setting a specific request header which would be the result of a regexp on the URI, and balancing on that header ? I don't understand what you've written. Could you say it again, but differently? Some examples would probably help too. Sorry for this. Imagine I see requests for /x/yy.z and I would like to balance according to the URI, though I don't want the y part to be fed to the hash function. As far as I understand, it cannot be done with 'balance uri'. That's why I would like to know if it can be circumvented through something like: backend be .. reqadd balhdr:\ xxx balance hdr(balhdr) .. Cheers, -- Sameh Ghane -- Sameh Ghane
Re: Need your suggestion
Hi, HAProxy can do it pretty easily :) You just need enough memory (2G is more than enough for 20K connections). Regards On Tue, Apr 3, 2012 at 10:00 AM, David Yeh da...@alleyoopcomm.com wrote: Hi Sir, After searching the web I found this site, and after carefully read them all, still can't make up my mind which load balance is better for us application. So I need your advice. Hope it's not disturbing you a lot. We are a game company in Taiwan, have about 10 game servers to handle about 2 tcp connection simultaneously , I have tried ipvs, pen (zen loadbalance) , somehow they have their own problem. Pen was a CPU killer program, and ipvs behavior was very strange that I can't describe it at all. Since we need a BL that can do NAT to back end server in a 2 tcp connections simultaneously loading, do you think HAProxy capable of it? Or HAProxy is only for HTTP not for raw TCP? I am very appreciated that take your time to answer my question, thanks very much. David Yeh
[no subject]
a href=http://toycarnival.in/wp-content/uploads/widget_imgs/jrklre.html; http://toycarnival.in/wp-content/uploads/widget_imgs/jrklre.html/a
Re: TCP retransmissions
Hi Willy, that sounds interesting because we are using Cisco firewall as well. So that issue might be related to that. Our current situation is that we switched back to apache2 with mod_balancer since we currently haven't enough time to investigate since a datacenter move is going on and we have to keep things stable until then. Since we switched back, our users didn't experience upload/download problems anymore. Not sure why it is working now. I will have more time to investigate after our datacenter move, which will be around June since we are planning to switch back to haproxy. So if I can do further investigation on that in June please let me know if I can provide you with more data, or at least I can do more tests regarding our firewall to make sure that we don't have an issue there. Please find the dump of a failing download at ftp://ftp.suse.com/pub/people/wengel/haproxy/haproxy-download.dump Thanks for your help so far. Regards, Wolfgang On 27.03.2012 20:01, Willy Tarreau wrote: Hi Wolfgang, On Tue, Mar 27, 2012 at 12:45:25PM +0200, Wolfgang Engel wrote: Hi, I just want to follow up on a thread from december 2011 where Simon experienced problems regarding TCP retransmissions. We are seeing this problem on our site (susestudio.com) as well for upload/download since we are using haproxy version 1.5-dev7 on SLES11SP1 with kernel version 2.6.32.54-0.3-default. Not all users seems to be affected and it is hard to reproduce since it happens randomly. I switched off TCP segment offload like Willy suggested for testing. I can provide tcpdump of some interrupted download transfer from our site where you might have a look at. Does someone else experience this problem ? Just for the record, Simon's issue was caused by an overzealous cisco firewall performing deep inspection but not exactly knowing the HTTP protocol, resulting in some WebSocket communications being blocked on port 80 (since it does not inspect other ports). Another user, Jason Strimpel faced the exact same issue with WebSocket, this time with a TrendMicro Officescan firewall installed on the client machine. Same observation, by switching to another port everything went smooth. So I'm not sure you're discussing exactly the same kind of issues, but just in case I'd be very interested in getting a trace showing these retransmits, and if possible a few session before related to the same source IP. It is indeed possible that what you're seeing is related to an incompatibility between your local TCP stack and the other side's, which unfortunately happens to be more and more common with mobile operators doing funny things in order to offer connectivity to more users than they have available IP addresses. Best regards, Willy
Re: acl re-using in another acl ?
On Tue, Apr 3, 2012 at 1:42 AM, Guillaume Demillecamps guilla...@multipurpose.be wrote: Hello all, I am starting to hit some limitations in haproxy's configuration options ! I have decided to use this product to sit in front of a cloud environment hosting several websites as I have used it many times for several years and have alays found it very, very efficient. At the moment I have over 20 hosted websites (much more might come in). So basically, at least as much ACL's for this (and actually much, much more). I do NOT want a default_backend. On the contrary, I need (security reasons) to block access excepted if the user comes with the correct http header (virtual hosting). I also have to grant access to the site forgive me if i've not comprehended your requirements correctly, but could you simplify your config by adding a default_backend that hosts all of the non-authorized requests? this way the default_backend is a trash bucket that hosts all the junk that your acl's haven't found a proper backend for based on host header or src ip. that backend could even be another haproxy frontend which simply blocks everything coming in. to a restricted list of IP's (developer agencies) before an official go-live is set up for specific sites, or for some admin areas of those websites. So, in other words, I have one HUGE line with is a : block unless is_website_01 || is_website02 || is_website_02 || is_allowed_agency_01 || is_allowed_agency_02 ... Then follow the use_backend website_01 if is_website_01 (and potentially also if_allowed_agency_01) And... I have faced a limit of over 2047 chracters in one line, now ! I tried to rewrite the ACL's list in this way : acl is_website_01 www.website01.com (already existing) acl is_website_02 www.website02.com (already existing) acl is_all_allowed_websites is_website_01 acl is_all_allowed_websites is_website_02 acl is_allowed_agency_01 x.x.x.x (already existing) acl is_allowed_agency_02 y.y.y.y (already existing) acl is_all_allowed_agencies is_allowed_agency_01 acl is_all_allowed_agencies is_allowed_agency_02 block unless is_all_allowed_websites || is_all_allowed_agencies But apparently this approach is not possible in haproxy ? That leaves me with the only option of re-writing all ACL's twice, then... acl is_website_01 www.website01.com (already existing) acl is_website_02 www.website02.com (already existing) acl is_all_allowed_websites www.website01.com acl is_all_allowed_websites www.website02.com acl is_allowed_agency_01 x.x.x.x (already existing) acl is_allowed_agency_02 y.y.y.y (already existing) acl is_all_allowed_agencies x.x.x.x acl is_all_allowed_agencies y.y.y.y block unless is_all_allowed_websites || is_all_allowed_agencies Is there no way to have the preferred option of re-using ACL's into another ACL's possible ? Many thanks !! Guillaume Demillecamps
