Re: reqadd srcIP for header
Hey, Using stunnel and haproxy, both with the proxy protocol may help when you want to add the X-SRC-IP header for a SSL connection. cheers
Re: programatically checking failed machines
Could you at least provide your config? cheers On Mon, Jun 4, 2012 at 10:13 PM, Cord MacLeod cordmacl...@gmail.com wrote: Just got around to attempting this, it appears not to work with SSL. Is there any other option (stunnel breaks a lot and doesn't support 1 off connections)? * Reloading haproxy haproxy [WARNING] 155/201028 (3442) : config : 'stats' statement ignored for proxy 'ssl-relay' as it requires HTTP mode. [ OK ] On May 30, 2012, at 12:03 PM, Miah Johnson wrote: Cord, Yes! You can use 'stats socket' or 'stats uri', and a small program that reads that to monitor your haproxy system. From there you can gather many statistics about your haproxy server, including but not limited to server status (up,down,etc), number of failed checks, and downtime. -- Miah Johnson On Wednesday, May 30, 2012 at 11:36 AM, Cord MacLeod wrote: List, I'm wondering, is there a hook or something of this nature I can call to find if/what are the down machines in my config file? Essentially the issue I'm attempting to solve is to automatically remove servers that haproxy believes are down. These machines are all SSL, so the ssl-hello-chk is the only way to tell if these machines are alive. Has anyone run into this?
Re: reqadd srcIP for header
Oh hey guys, I do the same thing and use the stunnel proxy protocol. You just add protocol = proxy to stunnel and use bind :81 accept-proxy for haproxy and it seems to take care of the rest. I do have to add some extra headers using addreq to identify port and protocol but that's just hardcoded in the https frontend (and http frontend for consistency). Hope that helps. I know newer versions of haproxy and stunnel are required so double check that. -- Chris LeBlanc On Tuesday, June 5, 2012 at 2:01 AM, Baptiste wrote: well, not adding a header, but getting connected on the server using the client IP (using a TPROXY patched kernel). cheers On Tue, Jun 5, 2012 at 9:00 AM, Baptiste bed...@gmail.com (mailto:bed...@gmail.com) wrote: Hey, Using stunnel and haproxy, both with the proxy protocol may help when you want to add the X-SRC-IP header for a SSL connection. cheers
Webinar on: Privacy Implications in Global Legal Outsourcing Industry: Ethical and Regulatory Compliances
Dear Reader, I invite you to attend GOAL’s upcoming webinar on Topic: Privacy Implications in Global Legal Outsourcing Industry: Ethical and Regulatory Compliances Speakers: - Daniel H. Vaswani, Chief Counsel, Kagu9, Inc., USA - James E. King, Attorney, King Law Corporation, USA - Dhiraj Phukan, Director - Events, Global Outsourcing Association of Lawyers, USA (Host) Venue: At your desk: On your Laptop/PC or Phone Date: 07 June 2012 Time: 9:00 am PDT/11:00 am CDT/12:00 pm EDT Duration: 60 minutes including Q A Live Participation Podcast: US$ 49 (Free for In-house Counsels Gold Members) Access the Recorded Version (Podcast) only: US$ 49 Attendee Benefits: The purchase price of this event will also give you FREE access to one additional LPO/IP Offshoring Podcasts from our library (same or less price!). Over and above, you will be awarded with the Silver Membership at GOAL absolutely free of cost. For registration, please contact: +1-562-366-4706 or reply to this email. Thanks Brooklyn Carter Executive, Global Outsourcing Association of Lawyers (GOAL) PS: All our web events are freely accessible to our GOLD Members. Please indicate if you wish to become a GOLD Member. If you don’t want to receive emails, please reply to this email with the subject line ‘unsubscribe’.
Re: programatically checking failed machines
On Mon, Jun 04, 2012 at 01:13:31PM -0700, Cord MacLeod wrote: Just got around to attempting this, it appears not to work with SSL. Is there any other option (stunnel breaks a lot and doesn't support 1 off connections)? * Reloading haproxy haproxy [WARNING] 155/201028 (3442) : config : 'stats' statement ignored for proxy 'ssl-relay' as it requires HTTP mode. That's expected since your SSL instance works in TCP mode. The stats uri statement must be in an HTTP instance (use a dedicated port as most people do, it looks like a waste at first but it's very convenient in the long run). Also the stats socket in the global section allows you much more fun, as you can connect there to get stats, change weights, start/stop servers, kill server connections, etc... Regards, Willy
RE: reqadd srcIP for header
Hey guys, Thank you for all your input, I ended up going with the xforwardedfor. --Karl. Karl Kloppenborg Programming Ninja Crucial Paradigm Pty Ltd Suite 1 Level 3 104-106 Commonwealth St Surry Hills NSW 2010 Australia 1300 884 839 – Sales Support (AU Only) https://support.crucialp.com/ Click here for the Support Desk d: 02 8202 9994 f: 02 92818261 m: 0416 236 908 e: k...@crucial.com.au Crucial Paradigm is a leading Web Hosting provider that specialises in Cloud solutions. We are commited to keeping our planet green, please reduce, reuse and recycle your office paper. Description: crucial_2012 From: Chris LeBlanc [mailto:ch...@blendedby.us] Sent: Tuesday, 5 June 2012 5:07 PM To: Baptiste Cc: Karl Kloppenborg; haproxy@formilux.org Subject: Re: reqadd srcIP for header Oh hey guys, I do the same thing and use the stunnel proxy protocol. You just add protocol = proxy to stunnel and use bind :81 accept-proxy for haproxy and it seems to take care of the rest. I do have to add some extra headers using addreq to identify port and protocol but that's just hardcoded in the https frontend (and http frontend for consistency). Hope that helps. I know newer versions of haproxy and stunnel are required so double check that. -- Chris LeBlanc On Tuesday, June 5, 2012 at 2:01 AM, Baptiste wrote: well, not adding a header, but getting connected on the server using the client IP (using a TPROXY patched kernel). cheers On Tue, Jun 5, 2012 at 9:00 AM, Baptiste bed...@gmail.com wrote: Hey, Using stunnel and haproxy, both with the proxy protocol may help when you want to add the X-SRC-IP header for a SSL connection. cheers image001.jpg
haproxy - varnish - backend server
Hi guys Originally we had haproxy in front and connecting to backend server haproxy - backend server and applications and backend server see the real client ip fine without any issues But we decided to try adding Varnish cache in between haproxy - varnish - backend server Problem now is backend server and ips are seeing the client ip of the haproxy server and not real visitor client ips. varnish has the appropriate forwarding of client ips, remove req.http.X-Forwarded-For; set req.http.X-Forwarded-For = client.ip; and works if Varnish only in front of backends. So what setting if any in haproxy would I need to add or check for, to get the proper client ip from haproxy through varnish into the backend ? Using haproxy v1.3 here with Varnish 3.0.2. thanks --- posted at http://www.serverphorums.com http://www.serverphorums.com/read.php?10,508289,508289#msg-508289
Re: haproxy - varnish - backend server
Is haproxy adding X-Forwarded-For to the request it sends varnish? If so, just don't have varnish manipulate X-Forwarded-For and your app will use the header added by HAProxy. David On 6/5/12 9:04 PM, hapr...@serverphorums.com wrote: Hi guys Originally we had haproxy in front and connecting to backend server haproxy - backend server and applications and backend server see the real client ip fine without any issues But we decided to try adding Varnish cache in between haproxy - varnish - backend server Problem now is backend server and ips are seeing the client ip of the haproxy server and not real visitor client ips. varnish has the appropriate forwarding of client ips, remove req.http.X-Forwarded-For; set req.http.X-Forwarded-For = client.ip; and works if Varnish only in front of backends. So what setting if any in haproxy would I need to add or check for, to get the proper client ip from haproxy through varnish into the backend ? Using haproxy v1.3 here with Varnish 3.0.2. thanks --- posted at http://www.serverphorums.com http://www.serverphorums.com/read.php?10,508289,508289#msg-508289
maxqueue for a backend?
Is there a way to set the maxqueu for an entire backend? I set maxqueu on default-server to 1, and on the frontend ACL required that connslots be greater than 1. I expected connslots to be maxconn * num server + 1 * num servers (connections + queue slots). I found that the backend still queued requests. My ACL could check the current queue size of the backend and route away from that backend if it's gt 0, but I figured connslots was the more correct approach. What am I doing or understanding wrongly?
Re: haproxy - varnish - backend server
why not put varnish in front of haproxy like this: haproxy listen on ip public 1 and localhost varnish listen on ip public 2 and forward to localhost so cached traffic is immediately served by varnish without hitting haproxy. and when you don t need to cache the traffic you use ip public 1 (haproxy) in our setup the varnish ip is s.mydomain.com serving image/css/js and haproxy is www.mydomain.com serving dynamic content containing urls in s.mydomain.com. as a bonus no cookie is sent to s.mydomain.com -- Sebastien E. Le 5 juin 2012 à 21:06, David Coulson da...@davidcoulson.net a écrit : Is haproxy adding X-Forwarded-For to the request it sends varnish? If so, just don't have varnish manipulate X-Forwarded-For and your app will use the header added by HAProxy. David On 6/5/12 9:04 PM, hapr...@serverphorums.com wrote: Hi guys Originally we had haproxy in front and connecting to backend server haproxy - backend server and applications and backend server see the real client ip fine without any issues But we decided to try adding Varnish cache in between haproxy - varnish - backend server Problem now is backend server and ips are seeing the client ip of the haproxy server and not real visitor client ips. varnish has the appropriate forwarding of client ips, remove req.http.X-Forwarded-For; set req.http.X-Forwarded-For = client.ip; and works if Varnish only in front of backends. So what setting if any in haproxy would I need to add or check for, to get the proper client ip from haproxy through varnish into the backend ? Using haproxy v1.3 here with Varnish 3.0.2. thanks --- posted at http://www.serverphorums.com http://www.serverphorums.com/read.php?10,508289,508289#msg-508289
Sporadic latency issues
Hey all, Is anyone aware of latency issues with 1.5dev7? I have checked that syncookies is enabled and syn_backlog = 1. My maxconn is also set to 1. This is Ubuntu Lucid installed from https://launchpad.net/~mojocode/+archive/ppa/+build/2947499 My ulimit is something like 2. Ideas? -- Chris LeBlanc If you see this rotate your iPhone 90°
Re: maxqueue for a backend?
Hi David, On Tue, Jun 05, 2012 at 06:17:15PM -0700, David Birdsong wrote: Is there a way to set the maxqueu for an entire backend? No. When I developped the server maxqueue, I remember having done the same for the backend (it even ended up in the doc, causing later confusion), but I removed it during tests because I noticed it was totally useless the way it worked. Initially the goal was to limit the queue in the backend, not counting the servers, which did not make much sense since it depends whether the clients have a cookie or not. Then I figured that we'd have to count all the queues of all servers, and I did not want to do that for each connection since it's always expensive for just a few use cases. So I removed it before ever releasing it. I set maxqueu on default-server to 1, and on the frontend ACL required that connslots be greater than 1. I expected connslots to be maxconn * num server + 1 * num servers (connections + queue slots). I found that the backend still queued requests. My ACL could check the current queue size of the backend and route away from that backend if it's gt 0, but I figured connslots was the more correct approach. What am I doing or understanding wrongly? You're not necessarily doing anything wrong, there are a wide number of reasons for deciding not to queue in a specific way, and that's the first thing to define. If you have maxconn on servers, you need some queues. The server queues are only for requests which target a specific server. If the request has no cookie set, it will remain in the backend queue so that any server may pick it. So as you can see, you *need* to support queues both in the backend and servers whenever you use maxconn. If you would prevent the backend from queueing any connections, you would constantly emit 503 to some visitors, because traffic is never perfectly smooth (which is observed by your backend queue inflating). The best way I found to manage queues is the avg_queue ACL. It returns the average number of queued connections per active server. This means that wherever the connections are queued, they're accounted for, and divided by the number of servers. The result is an average queue length for the backend, which translates in an average wait time for visitors. I think it's the most efficient way of taking decisions based on queue length because you don't need to consider any difference between new and past visitors, and you don't care whether a server has a deeper queue than another one. This way you can safely remove the maxqueue parameter from your servers and only decide to switch to another backend based on the avg_queue. BTW, what I'd suggest if you use this is to split the traffic depending on the persistence cookie : stop sending new visitors to a backend which has too many queued connections, but still send existing ones there. You will see that the load on the servers will then regulate itself very well and become extremely smooth. And that way you can finely control your response times. In short this looks like this : frontend pub acl bk_full avg_queue(bk_www) gt 10 acl has_cook cook_len(SRV) gt 0 use_backend bk_sorry if bk_full !has_cook default_backend bk_www backend bk_www cookie SRV insert indirect nocache server s1 1.1.1.1:80 cookie 1 maxconn 50 ... backend bk_sorry server sorry 127.0.0.1:8080 Regards, Willy
Re: Sporadic latency issues
Hi Chris, On Tue, Jun 05, 2012 at 09:54:17PM -0500, Chris LeBlanc wrote: Hey all, Is anyone aware of latency issues with 1.5dev7? Not particularly, how/when do you observe them ? On session establishment, to haproxy, to backend servers, during data transfers ? What's the order of magnitude ? 1ms, 200ms, 3s ? Willy
Re: haproxy - varnish - backend server
On Wed, Jun 6, 2012 at 3:50 AM, David Coulson da...@davidcoulson.net wrote: you might also want to look at having varnish prepend the haproxy IP to the X-Forwarded-For line, rather than just pass it along - Most proxies seem to comma delimit a list of IPs of all the devices the request has passed through. Hey David, Having a single line with values separated by commas or having multiple lines, one per value are both ways allowed by RFC to write headers in HTTP. So any HTTP processing tool should make no difference whatever way headers are written. Saying that, it is usually recommended to chain proxies like this: Varnish - HAProxy - server. Now, if you can't and must let Varnish behind HAProxy, it should be not complicated to do the X-Forwarded-For clean up (untill Varnish can understand proxy-protocol). cheers
