First of all, thanks so much to everyone for the responses so far! It's been really helpful, and I'd love to hear what anyone else thinks.
>>>Have you considered using Haproxy with Direct Server Return to work >>>around the routing issue? >> >>You cannot use HAProxy with "Direct Server Return", because HAProxy >>terminates the TCP connection on both sides. I actually hadn't considered using this, but now it sounds like it's not even possible. But if it was, it sounds like it would leave a lot of connections open until timeout, compounding my high connection issue. >Well, you can also use a couple of LVS clusters in DSR mode: >between clients and HAProxy, DSR mode, source hash load-balancing. >Between HAProxy and servers, DSR mode, destination hash load-balancing. >and you're done :) > >Your architecture becomes scallable and you avoid this dirty RRDNS trick! As much as I like the promise of scalability, that honestly sounds more complicated than I'd like it to be. >>> Now the customer has added the requirement that the requests look like >>> they came from the client ‹ i.e. transparent proxying. >> >> If you have all those requirements, you need "Direct Server Return", as >> Vivek mentioned. This is probably how the Foundry solution works. >> >> I suspect you can do this (Direct Return) only with LVS. Anyway, I >>wouldn't >> recommend such a solution. Things like SYN floods will directly hit your >> backends. Good to know, thanks for the note about SYN floods. (Since the Foundry is an appliance, I assume it just has enough hardware to handle a million connections or whatever it's rated at.) >>> I've asked if the customer can use the X-Forwarded-For header instead, >>>but >>> I'm not expecting them to agree to that. >> >> Keep in mind that you cannot insert any HTTP headers on the proxy when >>the >> traffic is passing SSL/TLS encrypted. You would have to offload SSL on >>the >> load balancer to be able to insert the IP in the header. Oy, I thought of that when I was thinking about cookie-based persistence, but totally spaced on that fact when considering X-Forwarded-For! Thanks for the reminder. >>>The best option I can come up with is to break the backends into groups >>>and >>> assign each group to an haproxy, which would then be the backends' >>>default >>> gateway. But then my backends are silo'd and I'm really relying on >>> round-robin DNS to keep things even, which I don't consider reliable. >> >> Thats correct. >> >> Consider that reverse proxying in TCP mode is extremely cheap in terms >>of >> CPU and memory. With decent hardware (10G NIC, lots of RAM and a beefy >>CPU) >> you will be able to forward 20Gbps on a single box. >> >> Depending on your load you may be able to do everything with a single >> active/standby proxy "group". This is a really good reminder. You've got me thinking that maybe I'm going about this wrong and should focus on using a single instance, with a corosync/pacemaker standby. I hadn't done the calculations before, assuming I'd need more than one instance, but now I see that about 7.5GB of RAM should handle 300,000 connections, based on some load testing we've done before. I can spin an instance with several times that, so that should work. I guess I had some http load balancing stuck in my head, because we got much lower numbers for that, as expected. >>> Persistence is a bonus (to reduce SSL negotiations) but not absolutely >>> required >> >> You probably want to make this a hard requirement, as those SSL >>negotiations >> will have a huge performance impact. OK thanks. Good to know how important you consider it. I wasn't really clear; what I meant was that if there is a temporary break in persistence -- specifically from a server being added/removed -- it's not the end of the world. Donovan
