Gary Smith wrote: >> All, >> >> >> Lets say hypothetically I have a director with two vips. The vips >> represent different services, different areas of responsibility, etc. (snipped explanation of two realservers communicating via their VIPs) > Use different subnets for the different classes of real servers. > > Ex: > > Data rail: 10.0.1.0/24 > Web rail: 10.0.2.0/24 > > Data server: > * IP 10.0.1.2/24 > > Web server: > * IP 10.0.2.2/24 > > Director: > * IP 10.0.1.1/24 > * IP 10.0.2.1/24 > * VIP 10.0.3.10/24 Data > * VIP 10.0.3.11/24 Web (or the public IP if ipvs is the firewall as > well) > > ipvsadm -A t 10.0.3.10:3306 -s wlc > ipvsadm -a t 10.0.3.10:3306 -r 10.0.1.2:3306 -m -we 100 > ipvsadm -A t 10.0.3.11:80 -s wlc > ipvsadm -a t 10.0.3.11:80 -r 10.0.2.2:80 -m -we 100 > > > no need to nat/snat at this point. > > > >> So, does my question make sense? I would like realservers for one vip >> to make connections to the vip of another virtual server on the same >> director. Anyone know how? >>
Ok - so I've had a look at this solution, and tried something like this. It may be that the solution could be made to work but it depends on having a different internal 'routing' vip per set of servers. For one or three sets this would probably be ok. I'm trying to develop a commercial offering based on lvs in which different customers are assigned one or more realservers behind the director. I can't predict ahead of time what customers will buy, and I can't renumber on a whim (though to some extent the private IPs behind the NAT are somewhat hidden...). The realservers are actually virtual servers on fairly beefy hardware and I can end up with many many (virtual) realservers and many 10s of vips. It turns out the performance is acceptable for such a solution according to the load tests we've done, but this vip-to-vip communication is somewhat of a deal-breaker for us. I've got a few ugly workarounds, but nothing I'd like to put real services on. Since I dont have direct control over the customer's use of the services, and the customers may want to communicate with each other and may not even be aware that whoever they're sending email to, etc are on another one of our realservers they rightly expect that they should just be able to reach the IP of someone else on the internet... The problem appears to be outbound LVS routing. It doesn't follow the normal iptables paths as far as I can tell. Let me see if I can give you some more detail: I have two real servers A and B. A has a VIP of 200.0.0.2 and a private IP of 10.0.0.2, B has 200.0.0.3 and 10.0.0.3. The director has a private vip of 10.0.0.1 used for outbound routing. If A tries to connect to B, packets arrive at the directory with a source IP of 10.0.0.2 and a dest IP of 200.0.0.3. LVS changes that destination to 10.0.0.3. At this point using symmetric dnat/snat rules, then iptables fires the snat/postrouting rules and changes the source IP, but LVS doesn't dot this. The packet is then dropped back out on the wire on the private interface (vlan in this case...). B gets a packet addressed to 10.0.0.3 from 10.0.0.2 (which should be 200.0.0.2), responds, and A sees the response hit the wire because its on the same lan, the director does nothing with the packet because it thinks it doesn't need to. At any rate, I've tried many many configurations and iptables rulesets to try and get this to work right. One thing that might affect this is that I'm using one physical lan and a couple of vlans for this configuration. If I used two physical interfaces (one for public, one for private) perhaps this would work. In short, do you have any other suggestions I might try? Fred This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you. _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users
