I guess I was under a different impression for OPS. It does exactly what I want it to do which is completely load balance perfectly over my realservers. Its entirely possible in my application that all the requests from a client could come from the same CIP:CPORT. My problem with the timeout is that the lowest I could get the timeout was 1 second. In the telecom world for me, this is an eternity.
Going a step further maybe it is possible to use LVS-DR instead of LVS-NAT. This would negate the need for the director to keep up with anything as the packet would be going directly out from the realserver. I've tried this with the SNAT rule as I've previously reported and had problems SNATing to port 53. My setup is 2 bonds: 1 with 2 vlans, 1 with 1 vlan Bond0.200 (public) Bond0.202 (private) Bond1.201 (public and vlan DNS traffic is used on) So I send my DNS query to my VIP on my directors. It gets routed to a realserver which I've attached the vip to bond1.201:0. According to others I've talked to I shouldn't need an iptables rule but I still don't see the packet out with the source ip address of the VIP. I see the packet with the source IP of the actual realserver. Its possible it is a routing issue though so I plan on digging deeper on that today. Should I need an iptables rule at all for LVS-DR? Thanks for your insight. Mike -----Original Message----- From: Julian Anastasov [mailto:[EMAIL PROTECTED] Sent: Tuesday, April 17, 2007 5:12 PM To: Rudd, Michael Cc: LinuxVirtualServer.org users mailing list. Subject: RE: SNAT / Masquerading problems using LVS-NAT Hello, On Thu, 12 Apr 2007, Rudd, Michael wrote: > After some more digging it appears this is related to the OPS or One > Packet Scheduling feature. With the OPS feature turned off, the source > IP address is correctly SNATed to my VIP. With the OPS feature on and > working correctly(which we need for our UDP service), the source IP > address isn't correctly SNATed. OPS is implemented for setups where there is no reply for the original packet. DNS and RADIUS have reply, so they need something different which is not done by OPS: - every original packet should pass scheduling step - reply (or replies) should go back properly, the hash connection should keep the needed information (VIP, VPORT) So, what you need is something different. OPS now works in this way: - schedule original packet but don't hash the connection - when reply packet is received it can not find connection and the reply packet is treated as non-IPVS packet (real server receives ICMP or packets goes further). That is why such replies don't have proper VIP:VPORT if passed to output device. > Is anybody aware of the code for this? I assume its related to not > looking up the connection in the hash table anymore with OPS thus not > SNATing. Maybe an iptables rule coudl fix this possibly? You can use rule but may be you will not get the right VPORT every time. Not sure why you need OPS. It was created when someone needed to generate many requests from same CIP:CPORT with the assumption that there are no replies. Only when many connections come from same CIP:CPORT in the configured UDP time period the connection reusing does not allow scheduling to be used for every packet. That is why OPS was needed to schedule many packets (coming before expiration) from same CIP:CPORT->VIP:VPORT to different real servers. May be what you need from OPS is impossible: when OPS is not used if reply is delayed IPVS will wait until the configured UDP timeout is expired but this value can be different from the timeout your clients is using. Difference in miliseconds can be fatal. What can happen is that a different request from same CPORT will go to the same real server as long as the UDP timeout is not expired. There can be different situations: - clients can retransmit on some timeout (DNS, RADIUS) - nobody is instructed how many requests should be passed (and the same number of replies if such application mode is used) before removing NAT connection explicitly before expiration to allow next request to be scheduled to different real server. So, the main problem is that it is not easy to balance single CIP to many real servers if there are replies that can be delayed or when requests can be retransmitted. There is no way IPVS to know when to forget one connection to allow scheduling for the next packet from same CIP:CPORT. So, if the client expects replies then OPS should not be used. Instead, short UDP timeout should be used and one should be ready single CIP:CPORT to be scheduled to same real server even if many distinct (from application point of view) requests are sent from same socket. > Update has anyone seen this? I am basically seeing using LVS-NAT that > the return packets are not being SNATed with the LVS directors VIP but > have a source IP address of the realservers. I saw this work in the > 2.4 kernel but havent been able to make it work in 2.6. Any clues? > > Thanks > Mike Regards -- Julian Anastasov <[EMAIL PROTECTED]> _______________________________________________ LinuxVirtualServer.org mailing list - [email protected] Send requests to [EMAIL PROTECTED] or go to http://www.in-addr.de/mailman/listinfo/lvs-users
