On Mon, Dec 20, 2010 at 09:43:35PM +0100, Patrick Schaaf wrote: > Is the following known / does a solution exist? > > I'm setting up two machines with kernel 2.6.36.2 as master/backup ipvs > directors, with keepalived checking real servers and implementing vrrp > failover. > > Virtual service is for HTTP connections, using NAT method towards the > real servers. > > The basic setup has been working fine, with an exemplary set of three > virtual IPs balancing to some real servers, replicating connection state > (ipvsadm -ln counters increasing on the backup, -lc state visible > there). > > However, for the production setup, I have to implement roughly 200 > different virtual IP addresses, all running onto the same (rather small) > set of real servers. > > As is well known, doing that with the corresponding number of different > ipvs virtual services presents problems, as the real server state > (connection count) is kept for each individual virtual service, > resulting in suboptimal balancing. > > As a solution to that, I have been testing two different approaches: > > 1) using fwmark, with --set-mark in the mangle table to mark the > incoming packets for the different virtual IPs, and an fwmark virtual > service set up as usual. > iptables -t mangle -A PREROUTING -m ... -j MARK --set-mark 80 > ipvsadm -A -f 80 ... > > and alternatively > > 2) using iptables DNAT in PREROUTING to rewrite the various virtual IPs > to specific (few) virtual IPs set up as ipvs services. > iptables -t nat -A PREROUTING -m ... -j DNAT --to-dest 10.0.0.1 > ipvsadm -A -t 10.0.0.1:80 ... > > Both approaches work fine WRT balancing, reaching the real servers, and > everything. > > BUT: no connection state is synchronized, in either of the approaches. > The backup server does not show -ln counter increase, nor -lc > connections, when I test it. > > I have even set up the fully working (normal) approach at the same time > as as 1) and/or 2), for different addresses, and the sync-to-backup is > working OK for the normal addresses, but not sending connection state > for stuff covered by approaches 1) or 2). > > Any suggestions as to why this happens? Patches to apply? Good chance > 2.6.37-rcX could work? More info needed?
Hi Patrick, while there are a number of limitations in the synchronisation code I believe that what you are trying to do should work. On the backup do you see connections showing up in the output of ipvsadm -Lcn ? On the topic of connection synchronisation, a new synchronisation protocol (v1) has been developed which addresses a number of the problems in the existing code (v0). This includes the explicit synchronisation of fwmarks. The code is currently available in the lvs-test-2.6 tree on git.kernel.org. It should appear in 2.6.38-rc1. _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users
