Sorry, my replies didn't go to the list. Here's the latest post. On Fri, Feb 4, 2011 at 12:24 PM, sven _ <svens...@gmail.com> wrote: > On Fri, Feb 4, 2011 at 9:47 AM, Julian Anastasov <j...@ssi.bg> wrote: >> On Fri, 4 Feb 2011, sven _ wrote: >>> I only have logging rules in mangle-POSTROUTING, so the packet >>> continues. After that chain, it should enter nat-POSTROUTING (where >>> I have logging rules and then the SNAT rule), but that chain is >>> bypassed. tcpdump on the realserver shows that there's no source >>> NAT happening. >> >> Then xt_ipvs does not match the packet. What is the >> used kernel and do you have CONFIG_IP_VS_NFCT=y ? In >> 2.6.37+ do you have /proc/sys/net/ipv4/vs/conntrack set to 1? > > I run 2.6.37 trunk (Debian-packaged), where I have > CONFIG_IP_VS_NFCT=y > > I did not have net.ipv4.vs.conntrack set, but I have set it now. The result > is still the same, though: Packets don't seem to traverse nat-POSTROUTING > at all on their way out towards the realserver. > >>> Where is it determined whether or not the LVS-constructed packet >>> should traverse this and that netfilter chain? >> >> In 2.6.37 if /proc/sys/net/ipv4/vs/conntrack is >> not enabled all IPVS packets are marked as untracked by >> ip_vs_notrack() and NAT can not match them. If conntrack=1 >> then you can try to add debugging into net/netfilter/xt_ipvs.c >> ipvs_mt() to see which branch sets match = false. > > Is it correct that as long as xt_ipvs is loaded, then I should see that the > nat-POSTROUTING chain is traversed? Or is a combination of xt_ipvs, > net.ipv4.vs.conntrack=1, and perhaps something else needed? > > Here are some details on my setup: > > *Two-node setup with both LVS and realserver on each. > > *mangle-PREROUTING set up to MARK packets with dst=virtual_ip > and mac-source != mac_of_other_node. > > *keepalived config to set up ipvs with fwmark balancing, lb_kind=NAT. > > *iptables log rules in all chains: > for t in raw mangle nat filter; do > for c in PREROUTING INPUT FORWARD OUTPUT POSTROUTING; do > iptables -t $t -I $c -p tcp --dport 80 -j LOG --log-prefix "$t-$c-dport: " > iptables -t $t -I $c -p tcp --sport 80 -j LOG --log-prefix "$t-$c-sport: " > done > done > > *iptables nat-POSTROUTING rule with ipvs match: > iptables -t nat -A POSTROUTING -o bond0.308 -m ipvs --vaddr virtual_ip > -p tcp --dport 80 -j SNAT --to real_node_ip > > The reason I'm using LVS-NAT (with SNAT) as opposed to LVS-DR, is > that the nodes are set up with two interfaces, with the same VIP on > both, and they serve clients from the same subnet (yes, it's crazy). > In short: Packets coming in on interface A have to be sent back out > interface A too. Unfortunately iproute's ip rule cannot match on the > netfilter connmark, so the return route can only be based on either > src or dst IP addresses, which are the same on these interfaces. > > A solution would be to SNAT or DNAT the packets on the router, > before they reach the two nodes. Unfortunately, this seems to be > impossible to do on Cisco IOS with my setup, which uses a combination > of crypto-map (which map tunnels into VRFs), VRFs (which map to > subinterfaces) and VLANs (on each subinterface). "ip nat destination > inside" does not have a vrf option, and "ip nat source" applies only > to the egress interface, which is in one VRF common to both VPN > peers (where the clients come from), i.e. you lose the end-to-end > separation of the overlapping client (and destination) subnets. > > So right now, xt_ipvs seems like it can do the trick. If not, adding > another two nodes to the setup would work, but it feels like a defeat. > > sven >
_______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users
