Hello! I try to do load-balance the service which is listening both TCP and UDP using same port number *1111*. There are set of real servers, each of them host few application instances listening on different ports (1111/udp, 1116/udp, 1115/tcp, 1117/tcp) I used fwmark to group connection by protocol, i.e. 1111/udp have *fwmark 1*, 1111/tcp have *fwmark 2.* LVS-NAT mode is in use. What I see is when the client connection coming to 1111/*TCP* the LVS try to pass it to RTI:1111/*TCP *while the RIP have TCP listeners on ports 1115 and 1117 and LVS was configured accordingly.
Here is test connection info: # ipvsadm -L -n -c IPVS connection entries pro expire state source virtual destination *TCP* 00:55 SYN_RECV 113.25.79.34:38204 115.5.6.138*:1111 *10.1.2.13 *:1111* *<<<< It should be RIP:1115 or RIP:1117 HERE!* IP 00:49 NONE 113.25.79.34:0 0.0.0.2:0 10.1.2.13:0 TCP 01:00 SYN_RECV 113.25.79.34:38206 115.5.6.138:1111 10.1.2.13:1111 Below are saved firewall and ipvs configurations: # Generated by iptables-save v1.4.21 on Tue Dec 11 17:02:29 2018 *mangle :PREROUTING ACCEPT [9507:869411] :INPUT ACCEPT [7637:700414] :FORWARD ACCEPT [70:10945] :OUTPUT ACCEPT [8781:1045717] :POSTROUTING ACCEPT [8851:1056662] COMMIT # Completed on Tue Dec 11 17:02:29 2018 # Generated by iptables-save v1.4.21 on Tue Dec 11 17:02:29 2018 *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [82:11111] :OUTPUT ACCEPT [16923:2797363] -A INPUT -p icmp -m comment --comment "000 accept all icmp" -j ACCEPT -A INPUT -i lo -m comment --comment "001 accept all to lo interface" -j ACCEPT -A INPUT -d 127.0.0.0/8 ! -i lo -m comment --comment "002 reject local traffic not on loopback interface" -j REJECT --reject-with icmp-port-unreachable -A INPUT -m comment --comment "003 accept related established rules" -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p tcp -m multiport --dports 22 -m comment --comment "100 accept ssh connection" -m state --state NEW -j ACCEPT -A INPUT -d 224.0.0.0/8 -p vrrp -m comment --comment "104 Accept VRRP proto" -j ACCEPT -A INPUT -m comment --comment "999 drop all" -j REJECT --reject-with icmp-port-unreachable COMMIT # Completed on Tue Dec 11 17:02:29 2018 # Generated by iptables-save v1.4.21 on Tue Dec 11 17:02:29 2018 *nat :PREROUTING ACCEPT [42:2928] :INPUT ACCEPT [0:0] :OUTPUT ACCEPT [4:360] :POSTROUTING ACCEPT [4:360] -A PREROUTING -p tcp -m multiport --dports 1111 -j MARK --set-xmark 0x2/0xffffffff -A PREROUTING -p *udp* -m multiport --dports 1111 -m comment --comment "101 MARK udp 1111 traffic from everywhere" -j MARK --set-xmark *0x1*/0xffffffff -A PREROUTING -p *tcp* -m multiport --dports 1111 -m comment --comment "101 MARK tcp 1111 traffic from everywhere" -j MARK --set-xmark *0x2*/0xffffffff COMMIT # Completed on Tue Dec 11 17:02:29 2018 # Generated by iptables-save v1.4.21 on Tue Dec 11 17:02:29 2018 *raw :PREROUTING ACCEPT [17569:3914462] :OUTPUT ACCEPT [16925:2798776] COMMIT # ipvsadm -S -A -f 1 -s wrr -p 50 -a -f 1 -r 10.1.2.12:1111 -m -w 2 -a -f 1 -r 10.1.2.12:1116 -m -w 2 -a -f 1 -r 10.1.2.13:1111 -m -w 2 -a -f 1 -r 10.1.2.13:1116 -m -w 2 -a -f 1 -r 10.1.2.14:1111 -m -w 2 -a -f 1 -r 10.1.2.14:1116 -m -w 2 -A -f 2 -s wrr -p 50 -a -f 2 -r 10.1.2.12:1115 -m -w 2 -a -f 2 -r 10.1.2.12:1117 -m -w 2 -a -f 2 -r 10.1.2.13:1115 -m -w 2 -a -f 2 -r 10.1.2.13:1117 -m -w 2 -a -f 2 -r 10.1.2.14:1115 -m -w 2 -a -f 2 -r 10.1.2.14:1117 -m -w 2 # ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn FWM 1 wrr persistent 50 -> 10.1.2.12:1111 Masq 2 0 0 -> 10.1.2.12:1116 Masq 2 0 0 -> 10.1.2.13:1111 Masq 2 0 0 -> 10.1.2.13:1116 Masq 2 0 0 -> 10.1.2.14:1111 Masq 2 0 0 -> 10.1.2.14:1116 Masq 2 0 0 FWM 2 wrr persistent 50 -> 10.1.2.12:1115 Masq 2 0 0 -> 10.1.2.12:1117 Masq 2 0 0 -> 10.1.2.13:1115 Masq 2 0 0 -> 10.1.2.13:1117 Masq 2 0 1 -> 10.1.2.14:1115 Masq 2 0 0 -> 10.1.2.14:1117 Masq 2 0 0 OS is CentOS 7 with kernel *3.10* Thank you. -- Bogdan Rudas Director of IT Europe Exadel Inc. http://www.exadel.com/ E-mail: bru...@exadel.com Skype ID: bogdan.rudas -- CONFIDENTIALITY NOTICE: This email and files attached to it are confidential. If you are not the intended recipient you are hereby notified that using, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error please notify the sender and delete this email. _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users
