So, it turns out that I am currently interviewing to find a new employer. I will probably not pick this back up again; I will not be able to see if this works in the “broken” environment.
Thanks for the help! -Timothy On Tue, Nov 7, 2017 at 12:38 PM Andrew Smalley <asmal...@loadbalancer.org> wrote: > Hello Timothy > > Aaron has told me that you are in AWS and this being the case I hope > ipip does indeed work with my hints. > > It is worth noting that I know in AWS we can not do Layer4 DR however > with the help of OpenVSwitch I know 100% that its possible to do > Layer4 DR to any real server connected with either a VXLAN or GRE > tunnel . > > I have personally done this in my private service and stretched a > VXLAN from AWS to my private lab and other remote datacenters where my > real servers reside. > > > > > Andruw Smalley > > Loadbalancer.org Ltd. > > www.loadbalancer.org > +1 888 867 9504 / +44 (0)330 380 1064 > asmal...@loadbalancer.org > > Leave a Review | Deployment Guides | Blog > > > On 7 November 2017 at 16:51, Timothy R. Weiand <timothy.wei...@gmail.com> > wrote: > > Ooohhhh!! Let me try this. > > > > Thanks for the directions, > > -Timothy > > On Mon, Nov 6, 2017 at 1:55 PM Andrew Smalley <asmal...@loadbalancer.org > > > > wrote: > > > >> Hi Timothy, > >> > >> I've been testing ipip (Tun mode) and I have it working, > >> > >> I note that the Weak ES Model needs to be accounted for and the tunl0 > >> should be a /32 not a /24 as you have it on the real server. > >> > >> ip addr add 192.168.200.11/24 brd 192.168.200.11 dev tunl1 > >> > >> becomes > >> > >> ip addr add dev tunl1 192.168.200.11/32 brd 192.168.200.11 dev tunl1 > >> > >> > >> I also have this in my /etc/sysctl.conf > >> > >> net.ipv4.conf.all.arp_ignore=1 > >> net.ipv4.conf.eth0.arp_ignore=1 > >> net.ipv4.conf.tunl1.arp_ignore=1 > >> net.ipv4.conf.all.arp_announce=2 > >> net.ipv4.conf.eth0.arp_announce=2 > >> net.ipv4.conf.tunl1.arp_announce=2 > >> > >> net.ipv4.conf.all.log_martians=1 > >> net.ipv4.conf.default.log_martians=1 > >> > >> And I ran this script portion to enable forwarding and disable the > >> rp_filter > >> > >> if [ $1 = "tunl1" ]; then > >> echo 1 > /proc/sys/net/ipv4/conf/tunl0/forwarding > >> echo 0 > /proc/sys/net/ipv4/conf/tunl0/rp_filter > >> fi > >> > >> > >> This is with a 4.4.49 Kernel > >> > >> I hope this help you > >> > >> > >> Andruw Smalley > >> > >> Loadbalancer.org Ltd. > >> > >> www.loadbalancer.org > >> +1 888 867 9504 / +44 (0)330 380 1064 > >> asmal...@loadbalancer.org > >> > >> Leave a Review | Deployment Guides | Blog > >> > >> > >> On 22 August 2017 at 12:43, Timothy R. Weiand <timothy.wei...@gmail.com > > > >> wrote: > >> > > >> > Aaron: > >> > > >> > I was able to get LVS-TUN working in between subnets in AWS if the > real > >> > server is a 3.x kernel (RHEL to be specific). I had to set the > >> destination > >> > side of the tunnel and the address on that interface to the VIP. > Then, I > >> > configured unbound to listen for the VIP address. > >> > > >> > We are working with RedHat to get the 4.x kernels fixed. But, no > >> timeframe > >> > on that fix. > >> > > >> > FYI - We are going to work towards open sourcing our work so that > >> everyone > >> > can see exactly what we have done. > >> > > >> > -Timothy > >> > > >> > On Thu, Aug 17, 2017 at 10:56 AM Aaron West <aa...@loadbalancer.org> > >> wrote: > >> > > >> > > Hi Timothy, > >> > > > >> > > What you had working was it in AWS or VMWare? > >> > > > >> > > I've pretty much given up trying to get the spoofed reply traffic > back > >> > > out of AWS it gets dropped always with either DR mode or TUN unless > I > >> > > send it back via the director and that case I may as well do NAT > mode > >> > > instead... > >> > > > >> > > My understanding is that when egress traffic hits their edge they > >> > > check the source matches the instance it came from and if it doesn't > >> > > the packet gets dropped. However, all works fine if the client is in > >> > > the same VPC so it doesn't need to go out past the edge of the VPC. > >> > > > >> > > Just interested if you found the same in your testing? > >> > > Aaron West > >> > > > >> > > Loadbalancer.org > >> > > > >> > > www.loadbalancer.org > >> > > +1 888 867 9504 / +44 (0)330 380 1064 > >> > > aa...@loadbalancer.org > >> > > > >> > > LEAVE A REVIEW | DEPLOYMENT GUIDES | BLOG > >> > > > >> > > > >> > > On 10 August 2017 at 18:08, Timothy R. Weiand < > >> timothy.wei...@gmail.com> > >> > > wrote: > >> > > > Julian: > >> > > > > >> > > > Good news and bad news: this guidance helped me get LVS-TUN > working > >> with > >> > > a > >> > > > 3.10.0 kernel. But, I am not able to use the same configuration to > >> get > >> > > > 4.4.x,4.8.x or 4.12.x working. We are working on doing further > >> analysis > >> > > to > >> > > > ensure it is kernel dependent and not distribution dependent. I > hope > >> to > >> > > > find the kernel version that this configuration stops working. > >> > > > > >> > > > Note on 4.x machines: If we do not have the correct ip assigned to > >> the > >> > > > tunnel interface we can see traffic on that interface. But, if we > >> use the > >> > > > correct VIP then we no longer see the traffic. > >> > > > > >> > > > I will keep you updated as I find information out. > >> > > > > >> > > > Thanks! > >> > > > -Timothy > >> > > > > >> > > > On Tue, Jul 25, 2017 at 4:25 PM Julian Anastasov <j...@ssi.bg> > wrote: > >> > > > > >> > > >> > >> > > >> Hello, > >> > > >> > >> > > >> On Fri, 14 Jul 2017, Timothy R. Weiand wrote: > >> > > >> > >> > > >> > I am building a DR-TUN configuration to load balance DNS > traffic. > >> My > >> > > >> issue > >> > > >> > is with the real servers: the ipip packet get unwrapped and > >> appears on > >> > > >> the > >> > > >> > tunnel interface and becomes a martian. Also, I have not been > >> able to > >> > > >> > determine how to bind to the tunnel interface to capture these > >> > > packets. I > >> > > >> > have verified details with tcpdump/dmesg. All my configuration > is > >> > > >> scripted > >> > > >> > to ensure reproducibility. tcpdumps below do not show real DNS > >> > > traffic; > >> > > >> > using netcat to send text. > >> > > >> > > >> > > >> > Much more detail can be supplied, please request it. > >> > > >> > > >> > > >> > Configuration: > >> > > >> > > >> > > >> > ============== > >> > > >> > > >> > > >> > - All machines are currently Debian 9 (4.9.30-2+deb9u2) > >> > > >> > > >> > > >> > - Client: 192.168.200.10 > >> > > >> > > >> > > >> > - Director: 192.168.200.11 > >> > > >> > > >> > > >> > - Real Server: 192.168.200.12 > >> > > >> > > >> > > >> > - No VIP: I am using the IP address for my director interface - > >> > > >> > 192.168.200.11 > >> > > >> > > >> > > >> > > >> > > >> > Director Configuration: > >> > > >> > > >> > > >> > ================== > >> > > >> > > >> > > >> > vagrant@debian-9-lb:~$ sudo ipvsadm -Ln > >> > > >> > > >> > > >> > IP Virtual Server version 1.2.1 (size=4096) > >> > > >> > > >> > > >> > Prot LocalAddress:Port Scheduler Flags > >> > > >> > > >> > > >> > -> RemoteAddress:Port Forward Weight ActiveConn > >> InActConn > >> > > >> > > >> > > >> > UDP 192.168.200.11:53 rr > >> > > >> > > >> > > >> > -> 192.168.200.12:53 Tunnel 1 0 0 > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > Real Server: > >> > > >> > > >> > > >> > ============ > >> > > >> > > >> > > >> > # modprobe ipip > >> > > >> > > >> > > >> > # echo 1 > /proc/sys/net/ipv4/ip_forward > >> > > >> > > >> > > >> > # ip tunnel add tunl1 mode ipip ttl 32 local 192.168.200.12 > remote > >> > > >> > 192.168.200.11 > >> > > >> > > >> > > >> > # ip link set tunl1 up arp off > >> > > >> > > >> > > >> > # echo 2 > /proc/sys/net/ipv4/conf/all/rp_filter > >> > > >> > > >> > > >> > # echo 2 > /proc/sys/net/ipv4/conf/default/rp_filter > >> > > >> > > >> > > >> > # echo 2 > /proc/sys/net/ipv4/conf/tunl0/rp_filter > >> > > >> > > >> > > >> > # echo 2 > /proc/sys/net/ipv4/conf/tunl1/rp_filter > >> > > >> > >> > > >> tunl1/rp_filter=2 is ignored if there are no > >> > > >> IPv4 addresses configured on tunl1, you can see > >> > > >> __fib_validate_source() for reference: > >> > > >> > >> > > >> if (no_addr) > >> > > >> goto last_resort; > >> > > >> > >> > > >> As result, packets are dropped as martians due to rpf=2. > >> > > >> > >> > > >> To check if packets are dropped by routing try: > >> > > >> > >> > > >> ip route get from 192.168.200.10 to 192.168.200.11 iif tunl1 > >> > > >> > >> > > >> Here is small howto for TUN (outdated): > >> > > >> > >> > > >> http://ja.ssi.bg/TUN-HOWTO.txt > >> > > >> > >> > > >> > vagrant@debian-9-dns:~$ sudo tcpdump -e -n -s 0 -i tunl1 -vv > >> > > >> > > >> > > >> > tcpdump: listening on tunl1, link-type RAW (Raw IP), capture > size > >> > > 262144 > >> > > >> > bytes > >> > > >> > > >> > > >> > 10:16:13.919646 ip: (tos 0x0, ttl 64, id 63545, offset 0, flags > >> [DF], > >> > > >> proto > >> > > >> > UDP (17), length 40) > >> > > >> > > >> > > >> > 192.168.200.10.51149 > 192.168.200.11.53: [udp sum ok] > 26226 > >> > > >> updateMA+ > >> > > >> > [b2&3=0x6f6d] [27753a] [11619q] [25966n] [29706au][|domain] > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > If I add an IP address to the interface the packet never > reaches > >> tunl1 > >> > > >> > > >> > > >> > # ip addr add 192.168.200.11/24 brd 192.168.200.11 dev tunl1 > >> > > >> > >> > > >> Try with (note mask /32): > >> > > >> > >> > > >> ip addr add 192.168.200.11/32 dev tunl1 > >> > > >> > >> > > >> as shown in LVS HOWTO (8.3): > >> > > >> > >> > > >> > http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.LVS-Tun.html > >> > > >> > >> > > >> > Notes: > >> > > >> > > >> > > >> > ====== > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > LVS-DR was not an option because the real servers will be in > >> different > >> > > >> > subnets (I was able to get LVS-DR working though). > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > I am looking for advice on how to better understand or resolve > >> this > >> > > >> issue. > >> > > >> > Or, who would be better to answer this question. > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > I have groked as many articles on how to setup this > configuration > >> as I > >> > > >> can. > >> > > >> > And, I have read a good deal of the archives of this mailing > list. > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > Linux distributions I have tried are Debian, Ubuntu and Amazon > >> linux. > >> > > >> This > >> > > >> > is been tested on both AWS and VMware fusion. > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > ------ > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > I have a feeling I am missing something simple... > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > Thanks! > >> > > >> > > >> > > >> > -Timothy > >> > > >> > >> > > >> Regards > >> > > >> > >> > > >> -- > >> > > >> Julian Anastasov <j...@ssi.bg> > >> > > >> > >> > > > _______________________________________________ > >> > > > Please read the documentation before posting - it's available at: > >> > > > http://www.linuxvirtualserver.org/ > >> > > > > >> > > > LinuxVirtualServer.org mailing list - > >> lvs-users@LinuxVirtualServer.org > >> > > > Send requests to lvs-users-requ...@linuxvirtualserver.org > >> > > > or go to http://lists.graemef.net/mailman/listinfo/lvs-users > >> > > > >> > > _______________________________________________ > >> > > Please read the documentation before posting - it's available at: > >> > > http://www.linuxvirtualserver.org/ > >> > > > >> > > LinuxVirtualServer.org mailing list - > lvs-users@LinuxVirtualServer.org > >> > > Send requests to lvs-users-requ...@linuxvirtualserver.org > >> > > or go to http://lists.graemef.net/mailman/listinfo/lvs-users > >> > > > >> > _______________________________________________ > >> > Please read the documentation before posting - it's available at: > >> > http://www.linuxvirtualserver.org/ > >> > > >> > LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org > >> > Send requests to lvs-users-requ...@linuxvirtualserver.org > >> > or go to http://lists.graemef.net/mailman/listinfo/lvs-users > >> > >> _______________________________________________ > >> Please read the documentation before posting - it's available at: > >> http://www.linuxvirtualserver.org/ > >> > >> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org > >> Send requests to lvs-users-requ...@linuxvirtualserver.org > >> or go to http://lists.graemef.net/mailman/listinfo/lvs-users > >> > > _______________________________________________ > > Please read the documentation before posting - it's available at: > > http://www.linuxvirtualserver.org/ > > > > LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org > > Send requests to lvs-users-requ...@linuxvirtualserver.org > > or go to http://lists.graemef.net/mailman/listinfo/lvs-users > > _______________________________________________ > Please read the documentation before posting - it's available at: > http://www.linuxvirtualserver.org/ > > LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org > Send requests to lvs-users-requ...@linuxvirtualserver.org > or go to http://lists.graemef.net/mailman/listinfo/lvs-users > _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users
