-----Original Message----- From: Tom Eastep Sent: Wednesday, April 17, 2019 11:21 PM To: shorewall-users@lists.sourceforge.net Subject: Re: [Shorewall-users] Switching between multi-ISP
On 4/14/19 11:37 PM, Andrei Andreev wrote: > > > -----Original Message----- From: Tom Eastep > Sent: Monday, April 15, 2019 12:46 AM > To: shorewall-users@lists.sourceforge.net > Subject: Re: [Shorewall-users] Switching between multi-ISP > > On 4/14/19 1:30 PM, Andrey Andreev wrote: >> Dear Tom, >> I have been using Shorewall on all company routers, thanks for the great >> work! >> Recently installed GPRS routers as a backup to cable connections and >> after an epic fight (like a pig with a pumpkin!) resorted to asking for >> help here. >> >> The installation where I make initial testing is: Fedora 26 >> 4.11.9-300.fc26.x86_64 & Shorewall 5.1.10.2-1, GPRS router HUAWEI >> B310s for ISP2, cable ISP1 with DHCP. >> >> Below is my ISP 1&2 structure. My plan is: >> (a) to route all traffic through the main ISP1 when it is available. No >> load ballancing. >> (b) if ISP1 goes down, all traffic is routed through the backup ISP2. >> (c) when ISP1 goes up, all traffic is routed again through ISP1 though >> ISP2 is still up. >> >> What I get successfully is a) and b), but c) does not happen. Is it >> possible to achieve this functionality, where do I make mistakes? >> >> image >> >> I have installed foolsm as described in Multi-ISP shorewall tutorial. >> Other settings: >> *interfaces* contains: >> net enp3s0 detect routeback,optional,dhcp,wait=20 >> net enp1s0 detect routeback,optional,dhcp,wait=20 >> >> *masq* contains: >> enp3s0 0.0.0.0/0 WAN1 >> enp1s0 0.0.0.0/0 192.168.42.254 >> >> *providers* contains: >> N3 1 1 - enp3s0 WAN1 track,primary - >> A1 2 2 - enp1s0 192.168.42.1 track - >> >> *shorewall.conf* contains: >> USE_DEFAULT_RT=Yes >> TRACK PROVIDERS=Yes >> BALLANCE_PROVIDERS=No >> >> In ifcfg files for enp1s0&enp3s0 DEFROUTE=no >> >> *lib.private* contains the following ISP1&2 description: >> name=N3 >> eventscript=/usr/libexec/foolsm/shorewall_script >> checkip=GW1 >> sourceip=WAN1 >> device=enp3s0 >> ttl=20 >> >> name=A1 >> eventscript=/usr/libexec/foolsm/shorewall_script >> checkip=192.168.42.1 # checkip=WAN2 – ??? >> sourceip=192.168.42.254 >> device=enp1s0 >> ttl=20 >> >> The output of systemctl status shorewall and foolsm log look normal. >> >> Checking ISP1&2 routing tables shows: >> *ip route ls table N3 (when connected):* >> default via GW1 dev enp3s0 src WAN1 >> GW1 dev enp3s0 scope link src WAN1 >> *ip route ls table A1 (when connected):* >> default via 192.168.42.1 dev enp1s0 src 192.168.42.254 >> 192.168.42.1 dev enp1s0 scope link src 192.168.42.254 >> *ip route ls table 253* – empty >> >> Now I fiddle with the cables to the LAN cards of the FW disconnecting >> and connecting them in turn and check routing table 254: >> *1. N3 up, A1 up restart FW CPU OK, traffic goes through ISP1 main >> as expected* >> *ip route ls table 254:* >> GW1.0/22 dev enp3s0 proto kernel scope link src WAN1 metric 100 >> GW1 dev enp3s0 scope link src WAN1 >> >> *2. N3 up --> down, A1 up OK, traffic goes through ISP2 as >> expected* >> *ip route ls table 254:* >> default via 192.168.42.1 dev enp1s0 proto static metric 100 >> >> *3. N3 down --> up, A1 up BAD, traffic goes through ISP2 but ISP1 >> is expected* >> *ip route ls table 254:* >> default via 192.168.42.1 dev enp1s0 proto static metric 100 >> default via GW1 dev enp3s0 proto static metric 101 >> GW1.0/22 dev enp3s0 proto kernel scope link src WAN1 metric 100 >> GW1 dev enp3s0 scope link src WAN1 >> >> *4. nothing changed (N3 up, A1 up), restart shorewall OK, traffic >> goes through ISP1 * >> *ip route ls table 254:* >> GW1.0/22 dev enp3s0 proto kernel scope link src WAN1 metric 100 >> GW1 dev enp3s0 scope link src WAN1 >> >> *5. . . . . after 1-2 minutes BAD, traffic goes through ISP2 but >> ISP1 is expected* >> *ip route ls table 254:* >> default via 192.168.42.1 dev enp1s0 proto static metric 100 >> GW1.0/22 dev enp3s0 proto kernel scope link src WAN1 metric 100 >> GW1 dev enp3s0 scope link src WAN1 >> >> *6. . . . disconnect A1:* >> *ip route ls table 254 OK, traffic goes through ISP1 main as >> expected* >> GW1.0/22 dev enp3s0 proto kernel scope link src WAN1 metric 100 >> GW1 dev enp3s0 scope link src WAN1 >> >> It looks to me that things mess up in 3. as the priority of the default >> routes are wrong, ideally there should be no default route through ISP2. >> I am not sure what IP should be in lib.private for ISP2: >> checkip=192.168.42.1 (LAN IP of the GPRS router) or checkip=WAN2 >> (WAN IP of the GPRS router. It has no GW IP). >> >> I will supply any additional info if needed. Thanks for Your advice! >> > > How are you simulating interface failure and restoration? Are you > downing and upping the devices? The reason that I ask is that FooLSM's > default configuration uses 'shorewall disable' and 'shorewall enable' to > react to an interface going down and up respectively. Those operations > should *never* add default routes to the main routing table. > If you simply 'shorewall disable N3', does a default route show up in table 254? -Tom -- Tom Eastep \ Q: What do you get when you cross a mobster with Shoreline, \ an international standard? Washington, USA \ A: Someone who makes you an offer you can't http://shorewall.org \ understand \_______________________________________________ Last weeks we had a lot of holidays and I did some more testing. Default route in table 254 showed up because on some adapters DEFROUTE=yes. I still cannot set DEFROUTE=no on ISP A1 adapter connecting to GPRS router with static IP. With DEFROUTE=no does not accept GW (in NetworkManager GUI) and there is no internet connection. Below is the ifcfg-enp1s0 file which NM creates for legacy as far as I understood: HWADDR=50:3E:AA:04:A5:80 MACADDR=50:3E:AA:04:A5:80 TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none IPADDR=192.168.42.253 PREFIX=24 GATEWAY=192.168.42.1 DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV4_DNS_PRIORITY=100 IPV6INIT=no NAME=enp1s0 UUID=56586d38-7ac7-4f21-ba06-21879d410363 DEVICE=enp1s0 ONBOOT=yes The adapter to ISP N3 has dhcp settings (static IP address over DHCP), it gets GW from ISP: HWADDR=84:16:F9:06:D9:F9 MACADDR=84:16:F9:06:D9:F9 TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=dhcp DNS1=8.8.8.8 DNS2=8.8.4.4 DNS3=10.10.10.10 DEFROUTE=no PEERDNS=no IPV4_FAILURE_FATAL=no IPV6INIT=no NAME=enp3s0 UUID=ded60b05-53c5-457d-adc5-58b54481ca67 ONBOOT=yes Some lines advised in http://www.shorewall.org/MultiISP.html in "DHCP with USE_DEFAULT_RT" section are missing in my config: PERSISTENT_DHCLIENT=yes PEERDNS=no PEERNTP=no DHCLIENTARGS="-nc" If not created from within NM GUI, these records are deleted when connection parameters are edited, how can I insert them in NM? Could it be that NM messes the routing? I noticed that NM adds default route when A1 NIC goes up-->down-->up and a manual shorewall restart is needed to clean it. I did one more test: stopped NM and tried to start the old simple network.service, but it fails with "Failed to start LSB: Bring up/down networking" which I could not solve. systemd-networkd starts OK but routing records do not change at all when ISP is up/down. Guess the NICs state is not monitored dynamically. Andrei _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
