HI Lonnie, Thanks. I'll take another look and test it. But my initial testing with that did not work and checking the documentation the way I understood it the fwmark is set on the outbound encrypted packets, not on packets going through the tunnel. But it would be simpler if that worked so worth another try.
Thanks David On Tue, Oct 9, 2018 at 10:47 PM Lonnie Abelbeck <li...@lonnie.abelbeck.com> wrote: > Hi David, > > Great stuff (as usual), though I'm thinking it may be simpler to not use > iptables, but something like: > -- > wg set "$INTERFACE" fwmark $table > -- > which marks the wireguard packets, and use that in "ip rule ..." logic, > just examples ... > > -- > ip $proto route add "$1" dev "$INTERFACE" table $table > ip $proto rule add not fwmark $table table $table > -- > > The above snippets are taken from Jason's wg-quick linux bash script ... > not easy to read but there are some gems in there: > https://git.zx2c4.com/WireGuard/tree/src/tools/wg-quick/linux.bash#n175 > > Jason uses the UDP port 51820 as the default "fwmark" and "table" value. > > Lonnie > > > > > On Oct 9, 2018, at 3:54 PM, David Kerr <da...@kerr.net> wrote: > > > > I have been wanting to get access to my PBX over my failover tunnel for > some time now but didn't know how to get it done (when failover was not > active -- works when astlinux is in failover mode). This thread prompted > me to try and get it setup, inspired by Lonnie pointing out fwmark. > Unfortunately what I thought would be a quick exercise took several hours > to get going. First a diagram... > > > > [RemoteDev] > > | public internet > > --------------------------------------- > > | eth0 | eth0 > > [failover] wg0 --------- wg0 [astlinux] > > 172.23.x.2 172.23.x.1 | eth1 192.168.x.1 > > -------------------------- > > | private LAN > > [internal system] 192.168.x.y > > > > > > failover could be astlinux or any linux that can act as a router/gateway > > failover is connected to public internet through its eth0 interface > > failover is connected to astlinux over wireguard. > > astlinux is connected to public internet through its eth0 interface > > astlinux is connected to failover over wireguard. > > astlinux is connected to a private LAN 192.168.x.0/24 > > > > Desired behavior is to allow [RemoteDev] to access [astlinux] (ssh or > https) or [internal system] connecting through either astlinux eth0 or > failover eth0. For this to work [failover] must NAT inbound ssh or https > over to [astlinux] or [internal system]. That part is easy enough and if > failover is active everything works fine as return traffic is routed back > over wireguard. But I want it to work even if failover is not active. > > > > First, the wireguard feature to set fwmark on the interface does nothing > to help. I tried. And the documentation ( > https://www.wireguard.com/netns/) states that this sets a mark on > outbound traffic to the wireguard UDP port. In other words the already > encrypted packets going out through eth0 port 51820. Actual packets > flowing in/out of the tunnel are not marked. But the principle of using > fwmark is sound, we just need another way of marking the packets. > > > > This is what worked for me... on the [astlinux] system... > > > > iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark > > iptables -t mangle -A PREROUTING -i wg0 -j MARK --set-xmark 0x4/0x4 > > iptables -t mangle -A PREROUTING -i wg0 -j CONNMARK --save-mark > > iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark > > ip route flush table 300 > > ip route add table 300 default dev wg0 via 172.23.x.2 > > ip rule add from 192.168.x.0/24 fwmark 0x4/0x4 table 300 priority 1000 > > > > Explanation... > > • We are setting a firewall mark on all inbound traffic from wg0. > fwmark is a 32-bit unsigned and can be treated as a value or a 32-bit > field. I have chosen to treat it as a 32-bit field and using mask only set > bit 2, this means I can use other bits for other purposes in > iptables/netfilter/route if I want. You can pick any bit or simply use a > integer value. Whatever you do needs to be consistent and be aware of any > other uses of fwmark in your firewall. > > • We must save this fwmark with the CONNMARK extension, so that we > can reattach the mark on related traffic. fwmarks do not attach to the IP > packet, they exist in the context of this system's kernel only. So > connection tracking is required to keep track of it. > > • PREROUTING chain is called for all inbound traffic from all > interfaces. So replies from LAN through eth1 will come here too and this > is where we need to restore the fwmark on returning packets using the > CONNMARK connection tracking. I do that first before setting mark on wg0 > traffic. > > • That works for traffic passing through [astlinux] to the > internal LAN. But for traffic to astlinux (e.g. the web interface) the > replies originate on the astlinux box itself and therefore do not pass > through PREROUTING. Therefore we must restore the fwmark on the OUTPUT > chain so that locally generated packets are tagged if required before we > get to routing. > https://www.frozentux.net/iptables-tutorial/iptables-tutorial.html#TRAVERSINGOFTABLES > > The above tags packets that need to be routed back through the wireguard > tunnel. Now have to mess with the routing tables to act on the marks. > > • Picking any free ip2route table (I randomly choose 300) delete > anything currently on that table. > > • Add a default route for that table to send all traffic through > the wireguard tunnel > > • Add an ip rule to select which packets should be routed through > this table based on the fwmark. Critical in this step is to only select > packets originating from the internal LAN -- ie, we are only interested in > outbound packets. if we don't do that then inbound packets from wg0 will > also get selected and we don't want that... because we did not duplicate > the content of the main routing table into this new table so there are no > routes to local destinations. > > The above rules can be set on POST_UP in the astlinux wireguard script. > And deleted in the POST_DOWN (or PRE_DOWN, should not matter). Though > beware that if you reload the firewall, without also restarting wireguard, > then the above would get deleted. > > > > What might this look like in wireguard.script POST_UP section.... > > > > # Setup routing table for traffic originating on $interface so that > > # we can set rules to route replies to that traffic over $interface > > # assume /etc/rc.conf read and interface="$2" > > if [ -n "$WAN_FAILOVER_SECONDARY_GW" ]; then > > echo "WireGuard: set iptables and ip route table > $WG0_TUNNEL_ROUTE_TABLE for $interface to reply to inbound traffic via > $WAN_FAILOVER_SECONDARY_GW" > > WG0_TUNNEL_ROUTE_TABLE="300" > > INTNET=$(netcalc $INTIP $INTNM | sed -n -r -e 's/^Network *: > *([0-9\.\/]+).*$/\1/p') > > iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark > > iptables -t mangle -A PREROUTING -i $interface -j MARK --set-xmark > 0x4/0x4 > > iptables -t mangle -A PREROUTING -i $interface -j CONNMARK --save-mark > > iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark > > ip route flush table $WG0_TUNNEL_ROUTE_TABLE > > ip route add table $WG0_TUNNEL_ROUTE_TABLE default dev $interface via > $WAN_FAILOVER_SECONDARY_GW > > ip rule add from $INTNET fwmark 0x4/0x4 table $WG0_TUNNEL_ROUTE_TABLE > priority 1000 > > fi > > > > Don't forget to delete on POST_DOWN. > > > > IPv6 is left as homework exercise for the reader. > > > > Enjoy !! > > > > David > > > > On Sat, Oct 6, 2018 at 7:54 PM Lonnie Abelbeck < > li...@lonnie.abelbeck.com> wrote: > > Yes, is all comes down to the routing at PBX2. > > > > Consider this ... the PC has IP 1.2.3.4, so the NAT forward will have a > SRC address of 1.2.3.4 when received by 172.29.253.2 on PBX2. If the > routing on PBX2 routes 1.2.3.4 back through the wireguard tunnel then it > will work as you want. On the other-hand if PBX2 routes 1.2.3.4 over it's > EXT interface then it will not work as you want. > > > > Probably the most elegant solution for routing on PBX2 is "policy > routing" using "ip rule ..." where traffic through the wireguard tunnel > could have a "fwmark" and add routing rules based on whether the packet > traversed the wireguard tunnel. I have only played with this ... all the > hooks are currently available using /mnt/kd/wireguard.script > > > https://doc.astlinux-project.org/userdoc:tt_wireguard_vpn#optional_action_script > > > > but if you are not familiar with policy-based routing in Linux, this > takes some research to get a handle on. > > > > Alternatively, if your public PC's are always off a know subnet, you > could add a static destination route on PBX2 to your PC's via the wireguard > tunnel. > > > > Lonnie > > > > > > > > > On Oct 6, 2018, at 6:11 PM, Michael Knill < > michael.kn...@ipcsolutions.com.au> wrote: > > > > > > Sorry Lonnie I am a little confused. > > > The setup is as follows: > > > > > > PC -- [internet] -- PBX1 -- [WG VPN] -- PBX2 > > > > > > I can ping the private Wireguard PBX2 address (172.29.253.2) from PBX1 > (172.29.253.2) > > > So I want to NAT PBX1 EXTIF on a particular port to PBX2 WG IP > 172.29.253.2. > > > I have set up the NAT_FOREIGN_NETWORK for the entire private address > space. > > > > > > Thanks > > > > > > Regards > > > Michael Knill > > > > > > On 7/10/18, 12:01 am, "Lonnie Abelbeck" <li...@lonnie.abelbeck.com> > wrote: > > > > > > > > > > > >> On Oct 5, 2018, at 10:29 PM, Michael Knill < > michael.kn...@ipcsolutions.com.au> wrote: > > >> > > >> Hi Group > > >> > > >> Im wanting to set up a NAT rule from NAT EXT to a Wireguard VPN > endpoint. Is this possible? > > >> It does not seem to work with NAT EXT -> LAN. > > >> If not, is there a custom rule I can try? > > >> > > >> Basically I want to SSH to the VPN endpoint directly, via the transit > DR server. > > >> > > >> Thanks so much. > > > > > > Hi Michael, short answer is yes, but depending on the routing. > > > > > > Start with a diagram ... > > > > > > public_1 -- pbx1 [ wg_1_ip ] -- wireguard -- [ wg_2_ip ] pbx2 -- > public_2 > > > > > > > > > My understanding is you want to SSH to wg_1_ip using public_2 ? > Correct me if I mis-understood. > > > > > > Yes, a "NAT EXT -> LAN" on public_2 to wg_1_ip will work *only if* > the SSH return path at pbx1 goes through the wireguard vpn. > > > > > > I have personally tried this when pbx1 was on failover using > wireguard over LTE/4G, as such all pbx1 traffic was routed over wireguard, > as such a "NAT EXT -> LAN" on public_2 to wg_1_ip worked since the SSH > return packets passed over wireguard to pbx2. > > > > > > Tip -> Similar, but if a "NAT EXT -> LAN" on public_2 to a LAN IP > on pbx1 you would need to set NAT_FOREIGN_NETWORK on pbx2 of the pbx1 LAN > so it is NAT'ed on pbx2. > > > > > > Lonnie > > > > > > > > > > > > > > > _______________________________________________ > > > Astlinux-users mailing list > > > Astlinux-users@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > > > > > Donations to support AstLinux are graciously accepted via PayPal to > pay...@krisk.org. > > > > > > > > > > > > _______________________________________________ > > > Astlinux-users mailing list > > > Astlinux-users@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > > > > > Donations to support AstLinux are graciously accepted via PayPal to > pay...@krisk.org. > > > > > > > > _______________________________________________ > > Astlinux-users mailing list > > Astlinux-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > > > Donations to support AstLinux are graciously accepted via PayPal to > pay...@krisk.org. > > _______________________________________________ > > Astlinux-users mailing list > > Astlinux-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > > > Donations to support AstLinux are graciously accepted via PayPal to > pay...@krisk.org. > > > > _______________________________________________ > Astlinux-users mailing list > Astlinux-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > Donations to support AstLinux are graciously accepted via PayPal to > pay...@krisk.org.
_______________________________________________ Astlinux-users mailing list Astlinux-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/astlinux-users Donations to support AstLinux are graciously accepted via PayPal to pay...@krisk.org.
