I have been wanting to get access to my PBX over my failover tunnel for
some time now but didn't know how to get it done (when failover was not
active -- works when astlinux is in failover mode). This thread prompted
me to try and get it setup, inspired by Lonnie pointing out fwmark.
Unfortunately what I thought would be a quick exercise took several hours
to get going. First a diagram...
[RemoteDev]
| public internet
---------------------------------------
| eth0 | eth0
[failover] wg0 --------- wg0 [astlinux]
172.23.x.2 172.23.x.1 | eth1 192.168.x.1
--------------------------
| private LAN
[internal system] 192.168.x.y
failover could be astlinux or any linux that can act as a router/gateway
failover is connected to public internet through its eth0 interface
failover is connected to astlinux over wireguard.
astlinux is connected to public internet through its eth0 interface
astlinux is connected to failover over wireguard.
astlinux is connected to a private LAN 192.168.x.0/24
Desired behavior is to allow [RemoteDev] to access [astlinux] (ssh or
https) or [internal system] connecting through either astlinux eth0 or
failover eth0. For this to work [failover] must NAT inbound ssh or https
over to [astlinux] or [internal system]. That part is easy enough and if
failover is active everything works fine as return traffic is routed back
over wireguard. But I want it to work even if failover is not active.
First, the wireguard feature to set fwmark on the interface does nothing to
help. I tried. And the documentation (https://www.wireguard.com/netns/)
states that this sets a mark on outbound traffic to the wireguard UDP
port. In other words the already encrypted packets going out through eth0
port 51820. Actual packets flowing in/out of the tunnel are not marked.
But the principle of using fwmark is sound, we just need another way of
marking the packets.
This is what worked for me... on the [astlinux] system...
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -i wg0 -j MARK --set-xmark 0x4/0x4
iptables -t mangle -A PREROUTING -i wg0 -j CONNMARK --save-mark
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark
ip route flush table 300
ip route add table 300 default dev wg0 via 172.23.x.2
ip rule add from 192.168.x.0/24 fwmark 0x4/0x4 table 300 priority 1000
Explanation...
- We are setting a firewall mark on all inbound traffic from wg0.
fwmark is a 32-bit unsigned and can be treated as a value or a 32-bit
field. I have chosen to treat it as a 32-bit field and using mask only set
bit 2, this means I can use other bits for other purposes in
iptables/netfilter/route if I want. You can pick any bit or simply use a
integer value. Whatever you do needs to be consistent and be aware of any
other uses of fwmark in your firewall.
- We must save this fwmark with the CONNMARK extension, so that we can
reattach the mark on related traffic. fwmarks do not attach to the IP
packet, they exist in the context of this system's kernel only. So
connection tracking is required to keep track of it.
- PREROUTING chain is called for all inbound traffic from all
interfaces. So replies from LAN through eth1 will come here too and this
is where we need to restore the fwmark on returning packets using the
CONNMARK connection tracking. I do that first before setting mark on wg0
traffic.
- That works for traffic passing through [astlinux] to the internal
LAN. But for traffic to astlinux (e.g. the web interface) the replies
originate on the astlinux box itself and therefore do not pass through
PREROUTING. Therefore we must restore the fwmark on the OUTPUT chain so
that locally generated packets are tagged if required before we get to
routing.
https://www.frozentux.net/iptables-tutorial/iptables-tutorial.html#TRAVERSINGOFTABLES
The above tags packets that need to be routed back through the wireguard
tunnel. Now have to mess with the routing tables to act on the marks.
- Picking any free ip2route table (I randomly choose 300) delete
anything currently on that table.
- Add a default route for that table to send all traffic through the
wireguard tunnel
- Add an ip rule to select which packets should be routed through this
table based on the fwmark. Critical in this step is to only select packets
originating from the internal LAN -- ie, we are only interested in outbound
packets. if we don't do that then inbound packets from wg0 will also get
selected and we don't want that... because we did not duplicate the content
of the main routing table into this new table so there are no routes to
local destinations.
The above rules can be set on POST_UP in the astlinux wireguard script.
And deleted in the POST_DOWN (or PRE_DOWN, should not matter). Though
beware that if you reload the firewall, without also restarting wireguard,
then the above would get deleted.
What might this look like in wireguard.script POST_UP section....
# Setup routing table for traffic originating on $interface so that
# we can set rules to route replies to that traffic over $interface
# assume /etc/rc.conf read and interface="$2"
if [ -n "$WAN_FAILOVER_SECONDARY_GW" ]; then
echo "WireGuard: set iptables and ip route table $WG0_TUNNEL_ROUTE_TABLE
for $interface to reply to inbound traffic via $WAN_FAILOVER_SECONDARY_GW"
WG0_TUNNEL_ROUTE_TABLE="300"
INTNET=$(netcalc $INTIP $INTNM | sed -n -r -e 's/^Network *:
*([0-9\.\/]+).*$/\1/p')
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -i $interface -j MARK --set-xmark 0x4/0x4
iptables -t mangle -A PREROUTING -i $interface -j CONNMARK --save-mark
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark
ip route flush table $WG0_TUNNEL_ROUTE_TABLE
ip route add table $WG0_TUNNEL_ROUTE_TABLE default dev $interface via
$WAN_FAILOVER_SECONDARY_GW
ip rule add from $INTNET fwmark 0x4/0x4 table $WG0_TUNNEL_ROUTE_TABLE
priority 1000
fi
Don't forget to delete on POST_DOWN.
IPv6 is left as homework exercise for the reader.
Enjoy !!
David
On Sat, Oct 6, 2018 at 7:54 PM Lonnie Abelbeck <li...@lonnie.abelbeck.com>
wrote:
> Yes, is all comes down to the routing at PBX2.
>
> Consider this ... the PC has IP 1.2.3.4, so the NAT forward will have a
> SRC address of 1.2.3.4 when received by 172.29.253.2 on PBX2. If the
> routing on PBX2 routes 1.2.3.4 back through the wireguard tunnel then it
> will work as you want. On the other-hand if PBX2 routes 1.2.3.4 over it's
> EXT interface then it will not work as you want.
>
> Probably the most elegant solution for routing on PBX2 is "policy routing"
> using "ip rule ..." where traffic through the wireguard tunnel could have a
> "fwmark" and add routing rules based on whether the packet traversed the
> wireguard tunnel. I have only played with this ... all the hooks are
> currently available using /mnt/kd/wireguard.script
>
> https://doc.astlinux-project.org/userdoc:tt_wireguard_vpn#optional_action_script
>
> but if you are not familiar with policy-based routing in Linux, this takes
> some research to get a handle on.
>
> Alternatively, if your public PC's are always off a know subnet, you could
> add a static destination route on PBX2 to your PC's via the wireguard
> tunnel.
>
> Lonnie
>
>
>
> > On Oct 6, 2018, at 6:11 PM, Michael Knill <
> michael.kn...@ipcsolutions.com.au> wrote:
> >
> > Sorry Lonnie I am a little confused.
> > The setup is as follows:
> >
> > PC -- [internet] -- PBX1 -- [WG VPN] -- PBX2
> >
> > I can ping the private Wireguard PBX2 address (172.29.253.2) from PBX1
> (172.29.253.2)
> > So I want to NAT PBX1 EXTIF on a particular port to PBX2 WG IP
> 172.29.253.2.
> > I have set up the NAT_FOREIGN_NETWORK for the entire private address
> space.
> >
> > Thanks
> >
> > Regards
> > Michael Knill
> >
> > On 7/10/18, 12:01 am, "Lonnie Abelbeck" <li...@lonnie.abelbeck.com>
> wrote:
> >
> >
> >
> >> On Oct 5, 2018, at 10:29 PM, Michael Knill <
> michael.kn...@ipcsolutions.com.au> wrote:
> >>
> >> Hi Group
> >>
> >> Im wanting to set up a NAT rule from NAT EXT to a Wireguard VPN
> endpoint. Is this possible?
> >> It does not seem to work with NAT EXT -> LAN.
> >> If not, is there a custom rule I can try?
> >>
> >> Basically I want to SSH to the VPN endpoint directly, via the transit
> DR server.
> >>
> >> Thanks so much.
> >
> > Hi Michael, short answer is yes, but depending on the routing.
> >
> > Start with a diagram ...
> >
> > public_1 -- pbx1 [ wg_1_ip ] -- wireguard -- [ wg_2_ip ] pbx2 --
> public_2
> >
> >
> > My understanding is you want to SSH to wg_1_ip using public_2 ?
> Correct me if I mis-understood.
> >
> > Yes, a "NAT EXT -> LAN" on public_2 to wg_1_ip will work *only if*
> the SSH return path at pbx1 goes through the wireguard vpn.
> >
> > I have personally tried this when pbx1 was on failover using
> wireguard over LTE/4G, as such all pbx1 traffic was routed over wireguard,
> as such a "NAT EXT -> LAN" on public_2 to wg_1_ip worked since the SSH
> return packets passed over wireguard to pbx2.
> >
> > Tip -> Similar, but if a "NAT EXT -> LAN" on public_2 to a LAN IP on
> pbx1 you would need to set NAT_FOREIGN_NETWORK on pbx2 of the pbx1 LAN so
> it is NAT'ed on pbx2.
> >
> > Lonnie
> >
> >
> >
> >
> > _______________________________________________
> > Astlinux-users mailing list
> > Astlinux-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/astlinux-users
> >
> > Donations to support AstLinux are graciously accepted via PayPal to
> pay...@krisk.org.
> >
> >
> >
> > _______________________________________________
> > Astlinux-users mailing list
> > Astlinux-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/astlinux-users
> >
> > Donations to support AstLinux are graciously accepted via PayPal to
> pay...@krisk.org.
>
>
>
> _______________________________________________
> Astlinux-users mailing list
> Astlinux-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/astlinux-users
>
> Donations to support AstLinux are graciously accepted via PayPal to
> pay...@krisk.org.
_______________________________________________
Astlinux-users mailing list
Astlinux-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/astlinux-users
Donations to support AstLinux are graciously accepted via PayPal to
pay...@krisk.org.
