Re: wg(4) listen on a specific interface / address
On Tue, 27 Oct 2020 22:36:38 +0100
Pierre Emeriaud wrote:
> Howdy misc@,
>
> I have a fairly complicated setup with lots of interfaces, a couple of
> rdomains etc.
>
> I'd like wireguard to listen only on an IP address, not all. But if my
> understanding of ifconfig(8) is correct, this doesn't seem possible
> currently:
>
> wgport port
> Set the UDP port that the tunnel operates on. _The
> interface will bind to INADDR_ANY and IN6ADDR_ANY_INIT._
>
> I guess this the reason for the following behaviour?
>
> $ doas ifconfig wg0 wgport 53
> ifconfig: SIOCSWG: Address already in use
> (the error message is generic I guess - but confusing imho)
>
> $ netstat -natfinet | grep 53
> tcp 0 0 127.0.0.1.53 *.*
> LISTEN udp 0 0 127.0.0.1.53 *.*
>
> $ netstat -T1 -natfinet | grep 53
> udp 0 0 127.0.0.1.53 *.*
>
> Is there a way to circumvent this restriction? (is there a reason
> behind it maybe?)
A lot has been said already, however I should clarify things.
wg(4)'s primary goal is to provide a secure network tunnel. We have no
desire to obfuscating or manipulating traffic to bypass restrictive
firewalls, which appears to be what you want to use port 53 for.
Why INADDR_ANY (and IN6ADDR_ANY_INIT)? We listen on all interfaces to
discard any notion of trusting IP addresses and rely entirely on the
crypto to authenticate packets. This ties directly the "roaming" feature
of WireGuard [1]. As Theo mentioned we don't want to monitor for
addressing changes, so INADDR_ANY is correct.
Why no configuration knob for bind address? Well, this is a "simple"
VPN and prides itself on minimising unnecessary configuration while
still achieving it's primary goals. Allowing configuration of the bind
address opens a whole can of complexity worms, including configuration
failure modes and security issues that we don't have consensus on. The
behaviour exhibited on wg(4) is also consistent with implementations of
WireGuard on other platforms. This has been discussed before: [2][3].
Finally, if you want to continue using port 53, bind wg first, then
unwind. Alternatively rdr-to rules will work and I'm guessing your
didn't do any debugging to figure out why your rules weren't working as
expected. If your goal is to bypass restrictive firewalls, you may also
want to add ports 123, 4500, 5060 to your redirect rules, but keep in
mind you're abusing software in ways it wasn't designed for so support
is minimal. I imagine it would look something like the following (with
wg(4) listening on port 53535 on the same rdomain):
pass in on $wan proto udp to (self) \
port { 53, 123, 4500, 5060 } rdr-to 127.0.0.1 port 53535
Cheers,
Matt
[1] https://www.wireguard.com/#built-in-roaming
[2] https://lists.zx2c4.com/pipermail/wireguard/2017-May/001280.html
[3] https://lists.zx2c4.com/pipermail/wireguard/2018-June/003013.html
Re: wg(4) listen on a specific interface / address
> On Oct 29, 2020, at 6:09 PM, Pierre Emeriaud > wrote: > > Le jeu. 29 oct. 2020 à 21:03, Stuart Henderson a > écrit : >> Which DNS server do you have bound on 53? > > unwind > > >>> Is there a reason why wg needs such a large bind? >> Unless/until it gets an option to bind to a specific IP that's all it >> can sanely do. It would definitely be useful IMO. > > This is maybe where it starts to make sense. By binding INADDR_ANY, > this allows wg to accept incoming packets whichever interface they > came from. Maybe to mimic what is done with other tunnels/protocols > operating at L3, while still operating at L4. You can achieve success using pf + routing domains. It’ll work just takes extra effort. I agree a bind IP parameter would be nice, but not a necessity to function. Where one function in the kernel isn’t a jack of all trades (wg) or perfect, another feature can help to achieve the goal (pf + rdomains, the network stack design used by OpenBSD for virtualizing the address and port space).
Re: wg(4) listen on a specific interface / address
Pierre Emeriaud wrote: > Totally agreed. This is because of my stupid idea to share port 53 for > this use. Maybe my understanding of sockets was wrong, but I thought > that applications could use the bind port _if and only_ they weren't > trying to bind the same IP+port, hence my question about this > conflict, which could happen with other ports as well. Such a weird perspective. I guess you've never setup a multhomed machine. INADDR_ANY means all interfaces, so a daemon doesn't need to open a new socket on each interfaces, and listen to the route socket for new interfaces to arrive, or old ones to be disconnected. > Thanks everyone who answered, and if anyone has the definitive answer > about why it wg binds INADDR_ANY, I'd be interested to know. Why does sshd bind to INADDR_ANY? Why does httpd bind to INADDR_ANY? The same reason for wg. It wants to respond to requests on all interfaces. And the loopback is not exempt.
Re: wg(4) listen on a specific interface / address
On 2020/10/29 23:08, Pierre Emeriaud wrote: > Le jeu. 29 oct. 2020 à 21:03, Stuart Henderson a écrit > : > > > > Which DNS server do you have bound on 53? > > unwind > > > > > Is there a reason why wg needs such a large bind? > > > > Unless/until it gets an option to bind to a specific IP that's all it > > can sanely do. It would definitely be useful IMO. > > This is maybe where it starts to make sense. By binding INADDR_ANY, > this allows wg to accept incoming packets whichever interface they > came from. Maybe to mimic what is done with other tunnels/protocols > operating at L3, while still operating at L4. Incoming is one side of it. The other side is the address used for outgoing packets - on a multihomed system it's quite common to bind to a specific port to get the correct source address (which can easily vary between different services running on a machine).
Re: wg(4) listen on a specific interface / address
Le jeu. 29 oct. 2020 à 21:17, Theo de Raadt a écrit : > > Or, don't try to overlay stuff onto a single port. Look, we can tell > what is going on here, you want to tunnel over the least-filtered port > on the internet, but if you do that trying to use that port for another > thing is quite a problem of your own making. Le jeu. 29 oct. 2020 à 23:10, Theo de Raadt a écrit : > > port 53 has a well known use. It is the firstcomer, for a critical service. > > You are abusing that port. Totally agreed. This is because of my stupid idea to share port 53 for this use. Maybe my understanding of sockets was wrong, but I thought that applications could use the bind port _if and only_ they weren't trying to bind the same IP+port, hence my question about this conflict, which could happen with other ports as well. > I could easily argue there is nothing to fix in our kernel. I think I agree, anyhow, this behavior might be intentional, so I'll avoid doing that in the future. Thanks everyone who answered, and if anyone has the definitive answer about why it wg binds INADDR_ANY, I'd be interested to know.
Re: wg(4) listen on a specific interface / address
Pierre Emeriaud wrote: > Le jeu. 29 oct. 2020 à 18:00, Brian Brombacher a écrit > : > > > > > > Then there’s a misconfiguration, wg driver bug, or the driver documentation > > is wrong in ifconfig about wgrtable. > > > > Routing domains are where you can specify multiple conflicting port binds > > and be fine, INADDR_ANY included. > > On that matter there are no issues, only me/my setup. wg has no issues > with binding INADDR_ANY if it is the only software binding on port 53 > _in that rdomain_. The issue I have is when I already have another > software, like a dns resolver here already listening on 127.0.0.1 in > that same rdomain. port 53 has a well known use. It is the firstcomer, for a critical service. You are abusing that port. I could easily argue there is nothing to fix in our kernel.
Re: wg(4) listen on a specific interface / address
Le jeu. 29 oct. 2020 à 21:03, Stuart Henderson a écrit : > > Which DNS server do you have bound on 53? unwind > > Is there a reason why wg needs such a large bind? > > Unless/until it gets an option to bind to a specific IP that's all it > can sanely do. It would definitely be useful IMO. This is maybe where it starts to make sense. By binding INADDR_ANY, this allows wg to accept incoming packets whichever interface they came from. Maybe to mimic what is done with other tunnels/protocols operating at L3, while still operating at L4.
Re: wg(4) listen on a specific interface / address
Le jeu. 29 oct. 2020 à 18:00, Brian Brombacher a écrit : > > > Then there’s a misconfiguration, wg driver bug, or the driver documentation > is wrong in ifconfig about wgrtable. > > Routing domains are where you can specify multiple conflicting port binds and > be fine, INADDR_ANY included. On that matter there are no issues, only me/my setup. wg has no issues with binding INADDR_ANY if it is the only software binding on port 53 _in that rdomain_. The issue I have is when I already have another software, like a dns resolver here already listening on 127.0.0.1 in that same rdomain.
Re: wg(4) listen on a specific interface / address
Stuart Henderson wrote: > On 2020-10-29, Pierre Emeriaud wrote: > > Le jeu. 29 oct. 2020 à 01:20, Theo de Raadt a écrit : > >> > >> I believe you are running into the restriction that we don't allow an > >> INADDR_ANY:port binding to be done after a ipaddr:port binding has been > >> done. It must be done beforehands. > > > > Sorry Theo, maybe things got lost in translation, but if my > > understanding is correct this is not exactly the case here. wg is > > trying to bind to INADDR_ANY, which fails because a dns daemon (in my > > case) is already bound to 127.0.0.1:53 (in wg_socket_open() -L700 of > > if_wg.c-, sin->sin_addr.s_addr = INADDR_ANY?). > > There are extra restrictions, to prevent some software "stealing" packets > intended for some other software. In userland software that wants to coexist > with other software on the same pprt hut a different bound IP needs to use > SO_REUSEADDR (I forgot exactly how the restriction work though). The problem > you are seeing might be related to this. > > Which DNS server do you have bound on 53? > > > Is there a reason why wg needs such a large bind? > > Unless/until it gets an option to bind to a specific IP that's all it > can sanely do. It would definitely be useful IMO. Or, configure it before the application software. Or, don't try to overlay stuff onto a single port. Look, we can tell what is going on here, you want to tunnel over the least-filtered port on the internet, but if you do that trying to use that port for another thing is quite a problem of your own making.
Re: wg(4) listen on a specific interface / address
On 2020-10-29, Pierre Emeriaud wrote: > Le jeu. 29 oct. 2020 à 01:20, Theo de Raadt a écrit : >> >> I believe you are running into the restriction that we don't allow an >> INADDR_ANY:port binding to be done after a ipaddr:port binding has been >> done. It must be done beforehands. > > Sorry Theo, maybe things got lost in translation, but if my > understanding is correct this is not exactly the case here. wg is > trying to bind to INADDR_ANY, which fails because a dns daemon (in my > case) is already bound to 127.0.0.1:53 (in wg_socket_open() -L700 of > if_wg.c-, sin->sin_addr.s_addr = INADDR_ANY?). There are extra restrictions, to prevent some software "stealing" packets intended for some other software. In userland software that wants to coexist with other software on the same pprt hut a different bound IP needs to use SO_REUSEADDR (I forgot exactly how the restriction work though). The problem you are seeing might be related to this. Which DNS server do you have bound on 53? > Is there a reason why wg needs such a large bind? Unless/until it gets an option to bind to a specific IP that's all it can sanely do. It would definitely be useful IMO.
Re: wg(4) listen on a specific interface / address
> On Oct 29, 2020, at 11:21 AM, Pierre Emeriaud > wrote: > > Le jeu. 29 oct. 2020 à 00:09, Brian Brombacher a > écrit : >> >> Scratch that, use the ifconfig wgrtable option to specify separate routing >> domains for the port 53. This lets you initiate many. You still need to >> deal with getting the IP pointing at the right routing domain now. > > I'm already using wgrtable and rdomains, and I can't change the > outside interface to use another rtable. This won't solve the fact > that wg is still trying to bind to INADDR_ANY. > Then there’s a misconfiguration, wg driver bug, or the driver documentation is wrong in ifconfig about wgrtable. Routing domains are where you can specify multiple conflicting port binds and be fine, INADDR_ANY included.
Re: wg(4) listen on a specific interface / address
On 10/29/20 5:20 PM, Kevin Chadwick wrote: > I believe it actually operates at layer 2/3 below IP and uses the default gw > IP > to decide where to operate for a peer to peer link. I'm not actually sure how that makes any sense as it uses UDP which is layer 4. But this says layer 3 "https://www.wireguard.com/papers/wireguard.pdf;
Re: wg(4) listen on a specific interface / address
On 10/29/20 4:00 PM, Pierre Emeriaud wrote: >>> Is there a reason why wg needs such a large bind? >> I don't know why wg does that, because I haven't looked at the code. >> Your configuration is definately pushing the limits. > Allright many thanks Theo. Maybe Jason can chime in on this topic. I believe it actually operates at layer 2/3 below IP and uses the default gw IP to decide where to operate for a peer to peer link. I could be totally wrong and I just hit some Windows bugs but I had issues getting the Windows clients to work for more than a few seconds on a LAN around 6 months ago and that was my conclusion (designed for easy internet use). I just used OpenSSH tunnels instead?
Re: wg(4) listen on a specific interface / address
Le jeu. 29 oct. 2020 à 16:40, Theo de Raadt a écrit : > > > Is there a reason why wg needs such a large bind? > > I don't know why wg does that, because I haven't looked at the code. > Your configuration is definately pushing the limits. Allright many thanks Theo. Maybe Jason can chime in on this topic. regards pierre
Re: wg(4) listen on a specific interface / address
Pierre Emeriaud wrote: > Le jeu. 29 oct. 2020 à 01:20, Theo de Raadt a écrit : > > > > I believe you are running into the restriction that we don't allow an > > INADDR_ANY:port binding to be done after a ipaddr:port binding has been > > done. It must be done beforehands. > > Sorry Theo, maybe things got lost in translation, but if my > understanding is correct this is not exactly the case here. wg is > trying to bind to INADDR_ANY, which fails because a dns daemon (in my > case) is already bound to 127.0.0.1:53 (in wg_socket_open() -L700 of > if_wg.c-, sin->sin_addr.s_addr = INADDR_ANY?). > > Is there a reason why wg needs such a large bind? I don't know why wg does that, because I haven't looked at the code. Your configuration is definately pushing the limits.
Re: wg(4) listen on a specific interface / address
Le jeu. 29 oct. 2020 à 01:20, Theo de Raadt a écrit : > > I believe you are running into the restriction that we don't allow an > INADDR_ANY:port binding to be done after a ipaddr:port binding has been > done. It must be done beforehands. Sorry Theo, maybe things got lost in translation, but if my understanding is correct this is not exactly the case here. wg is trying to bind to INADDR_ANY, which fails because a dns daemon (in my case) is already bound to 127.0.0.1:53 (in wg_socket_open() -L700 of if_wg.c-, sin->sin_addr.s_addr = INADDR_ANY?). Is there a reason why wg needs such a large bind?
Re: wg(4) listen on a specific interface / address
Le jeu. 29 oct. 2020 à 00:09, Brian Brombacher a écrit : > > Scratch that, use the ifconfig wgrtable option to specify separate routing > domains for the port 53. This lets you initiate many. You still need to > deal with getting the IP pointing at the right routing domain now. I'm already using wgrtable and rdomains, and I can't change the outside interface to use another rtable. This won't solve the fact that wg is still trying to bind to INADDR_ANY.
Re: wg(4) listen on a specific interface / address
Pierre Emeriaud wrote: > Le mar. 27 oct. 2020 à 23:46, j...@snoopy.net.nz a écrit > : > > > > > > > > Hi Pierre, > > > > The error may indicate that port 53 on 127.0.0.1 is already used by another > > service. This appears to be confirmed by your netstat example. This is > > probably a dns service. > > Thanks Joe. This is indeed a dns daemon, several in fact. But nothing > should prevent wireguard from using port 53 on any other IP address > than 127.0.0.1 here. (well, nothing but the code that has been > implemented) I believe you are running into the restriction that we don't allow an INADDR_ANY:port binding to be done after a ipaddr:port binding has been done. It must be done beforehands.
Re: wg(4) listen on a specific interface / address
> On Oct 28, 2020, at 6:21 PM, Brian Brombacher wrote: > > > >> On Oct 28, 2020, at 5:07 PM, Pierre Emeriaud >> wrote: >> >> Le mar. 27 oct. 2020 à 23:46, j...@snoopy.net.nz a >> écrit : >>> >>> >>> >>> Hi Pierre, >>> >>> The error may indicate that port 53 on 127.0.0.1 is already used by another >>> service. This appears to be confirmed by your netstat example. This is >>> probably a dns service. >> >> Thanks Joe. This is indeed a dns daemon, several in fact. But nothing >> should prevent wireguard from using port 53 on any other IP address >> than 127.0.0.1 here. (well, nothing but the code that has been >> implemented) >> > > Can you specify separate rdomains for the wg interfaces and still use port 53 > on all plus a dns daemon? > > I have not experimented with any of this guidance. > Scratch that, use the ifconfig wgrtable option to specify separate routing domains for the port 53. This lets you initiate many. You still need to deal with getting the IP pointing at the right routing domain now. https://man.openbsd.org/ifconfig#wgrtable
Re: wg(4) listen on a specific interface / address
> On Oct 28, 2020, at 5:07 PM, Pierre Emeriaud > wrote: > > Le mar. 27 oct. 2020 à 23:46, j...@snoopy.net.nz a > écrit : >> >> >> >> Hi Pierre, >> >> The error may indicate that port 53 on 127.0.0.1 is already used by another >> service. This appears to be confirmed by your netstat example. This is >> probably a dns service. > > Thanks Joe. This is indeed a dns daemon, several in fact. But nothing > should prevent wireguard from using port 53 on any other IP address > than 127.0.0.1 here. (well, nothing but the code that has been > implemented) > Can you specify separate rdomains for the wg interfaces and still use port 53 on all plus a dns daemon? I have not experimented with any of this guidance.
Re: wg(4) listen on a specific interface / address
Le mar. 27 oct. 2020 à 23:46, j...@snoopy.net.nz a écrit : > > > > Hi Pierre, > > The error may indicate that port 53 on 127.0.0.1 is already used by another > service. This appears to be confirmed by your netstat example. This is > probably a dns service. Thanks Joe. This is indeed a dns daemon, several in fact. But nothing should prevent wireguard from using port 53 on any other IP address than 127.0.0.1 here. (well, nothing but the code that has been implemented)
Re: wg(4) listen on a specific interface / address
Hi Brian Le mar. 27 oct. 2020 à 23:07, Brian Brombacher a écrit : > > I wonder if multiple ports, 5053, 5153 (and so on) redirected using pf rdr-to > rules may work? That way you can setup rules like first IP + port 53 > redirect to 5053, second IP + 53 redirect to 5153? > > May be worth a shot trying. Not an answer to your question, but as a > workaround for others. I just tried that, with rdr-to for inbound and nat-to for outbound. It could work indeed, but I did not manage to make it work properly. match in quick on $wan proto udp from any to $vpnip port 53 rdr-to self port 24854 rtable 1 match out quick on $wan proto udp from $vpnip to any port 24854 nat-to $vpnip port 53 rtable 1 Anyhow this is unfortunately painful. This means that any port shown on 'ifconfig wg' has to be mentally merged with pf rules, and while this could technically work, this is difficult to troubleshoot :(
Re: wg(4) listen on a specific interface / address
Hi Pierre, The error may indicate that port 53 on 127.0.0.1 is already used by another service. This appears to be confirmed by your netstat example. This is probably a dns service. Hope this helps explain the error. Joe Original message From: Pierre Emeriaud Date: Wed, 28 Oct 2020, 10:36 am To: misc Subject: wg(4) listen on a specific interface / address Howdy misc@, I have a fairly complicated setup with lots of interfaces, a couple of rdomains etc. I'd like wireguard to listen only on an IP address, not all. But if my understanding of ifconfig(8) is correct, this doesn't seem possible currently: wgport port Set the UDP port that the tunnel operates on. _The interface will bind to INADDR_ANY and IN6ADDR_ANY_INIT._ I guess this the reason for the following behaviour? $ doas ifconfig wg0 wgport 53 ifconfig: SIOCSWG: Address already in use (the error message is generic I guess - but confusing imho) $ netstat -natfinet | grep 53 tcp 0 0 127.0.0.1.53 *.* LISTEN udp 0 0 127.0.0.1.53 *.* $ netstat -T1 -natfinet | grep 53 udp 0 0 127.0.0.1.53 *.* Is there a way to circumvent this restriction? (is there a reason behind it maybe?) thanks -- pierre
Re: wg(4) listen on a specific interface / address
> On Oct 27, 2020, at 5:33 PM, Pierre Emeriaud > wrote: > > Howdy misc@, > > I have a fairly complicated setup with lots of interfaces, a couple of > rdomains etc. > > I'd like wireguard to listen only on an IP address, not all. But if my > understanding of ifconfig(8) is correct, this doesn't seem possible > currently: > > wgport port > Set the UDP port that the tunnel operates on. _The interface will > bind to INADDR_ANY and IN6ADDR_ANY_INIT._ > > I guess this the reason for the following behaviour? > > $ doas ifconfig wg0 wgport 53 > ifconfig: SIOCSWG: Address already in use > (the error message is generic I guess - but confusing imho) > > $ netstat -natfinet | grep 53 > tcp 0 0 127.0.0.1.53 *.*LISTEN > udp 0 0 127.0.0.1.53 *.* > > $ netstat -T1 -natfinet | grep 53 > udp 0 0 127.0.0.1.53 *.* > > Is there a way to circumvent this restriction? (is there a reason > behind it maybe?) > > thanks > -- > pierre > I wonder if multiple ports, 5053, 5153 (and so on) redirected using pf rdr-to rules may work? That way you can setup rules like first IP + port 53 redirect to 5053, second IP + 53 redirect to 5153? May be worth a shot trying. Not an answer to your question, but as a workaround for others.
wg(4) listen on a specific interface / address
Howdy misc@, I have a fairly complicated setup with lots of interfaces, a couple of rdomains etc. I'd like wireguard to listen only on an IP address, not all. But if my understanding of ifconfig(8) is correct, this doesn't seem possible currently: wgport port Set the UDP port that the tunnel operates on. _The interface will bind to INADDR_ANY and IN6ADDR_ANY_INIT._ I guess this the reason for the following behaviour? $ doas ifconfig wg0 wgport 53 ifconfig: SIOCSWG: Address already in use (the error message is generic I guess - but confusing imho) $ netstat -natfinet | grep 53 tcp 0 0 127.0.0.1.53 *.*LISTEN udp 0 0 127.0.0.1.53 *.* $ netstat -T1 -natfinet | grep 53 udp 0 0 127.0.0.1.53 *.* Is there a way to circumvent this restriction? (is there a reason behind it maybe?) thanks -- pierre
