[PATCH 1/2] ipc: freebsd: avoid leaking memory in kernel_get_device()
From: Kyle Evans
Primarily, front-load validation of an allowed-ip entry to before we
allocate `aip`, so that we don't need to free() it if we end up skipping
this entry. Assert that `aip` is NULL after we exit the loop, as we
should have transfered ownership to the `peer` or freed it in all paths
through the allowed-ip loop.
FreeBSD-Coverity: 1500405
Signed-off-by: Kyle Evans
---
src/ipc-freebsd.h | 23 +--
1 file changed, 17 insertions(+), 6 deletions(-)
diff --git a/src/ipc-freebsd.h b/src/ipc-freebsd.h
index b5be15b..78064e9 100644
--- a/src/ipc-freebsd.h
+++ b/src/ipc-freebsd.h
@@ -8,6 +8,8 @@
#include
#include
+#include
+
#define IPC_SUPPORTS_KERNEL_INTERFACE
static int get_dgram_socket(void)
@@ -118,7 +120,7 @@ static int kernel_get_device(struct wgdevice **device,
const char *ifname)
goto skip_peers;
for (i = 0; i < peer_count; ++i) {
struct wgpeer *peer;
- struct wgallowedip *aip;
+ struct wgallowedip *aip = NULL;
const nvlist_t *const *nvl_aips;
size_t aip_count, j;
@@ -169,11 +171,13 @@ static int kernel_get_device(struct wgdevice **device,
const char *ifname)
if (!aip_count || !nvl_aips)
goto skip_allowed_ips;
for (j = 0; j < aip_count; ++j) {
+ if (!nvlist_exists_number(nvl_aips[j], "cidr"))
+ continue;
+ if (!nvlist_exists_binary(nvl_aips[j], "ipv4") &&
!nvlist_exists_binary(nvl_aips[j], "ipv6"))
+ continue;
aip = calloc(1, sizeof(*aip));
if (!aip)
goto err_allowed_ips;
- if (!nvlist_exists_number(nvl_aips[j], "cidr"))
- continue;
number = nvlist_get_number(nvl_aips[j], "cidr");
if (nvlist_exists_binary(nvl_aips[j], "ipv4")) {
binary = nvlist_get_binary(nvl_aips[j], "ipv4",
);
@@ -184,7 +188,8 @@ static int kernel_get_device(struct wgdevice **device,
const char *ifname)
aip->family = AF_INET;
aip->cidr = number;
memcpy(>ip4, binary, sizeof(aip->ip4));
- } else if (nvlist_exists_binary(nvl_aips[j], "ipv6")) {
+ } else {
+ assert(nvlist_exists_binary(nvl_aips[j],
"ipv6"));
binary = nvlist_get_binary(nvl_aips[j], "ipv6",
);
if (!binary || number > 128) {
ret = EINVAL;
@@ -193,14 +198,14 @@ static int kernel_get_device(struct wgdevice **device,
const char *ifname)
aip->family = AF_INET6;
aip->cidr = number;
memcpy(>ip6, binary, sizeof(aip->ip6));
- } else
- continue;
+ }
if (!peer->first_allowedip)
peer->first_allowedip = aip;
else
peer->last_allowedip->next_allowedip = aip;
peer->last_allowedip = aip;
+ aip = NULL;
continue;
err_allowed_ips:
@@ -209,6 +214,12 @@ static int kernel_get_device(struct wgdevice **device,
const char *ifname)
free(aip);
goto err_peer;
}
+
+ /*
+* Nothing leaked, hopefully -- ownership transferred or aip
+* freed.
+*/
+ assert(aip == NULL);
skip_allowed_ips:
if (!dev->first_peer)
dev->first_peer = peer;
--
2.36.1
[PATCH 2/2] ipc: freebsd: NULL out some freed memory in kernel_set_device()
From: Kyle Evans
The `err` path in kernel_set_device() will attempt to free() allocated
nvl_peers, but these two cases meant we could end up attempting a use
after free or a double free, as we rely on nvlist_destroy(NULL) being
a NOP as well as free(NULL).
FreeBSD-Coverity: 1500421
Signed-off-by: Kyle Evans
---
src/ipc-freebsd.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/ipc-freebsd.h b/src/ipc-freebsd.h
index 78064e9..f9d3550 100644
--- a/src/ipc-freebsd.h
+++ b/src/ipc-freebsd.h
@@ -333,6 +333,7 @@ static int kernel_set_device(struct wgdevice *dev)
nvlist_destroy(nvl_aips[j]);
free(nvl_aips);
nvlist_destroy(nvl_peers[i]);
+ nvl_peers[i] = NULL;
goto err;
}
if (i) {
@@ -340,9 +341,11 @@ static int kernel_set_device(struct wgdevice *dev)
for (i = 0; i < peer_count; ++i)
nvlist_destroy(nvl_peers[i]);
free(nvl_peers);
+ nvl_peers = NULL;
}
wgd.wgd_data = nvlist_pack(nvl_device, _size);
nvlist_destroy(nvl_device);
+ nvl_device = NULL;
if (!wgd.wgd_data)
goto err;
s = get_dgram_socket();
--
2.36.1
Re: [PATCH] wg: freebsd: move if_wg path to reflect new in-tree location
https://git.zx2c4.com/wireguard-tools/commit/?id=7b2ae7aa2f52fbac65874a641cbfbb0182d0ba46 Applied, thanks! (Very excited about all this...) Jason
[PATCH] wg: freebsd: move if_wg path to reflect new in-tree location
From: Kyle Evans
When we re-added if_wg to the tree, we changed directories in dev to
strip the if_ (we don't use this prefix for other interfaces'
directories). Adjust it here as a convenience, so that when we import
wireguard-tools to FreeBSD the path will just work as-is with our usual
build.
Signed-off-by: Kyle Evans
---
src/ipc-freebsd.h | 2 +-
src/uapi/freebsd/dev/{if_wg => wg}/if_wg.h | 0
2 files changed, 1 insertion(+), 1 deletion(-)
rename src/uapi/freebsd/dev/{if_wg => wg}/if_wg.h (100%)
diff --git a/src/ipc-freebsd.h b/src/ipc-freebsd.h
index 2c10c10..b5be15b 100644
--- a/src/ipc-freebsd.h
+++ b/src/ipc-freebsd.h
@@ -6,7 +6,7 @@
#include
#include
-#include
+#include
#define IPC_SUPPORTS_KERNEL_INTERFACE
diff --git a/src/uapi/freebsd/dev/if_wg/if_wg.h
b/src/uapi/freebsd/dev/wg/if_wg.h
similarity index 100%
rename from src/uapi/freebsd/dev/if_wg/if_wg.h
rename to src/uapi/freebsd/dev/wg/if_wg.h
--
2.36.1
Re: FreeBSD current socket-src changed. Wireguard not compiling.
On Sun, Sep 4, 2022 at 7:07 PM Jason A. Donenfeld wrote: > > Hi Ed, > > On Wed, Aug 31, 2022 at 8:18 PM Ed Maste wrote: > > > > On Mon, 29 Aug 2022 at 12:18, Jason A. Donenfeld wrote: > > > On Tue, Aug 23, 2022 at 12:26:21PM +0300, Michael Pro wrote: > > > > Tonight after updating kernel freebsd current I got coredump with > > > > wireguard enabled kernel module. > > > > ... > > > Thanks. Is there a __FreeBSD_version__ change that corresponds? > > > > It is not an exact match, but was bumped to 1400066 shortly after and > > should be a suitable value to check. > > Thanks. > > Michael - could you give this a try? > https://git.zx2c4.com/wireguard-freebsd/commit/?id=89b7292797c17449e24f5b8b08cb2412a51d2484 Err, that would be the wrong code copy This, I meant: https://git.zx2c4.com/wireguard-freebsd/commit/?id=20584d1c329ed2a71893375fa11ca4c56ed9f642
Re: FreeBSD current socket-src changed. Wireguard not compiling.
Hi Ed, On Wed, Aug 31, 2022 at 8:18 PM Ed Maste wrote: > > On Mon, 29 Aug 2022 at 12:18, Jason A. Donenfeld wrote: > > On Tue, Aug 23, 2022 at 12:26:21PM +0300, Michael Pro wrote: > > > Tonight after updating kernel freebsd current I got coredump with > > > wireguard enabled kernel module. > > > ... > > Thanks. Is there a __FreeBSD_version__ change that corresponds? > > It is not an exact match, but was bumped to 1400066 shortly after and > should be a suitable value to check. Thanks. Michael - could you give this a try? https://git.zx2c4.com/wireguard-freebsd/commit/?id=89b7292797c17449e24f5b8b08cb2412a51d2484 Jason
Re: FreeBSD current socket-src changed. Wireguard not compiling.
On Mon, 29 Aug 2022 at 12:18, Jason A. Donenfeld wrote: > On Tue, Aug 23, 2022 at 12:26:21PM +0300, Michael Pro wrote: > > Tonight after updating kernel freebsd current I got coredump with > > wireguard enabled kernel module. > > ... > Thanks. Is there a __FreeBSD_version__ change that corresponds? It is not an exact match, but was bumped to 1400066 shortly after and should be a suitable value to check.
Re: FreeBSD current socket-src changed. Wireguard not compiling.
FreeBSD Current (14.0) http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/14.0/ sources from iso or github - no matther Changes to src are made by commit e7d02be19d40063783d6b8f1ff2bc4c7170fd434 Author Gleb Smirnoff Wed, 17 Aug 2022 21:50:32 +0300 (11:50 -0700) ср, 31 авг. 2022 г. в 21:18, Ed Maste : > > On Mon, 29 Aug 2022 at 12:18, Jason A. Donenfeld wrote: > > On Tue, Aug 23, 2022 at 12:26:21PM +0300, Michael Pro wrote: > > > Tonight after updating kernel freebsd current I got coredump with > > > wireguard enabled kernel module. > > > ... > > Thanks. Is there a __FreeBSD_version__ change that corresponds? > > It is not an exact match, but was bumped to 1400066 shortly after and > should be a suitable value to check.
Re: FreeBSD current socket-src changed. Wireguard not compiling.
On Tue, Aug 23, 2022 at 12:26:21PM +0300, Michael Pro wrote: > Tonight after updating kernel freebsd current I got coredump with > wireguard enabled kernel module. > > As is > https://reviews.freebsd.org/D36232 > D36232 protosw: refactor protosw and domain static declaration and > load (freebsd.org) > in wireguard-freebsd/src/support.h > ... > error = (*so->so_proto->pr_usrreqs->pru_sockaddr)(so, nam); > ... > replace to > ... > error = solisten_proto_check(so); > ... > Recompile - no more core dumps. All works now, perhaps... > > Is this the right way to fix the problem? Is this enough or should we > wait for a full-scale refactoring of the code? > > Thanks in advance for your reply. Thanks. Is there a __FreeBSD_version__ change that corresponds? Jason
FreeBSD current socket-src changed. Wireguard not compiling.
Tonight after updating kernel freebsd current I got coredump with wireguard enabled kernel module. As is https://reviews.freebsd.org/D36232 D36232 protosw: refactor protosw and domain static declaration and load (freebsd.org) in wireguard-freebsd/src/support.h ... error = (*so->so_proto->pr_usrreqs->pru_sockaddr)(so, nam); ... replace to ... error = solisten_proto_check(so); ... Recompile - no more core dumps. All works now, perhaps... Is this the right way to fix the problem? Is this enough or should we wait for a full-scale refactoring of the code? Thanks in advance for your reply.
Re: [ANNOUNCE] wireguard-freebsd snapshot v0.0.20220615 is available
Hi, I tested this snapshot on my setup for 10 days now and it works perfectly without any problems. My setup is fairly simple: central VPN server (FreeBSD 13.1), two servers on branch sites (each behind NAT) and a mobile Android client. VPN's are used for management type activities, so no high bandwidth or low latency requirements needed. So not sure if this setup is representative enough, but I'm happy with how it behaves. There is one small feature that I would like to see: My central server has multiple public IP addresses and sometimes Wireguard needs to initiate a connection to one of the branch servers. Unfortunately, there is no way to specify which source address to use for that. Currently it appears to use a random IP address from the outgoing interface (mostly the first IP address configured on the external interface). I would like to see the option to specify the IP address to be used for outgoing connections, that would be much more convenient when you have to deal with upstream firewalls. Not sure if this is a difficult thing to implement, but I would love to have it. Anyway, thanks for all the work you guys did on this great product!!! Kind regards, Frank Op 15-6-2022 om 16:11 schreef Jason A. Donenfeld: Hi, An experimental snapshot, v0.0.20220615, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * ci: add FreeBSD 12.3 and 13.1 * compat: update version to handle sbcreatecontrol() changes More fixes to the compat layer. * wg_noise: import hmac from crypto * crypto: inline blake2s convenience function A few crypto cleanups. This snapshot contains commits from: Jason A. Donenfeld, Joseph Mingrone, and Ed Maste. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20220615.tar.xz SHA2-256: ad6c42d20a7c0ad2989e729dd41ea5a6a019426b762dfd0c6417e340935cca82 Thank you, Jason Donenfeld >
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20220615 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20220615, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * ci: add FreeBSD 12.3 and 13.1 * compat: update version to handle sbcreatecontrol() changes More fixes to the compat layer. * wg_noise: import hmac from crypto * crypto: inline blake2s convenience function A few crypto cleanups. This snapshot contains commits from: Jason A. Donenfeld, Joseph Mingrone, and Ed Maste. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20220615.tar.xz SHA2-256: ad6c42d20a7c0ad2989e729dd41ea5a6a019426b762dfd0c6417e340935cca82 Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmKp6JIQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrgjTEAC5p4+osYbOTZG0uBlpAQ4ETTmlbJ8yV0av fZ/P0rLJkL2iYifgynnvO6GyKcRBb9rXa37OKb8+Mt350UU6mgwCB9FaGvWYxEGM cZYN+BVCN49DdFBtiLeCKnDe8ufeLahs87eSQDI1LDbiX9gYkuWR2cxHfN6b5wZX r/M80gxbOBR7/pFWm5Hplfj3CRyJkb4i89/BSJ+5Zmbo+qGd2CoKIxHsUXLQeatq 0LhJuFxIrol1Hm/L4pA5k4MTa/UmOVrDkB04cR7eCFp7yxEVKE9aol6tO1uSdD7S XEdNwsARQ2ehf7DoYdRV+1cl/rTnfaNo7Qp0fAu2+dmd7F0cwRtYCnP45Bcw1F0c 83fImSL4ck0MIFzo9brNK1sa15J+jsYsRsnlqLA0ObPBtOPf+ag/FSp1wfuAwHS6 ugcGNsaX+fOZ9yVf1dxpZHDOiw5TACW221TFb979jaCbHFncf1Ix2GTUeUrEImJa vuS1uhZOWwJza+ZqdsY2SyNPOewvaaXO5owBPZiD2Qz56Tv7c2v1mdYsT21qldah +cyffwJKi9if8eGLMH2ao9pfdjZ12Q93GR5nDACqRJtcB4lyMJbX5cMCESmHiCGX A4C3NUyfiaBxi3Pr4eFWeXuAZLCKmj+/GCoaTdlc34a/LJYuA9GVdR/aIcclRn9e a6zVVuSnPA== =Ktyi -END PGP SIGNATURE-
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20220614 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20220614, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * build: include compat.h for all files * compat: fix version stamp Two lingering compat issues. * if_wg: wg_peer_alloc and wg_aip_add: Use M_WAITOK with malloc A small cleanup to memory allocation and error path logic. * crypto: use OCF to encrypt/decrypt packets when supported * crypto: use when present * crypto: use curve25519 API from the kernel when available * crypto: harmonize with compat and clean up Crypto from OCF and FreeBSD's library code will now be used when available. This snapshot contains commits from: John Baldwin and Jason A. Donenfeld. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20220614.tar.xz SHA2-256: b048049561f18fd48ab3fa6455f9e0ecca25c6314880f48e68c56ceacf7f6719 Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmKoUwMQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrtWQD/9SrtU+zAXlB50zLYO1LhNexPLlgWMcAZsM qwJaIzxMdsXPCCfqLOfqxUXOAjPf7EWxFE8cj1iV6hLduzl0wnu84OZIxr7yvSFT qhk6aZxcgiUTpxIHa4WdJkxiqMNRafv6le33F233VWNBbJCn8f9JjxON8Fph0W/N DZ0X+fGwPJ25bnCWc//Cl7WhQulT7BxNi2jrsgpuJNmCGOeSRyfTh9ygDRX5WLD6 mZiIfqTs4+l9gVjeUcC7b49qbZeRtpsmePfaXp5fN2U6s5cyk76SbarCHD62aFlP OM4xdl07J+YhsXdfj98whE1RVTS8jFvz3iADKW0MuwDEeEcDCh4vpspiNerYDIvQ J+aFT8+zexmv9IdNQxYK3u9W1G6COz0wspBWixZGfeqfNZyPpXVOqgdKarsdFLnz wYFFz8FPdMlB8A8J+aehNqqsrH6rXbE4J2poz3upxrntKMxG+u0YB5pDoxY0JkAQ uvT3Ij/7CdsXYHnbKwFtiv9dmSzgSpFNOzSbffBWYb24i1UCCAbJqe5Eae9b7MIh 6PuaBp3YzlR3NHsXcliss2SbEMf1nf2WStTtKHDKW5hAyRdg4dqDMZxvPc8OU6vC BF1gVIEYC9wRLaNfxU/mW5ugdiN0b3u6uT9ELrN/mV56z45xPKmedpDNXKHqz3b2 JF9KsyRjDg== =xgbb -END PGP SIGNATURE-
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20220610 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20220610, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * if_wg: wg_queue_len: remove locking * if_wg: wg_queue_delist_staged: use more standard STAILQ_CONCAT * if_wg: wgc_get/set: use M_WAITOK with malloc() * if_wg: wg_clone_create: Use M_WAITOK with malloc * wg_cookie: ratelimit_init: use callout_init_mtx * if_wg: wg_mbuf_reset: don't free send tags * if_wg: wg_module_init: clean up more if the self tests fail Numerous cleanups. * if_wg: avoid scheduling excessive tasks for encryption/decryption A nice performance improvement - John reports double on iperf3 TCP throughput. * crypto: return an error code from mbuf crypt routines Preparation for the OCF work John's been working on. * if_wg: account for input function returning a boolean * if_wg: account for added argument to sbcreatecontrol These allow building on recent FreeBSD 14 main. * if_wg: do not use continue statement on \!VIMAGE * build: only include compat.h for if_wg.c and fix build with an obj directory These allow building in different configurations. This snapshot contains commits from: John Baldwin and Jason A. Donenfeld. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20220610.tar.xz SHA2-256: 518779c383e8087a60a7ec7b0969158a48b2f19a34a13b62683c982960295f17 Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmKjfhcQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrszsD/9rSo3Cqx8AqilVII4dpbxGzHXnkSCetUc8 e0nBLtvSZ0C/MiHSJtOD1iwdz8ndPX57P2VUdBVV/xmSeWzblfWDyB2nOoiLUq4P EoJaAkMD1isErVdJizlq0QQEq6UFegBcb9ttna33IZJm67uRXICzWq55yl/pJMSV zhsv2cQOKK8S9dUKgBtY31YKnDMZLvbtJbPP5hiDyCmrSVYrQKFd6owhCubRzGKE thNP/+rDgitz7vFB8Uu84Ib/dY8dVPjFbK/k3DqTdSKhefI4uc0y1buM8mV96zGi pXXYqi1qrgnMVpO2plhR04thACbAqT1Gbsx402q48LeJ7cD+oL0WxTJ2H9gEv/tA lJ4e7YnWt3zyu4SW8CvDPbVfWLVTTHR9jvV+/qJp4SVbFsDkmdHT7mWURRUtwawD C0p+wPlBcqdENXVtFmvaKeqclxCXFUkIC1SorIStKhYDPsFdRexOsiXAqAVdXmvJ ZdeX8BJWHXz7cxGNdE96ep6qBMAPJ2+9f3nLIoG/1ca0hWvDpWTGbepoekv0Ab0/ gwXqlxUfOmCjjLBRA4vrp0JJpWJJaWWdQSVKmeDwn/z7h/kASjeL1Z7gDlkEq3yI hKg47SUqupG2/iSoL8EUglAPi1o498BhZANtf3LgYH3z+fd7GU4k/ugfp2KKjsOq +dNT9WYX9Q== =yY95 -END PGP SIGNATURE-
[PATCH] wg-quick: freebsd: save mask of interface in save_config
Other than the obvious benefits, this prevents "ifconfig: WARNING: setting
interface
address without mask is deprecated, default mask may not be correct." from
appearing
on the next start after save.
Signed-off-by: Dan Ponte
---
src/wg-quick/freebsd.bash | 12 +++-
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/src/wg-quick/freebsd.bash b/src/wg-quick/freebsd.bash
index b529ab2..5864a17 100755
--- a/src/wg-quick/freebsd.bash
+++ b/src/wg-quick/freebsd.bash
@@ -341,13 +341,15 @@ set_config() {
}
save_config() {
- local old_umask new_config current_config address cmd
+ local old_umask new_config current_config network cidr address cmd
new_config=$'[Interface]\n'
- { read -r _; while read -r _ _ _ address _; do
- new_config+="Address = $address"$'\n'
+ { read -r _; while read -r _ _ network address _; do
+ cidr=(${network//\// })
+ new_config+="Address = $address/${cidr[1]}"$'\n'
done } < <(netstat -I "$INTERFACE" -n -W -f inet)
- { read -r _; while read -r _ _ _ address _; do
- new_config+="Address = $address"$'\n'
+ { read -r _; while read -r _ _ network address _; do
+ cidr=(${network//\// })
+ new_config+="Address = $address/${cidr[1]}"$'\n'
done } < <(netstat -I "$INTERFACE" -n -W -f inet6)
while read -r address; do
[[ $address =~ ^nameserver\ ([a-zA-Z0-9_=+:%.-]+)$ ]] &&
new_config+="DNS = ${BASH_REMATCH[1]}"$'\n'
--
2.36.0
Re: Patches against wireguard-freebsd
On Tue, Apr 5, 2022 at 5:49 PM John Baldwin wrote: > > I have a series of patches against the FreeBSD driver available on > the jb-wip branch which I believe is (mostly) a candidate for merging > to master. The changes in the branch include: > > - 9d1881a Only include compat.h for if_wg.c and fix build with an obj > directory. > > This patch permits building the module as part of a kernel build via > the LOCAL_MODULES hook in 'make buildkernel'. > LGTM. > - 2c4b941 wg_queue_len: Remove locking. > - 61b401e wg_queue_delist_staged: Use more standard STAILQ_CONCAT. > > Misc small fixes, generally cosmetic. > I wanted to argue against 2c4b941, but it occurred to me that since the caller's not holding the lock we don't have any guarantees about the state of the queue by the time we use the result anyways. LGTM. > - d746eed wgc_get/set: Use M_WAITOK with malloc(). > - 9223fbb wg_clone_create: Use M_WAITOK with malloc. > - 9095ebe wg_peer_alloc and wg_aip_add: Use M_WAITOK with malloc. > > The ioctl path in FreeBSD generally uses M_WAITOK rather than M_NOWAIT > (e.g. the non-smalldata case in sys_ioctl()). This slightly simplifies > the code by avoiding additional edge cases to handle. It is also true > that FreeBSD admins do not generally expect 'ifconfig create' to fail > non-deterministically (that is, they only expect failure for things > like invalid arguments, missing kernel module, etc.). > LGTM. > - 4fbba97 wg_mbuf_reset: Don't free send tags. > - 0a5fa77 ratelimit_init: Use callout_init_mtx. > > More misc small fixes. LGTM. > > - a679db5 DNM: Add counters for the times en/decrypt tasks do no work. > > This is the one commit that I don't think is a merge candidate, instead > it adds some counters useful for evaluating the effect of the next > commit. > > - 89f91dc Avoid scheduling excessive tasks for encryption/decryption. > > There is more detail in the commit log, but this commit changes the > scheduling of encryption/decryption tasks to match the behavior of the > WireGuard driver in Linux instead of the current approach of > scheduling a task on all available CPUs for every packet. In my > performance benchmarks of this series, this commit had the single > largest effect of any of the changes. My benchmark consisted of > running iperf across a tunnel between two jails on the same host (an > X1 Carbon laptop with 8 CPU threads). This commit generally resulted > in a doubling of throughput for both UDP and TCP with 1, 2, 4, or 8 > streams. > > To better explain why this change matters, I sampled the counters > added in the previous commit for a sample run of iperf with a single > TCP stream. The "empty" counters count the number of tasks which ran > on a CPU but had no work to do, the "work" counters count the number > of tasks which encrypted or decrypted at least one packet. > > Using the current code gave the following counts: > > hw.wg.encrypt_work: 992858 > hw.wg.encrypt_empty: 6274830 > hw.wg.decrypt_work: 1114235 > hw.wg.decrypt_empty: 7064707 > > encrypt efficiency: 13.7% > decrypt efficiency: 13.6% > > The efficiency is close to the 12.5% worst-case for an 8 CPU system. > > Using the code in this commit gave the following: > > hw.wg.encrypt_work: 1486616 > hw.wg.encrypt_empty: 783377 > hw.wg.decrypt_work: 1880567 > hw.wg.decrypt_empty: 657807 > > encrypt efficiency: 65.5% > decrypt efficiency: 74.1% > > Note: The increased "work" counts here are a result of the increased > throughput > > In addition, a user recently mailed Jason and I directly to say that > this commit greatly reduced the power usage for a WG endpoint in an > ESXi VM putting the FreeBSD VM nearly on par with a Linux VM > performing the same work. > Nice! LGTM > - 4e0478f wg_module_init: Clean up more if the self tests fail. > - b885223 Return an error code from mbuf crypt routines. > > Small fixes preparing to use crypto support from FreeBSD. > LGTM. > - ce85779 Use OCF to encrpyt/decrypt packets when supported. > - 8ad55a8 Use when present. > - 447abb Use curve25519 API from the kernel when available. > > Use crypto support from FreeBSD's kernel on new-enough versions (the > OCF bits are available on 13.0-stable and later, the rest are only > present in 14.0-current). FreeBSD's kernel does not currently provide > a suitable API for Blake2 that matches WireGuard's needs, but does > provide suitable APIs for the other crypto algorithms used by > WireGuard in 14.0-current. > Seems to LGTM. Thanks! Kyle Evans
Patches against wireguard-freebsd
I have a series of patches against the FreeBSD driver available on the jb-wip branch which I believe is (mostly) a candidate for merging to master. The changes in the branch include: - 9d1881a Only include compat.h for if_wg.c and fix build with an obj directory. This patch permits building the module as part of a kernel build via the LOCAL_MODULES hook in 'make buildkernel'. - 2c4b941 wg_queue_len: Remove locking. - 61b401e wg_queue_delist_staged: Use more standard STAILQ_CONCAT. Misc small fixes, generally cosmetic. - d746eed wgc_get/set: Use M_WAITOK with malloc(). - 9223fbb wg_clone_create: Use M_WAITOK with malloc. - 9095ebe wg_peer_alloc and wg_aip_add: Use M_WAITOK with malloc. The ioctl path in FreeBSD generally uses M_WAITOK rather than M_NOWAIT (e.g. the non-smalldata case in sys_ioctl()). This slightly simplifies the code by avoiding additional edge cases to handle. It is also true that FreeBSD admins do not generally expect 'ifconfig create' to fail non-deterministically (that is, they only expect failure for things like invalid arguments, missing kernel module, etc.). - 4fbba97 wg_mbuf_reset: Don't free send tags. - 0a5fa77 ratelimit_init: Use callout_init_mtx. More misc small fixes. - a679db5 DNM: Add counters for the times en/decrypt tasks do no work. This is the one commit that I don't think is a merge candidate, instead it adds some counters useful for evaluating the effect of the next commit. - 89f91dc Avoid scheduling excessive tasks for encryption/decryption. There is more detail in the commit log, but this commit changes the scheduling of encryption/decryption tasks to match the behavior of the WireGuard driver in Linux instead of the current approach of scheduling a task on all available CPUs for every packet. In my performance benchmarks of this series, this commit had the single largest effect of any of the changes. My benchmark consisted of running iperf across a tunnel between two jails on the same host (an X1 Carbon laptop with 8 CPU threads). This commit generally resulted in a doubling of throughput for both UDP and TCP with 1, 2, 4, or 8 streams. To better explain why this change matters, I sampled the counters added in the previous commit for a sample run of iperf with a single TCP stream. The "empty" counters count the number of tasks which ran on a CPU but had no work to do, the "work" counters count the number of tasks which encrypted or decrypted at least one packet. Using the current code gave the following counts: hw.wg.encrypt_work: 992858 hw.wg.encrypt_empty: 6274830 hw.wg.decrypt_work: 1114235 hw.wg.decrypt_empty: 7064707 encrypt efficiency: 13.7% decrypt efficiency: 13.6% The efficiency is close to the 12.5% worst-case for an 8 CPU system. Using the code in this commit gave the following: hw.wg.encrypt_work: 1486616 hw.wg.encrypt_empty: 783377 hw.wg.decrypt_work: 1880567 hw.wg.decrypt_empty: 657807 encrypt efficiency: 65.5% decrypt efficiency: 74.1% Note: The increased "work" counts here are a result of the increased throughput In addition, a user recently mailed Jason and I directly to say that this commit greatly reduced the power usage for a WG endpoint in an ESXi VM putting the FreeBSD VM nearly on par with a Linux VM performing the same work. - 4e0478f wg_module_init: Clean up more if the self tests fail. - b885223 Return an error code from mbuf crypt routines. Small fixes preparing to use crypto support from FreeBSD. - ce85779 Use OCF to encrpyt/decrypt packets when supported. - 8ad55a8 Use when present. - 447abb Use curve25519 API from the kernel when available. Use crypto support from FreeBSD's kernel on new-enough versions (the OCF bits are available on 13.0-stable and later, the rest are only present in 14.0-current). FreeBSD's kernel does not currently provide a suitable API for Blake2 that matches WireGuard's needs, but does provide suitable APIs for the other crypto algorithms used by WireGuard in 14.0-current. -- John Baldwin
Re: [ANNOUNCE] wireguard-freebsd snapshot v0.0.20211105 is available
I've done some preliminary tests with the new if_wg kernel module on amd64 with freebsd 13-p4 and 14. It works fine AFAICT. My previous issue's with wireguard-freebsd are also resolved. Good work! Thanks Peter
Re: wireguard-freebsd handshaking issue upon underlying WAN
On Tue, Nov 9, 2021 at 11:19 AM Ryan Roosa wrote: > > On Wed, Oct 27, 2021 at 7:45 PM Ryan Roosa wrote: > > > > On Tue, Oct 26, 2021 at 5:29 AM Jason A. Donenfeld wrote: > > > > > > Hi Ryan, > > > > > > Thanks for the report. Kyle saw your reddit post earlier and tracked > > > this down, I think/hope, to a bug in the state machine cranking. I > > > committed the fix here -- https://w-g.pw/l/yQTw -- which will be part > > > of the next snapshot. Hopefully that will fix the issue, but if it > > > doesn't, please do update this thread so we can keep searching. > > > > > > Regards, > > > Jason > > > > Hi Jason, > > Thank you very much for this! I received word from the OPNSense team > > that the referenced snapshot should be made available in OPNSense > > 21.7.5. I will test and provide feedback just as soon as I can get on > > the aforementioned OPNSense release which includes the fix. > > > > Cheers, > > -Ryan > > > > Just wanted to provide some feedback that pfSense development > snapshots of 2.6.0 running WireGuard package v0.1.5_2 include the fix > and there I have validated that removing WAN connectivity at various > intervals up to 10 minutes no longer impacts subsequent handshaking > once the connection is restored. I have not yet tested on OPNSense but > I imagine the results will match once I do (if not I will reach out). > Thanks to everyone for their efforts on resolving this one, I really > appreciate it. > > -Ryan > That's good to hear, thanks for following up! :-) Kyle Evans
Re: wireguard-freebsd handshaking issue upon underlying WAN
Just wanted to provide some feedback that pfSense development snapshots of 2.6.0 running WireGuard package v0.1.5_2 include the fix and there I have validated that removing WAN connectivity at various intervals up to 10 minutes no longer impacts subsequent handshaking once the connection is restored. I have not yet tested on OPNSense but I imagine the results will match once I do (if not I will reach out). Thanks to everyone for their efforts on resolving this one, I really appreciate it. -Ryan On Wed, Oct 27, 2021 at 7:45 PM Ryan Roosa wrote: > > Hi Jason, > Thank you very much for this! I received word from the OPNSense team > that the referenced snapshot should be made available in OPNSense > 21.7.5. I will test and provide feedback just as soon as I can get on > the aforementioned OPNSense release which includes the fix. > > Cheers, > -Ryan > > On Tue, Oct 26, 2021 at 5:29 AM Jason A. Donenfeld wrote: > > > > Hi Ryan, > > > > Thanks for the report. Kyle saw your reddit post earlier and tracked > > this down, I think/hope, to a bug in the state machine cranking. I > > committed the fix here -- https://w-g.pw/l/yQTw -- which will be part > > of the next snapshot. Hopefully that will fix the issue, but if it > > doesn't, please do update this thread so we can keep searching. > > > > Regards, > > Jason
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20211105 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20211105, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * compat: taskqueue draining was backported to stable/13 Compilation should resume working with the latest stable/13. * if_wg: bump keepalive timers unconditionally on send Important fix to the timer state machine, which matches behavior on Linux and elsewhere. * if_wg: protect in6_mask2len with INET6 Fix compilation on 12 with !INET6. This snapshot contains commits from: Jason A. Donenfeld. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20211105.tar.xz SHA2-256: 454f2cf15329ccf40da15f8df61f8d70bb6d382efdfe4a5899ca16474f91903f Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmGFQrYQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrucXD/9cnVv050l3zzSKriaIBnrZfVgUVfKtZWmg KVmc/tKViFph1Mnx+n8EAHAf05n93v7J9b65l7syb7OUnv8IseoG16Iw38erbEsu kkaF1BK9lJ4pLjBKtAcTnzwipm8aXf6Ge+HPwzRO30mOL11DnD1gbLqrFwIKQHUC HI7K0Bv+6t54t17vnaVvO51JJME4NpRDQ0zqIMROROGIRYJLbnivZRHcIxN4lQuJ IlY8Bzj3dyzYKJqdB6mvAm2XShGx4265Qo6uo31Qyuo8SJVXFnb5WEnc30Aw1tHc Fi3cqWnia84TW/X+PyH3wHhRjC4JJNms7qpIvE4GN5uil6Z2sqfOF7WxkVAhiSX9 RltirnHuw8KELsomYmje2wDcDE//Yxucj9cV0/qOqCzhvgiuggNlIIRdQHkbVhsm RAAh6N9fcFWxWQTfXbWqToFVOImzPBS7r1ewYZbT8VFvp41CtSArio+yc5feVbei s78KMWPhJWX5s9cxXoyeJAL9YJAaeUvGvB1d0590+N8iC/jQwJrvViCd5fqlzXLe 7x6Ge02ekkYsWc+MubfEm1zFYGMrTxmCRfElIZz48BweuzMtOQRzRlGQQR88NyUx vt71YDb8xSC1/zIeQdlfOgvqvtcPDC9ZkneHxWOV39Iwc7W9KfopXOTUVgfC4NEs oDs2uLmsSQ== =Q4TN -END PGP SIGNATURE-
Re: Wireguard on FreeBSD - a few questions
Hi Kyle, 1) Is it possible on FreeBSD to enable some kind of logging? I did made a small configuration error with my first client and it was hard to find the error, because there does not seem to be any logging at all. Some logging information would be appreciated and probably wold have pointed me faster to the fact that I needed to switch two keys in my config. If you set 'debug' on the interface (`ifconfig wg0 debug`) then it'll write some useful bits to syslog for your perusal. O.k. good to know this. It would be even better if this was documented, I think a if_wg manual page for FreeBSD would be appropriate. 2) I noticed that Wireguard uses a wildcard to listen to all IP addresses on my multi-homed machine on his dedicated UDP port. I would prefer if Wireguard would only bind to the specific IP address on the outside interface that is designated for that use. Is this possible? I think it is useful if you could bind Wireguard to use/listen on a specific IP address, instead of the wildcard. For example, for my tests I used a secondary (alias) IP address on a server as the entry point for Wireguard tunnels. However, if the server starts a session to the client (or tries to check if the client is still alive), it uses the primary interface address instead. Binding it to a specific IP address would solve this. Kind regards, Frank
Re: Wireguard on FreeBSD - a few questions
On Wed, Nov 3, 2021 at 5:55 AM Frank Volf wrote: > > > Hi, > > This weekend I installed Wireguard on FreeBSD 13.0 and until now > everything seems to work fine (I use the kernel module). > Installation and configuration was easy and connecting with the Android > app works great as well. > Excellent, that's good to hear! :-) > I do have a few questions. > > 1) Is it possible on FreeBSD to enable some kind of logging? I did made > a small configuration error with my first client and it was hard to find > the error, because there does not seem to be any logging at all. Some > logging information would be appreciated and probably wold have pointed > me faster to the fact that I needed to switch two keys in my config. > If you set 'debug' on the interface (`ifconfig wg0 debug`) then it'll write some useful bits to syslog for your perusal. > 2) I noticed that Wireguard uses a wildcard to listen to all IP > addresses on my multi-homed machine on his dedicated UDP port. I would > prefer if Wireguard would only bind to the specific IP address on the > outside interface that is designated for that use. Is this possible? > > 3) Final question: is it possible on the server side to restrict the > destinations that clients can connect to it? I know, that I can set the > AllowedIPs on the client side to restrict that, but that setting can be > changed at the client side. It would be nice if I could restrict > destinations at the server side (so client X can only connect to an IP > address of an internal server that it needs access to but nothing else). > I can probably use a state full packet filtering firewall for this, but > it would it be possible to configure this on the Wireguard server side > as well? > For these last two, I'll defer to somebody else -- I'm not aware of any such functionality on other platforms, but wireguard-freebsd will follow suit if this is or will become an accepted concept elsewhere. > That said, I'm pleased with the first test results of Wireguard on > FreeBSD and hopefully it keeps on running fine. Great product! > Great, thanks for testing! =) Kyle Evans
Wireguard on FreeBSD - a few questions
Hi, This weekend I installed Wireguard on FreeBSD 13.0 and until now everything seems to work fine (I use the kernel module). Installation and configuration was easy and connecting with the Android app works great as well. I do have a few questions. 1) Is it possible on FreeBSD to enable some kind of logging? I did made a small configuration error with my first client and it was hard to find the error, because there does not seem to be any logging at all. Some logging information would be appreciated and probably wold have pointed me faster to the fact that I needed to switch two keys in my config. 2) I noticed that Wireguard uses a wildcard to listen to all IP addresses on my multi-homed machine on his dedicated UDP port. I would prefer if Wireguard would only bind to the specific IP address on the outside interface that is designated for that use. Is this possible? 3) Final question: is it possible on the server side to restrict the destinations that clients can connect to it? I know, that I can set the AllowedIPs on the client side to restrict that, but that setting can be changed at the client side. It would be nice if I could restrict destinations at the server side (so client X can only connect to an IP address of an internal server that it needs access to but nothing else). I can probably use a state full packet filtering firewall for this, but it would it be possible to configure this on the Wireguard server side as well? That said, I'm pleased with the first test results of Wireguard on FreeBSD and hopefully it keeps on running fine. Great product! Kind regards, Frank
Re: wireguard-freebsd handshaking issue upon underlying WAN
Hi Ryan, Thanks for the report. Kyle saw your reddit post earlier and tracked this down, I think/hope, to a bug in the state machine cranking. I committed the fix here -- https://w-g.pw/l/yQTw -- which will be part of the next snapshot. Hopefully that will fix the issue, but if it doesn't, please do update this thread so we can keep searching. Regards, Jason
wireguard-freebsd handshaking issue upon underlying WAN
Hello, First off, I want to say thank you for the FreeBSD kernel module work as it is greatly appreciated by myself and many others running *sense firewalls :) Generally wireguard-freebsd (wireguard-kmod 0.0.20210606_1) is running quite well in my experience however, there is one issue which I have been able to reproduce consistently: when the underlying WAN connection that a tunnel is using is disrupted for the span of time amounting to two missed handshake attempts (~4-5 minutes giving the ~2 minute average of handshake attempts), the tunnel will never handshake again upon subsequent WAN restoration. This is the case even if one resets the tunnel with 'wg-quick down ; wg-quick up' or restarts the underlying OS (tried with both the latest stable versions of pfSense and OPNSense community). For reference I am using a keep alive value of 30 seconds in this scenario. The only thing I've been able to do to get an existing tunnel configuration handshaking with a peer endpoint again after its Internet connection has been disrupted (outside of a complete removal and rebuild) is to arbitrarily change the configured tunnel's listening port (ex. 51820 to 51821 etc.). Upon saving and application of the port change, the tunnel then handshakes with the peer endpoint again immediately. Given the symptom, it seems there may be some issue surrounding tunnel handshaking resiliency when the underlying WAN drops out unexpectedly for an extended period. If there is any way to look into this to improve upon it so that after a 5+ minute internet outage a tunnel could resume handshaking on its own without manual intervention, this would be greatly appreciated. I've got a 'bandaid' script running every 5 minutes currently which checks the peer's handshake age and then changes the tunnel listen port arbitrarily to restore connectivity then changes it back after 5 minutes of successful handshaking but obviously this is less than ideal. As an additional data point I found if I switched the port and tried to switch it back before another 5 minutes had passed, it would stop handshaking again so there seems to be something special around the 5 minute number regarding handshakes. Not sure if this is helpful or not but thought I would include it. Thank you in advance for looking into this and if there is any additional information I can provide which may be of assistance I would be happy to provide it. Cheers, Ryan Roosa
Re: Certain private keys being mangled by wg on FreeBSD
On Tue, Jun 8, 2021 at 1:00 PM ben edmunds wrote: > By not showing this to the user to avoid confusion we actually would > create confusion in this scenario as the kernel module is performing the > clamping but the user would have no knowledge of this and leads to > issues being opened that are a non issue. The aim is not to show the > users anything about clamping unless the key needs to be clamped as it > was not clamped already. I think you are making a mistake, and introducing users to the concept of clamping will just breed confusion. > I belive it is key to remember that pfSense is not an end user > application/tool and designed to be used by admins & network engineers You made that same point on some Github bug report; it's not news to me, and it still doesn't change my point of view. Introducing excessive complexity and superfluous technical information results in problematic decisions, configurations, and decision calculuses, even for the most powerful of power users. In particular, here, I think it's only going to sow confusion and bad information to expose users to contingent details about "valid private keys" and "less valid private keys" with weird nerdy language like "unclamped". Because the fact is, any 256-bit bitstring generated from a csprng is a fine private key.
Re: Certain private keys being mangled by wg on FreeBSD
The issue here for pfSense is that the private key will be viewable just like it is within native wireguard clients in the peer config options and needs to be viewable here for admin and debug purposes. With regards to clamping and hiding this from users its tricky as it leads to red heroin issues as people debug the tunnels via showcase for example and will see a different key to which they entered in the UI. So the only logical option is to: 1) inform the admin that the key has been clamped 2) show the admin the clamped key which they can see whilst debugging. By not showing this to the user to avoid confusion we actually would create confusion in this scenario as the kernel module is performing the clamping but the user would have no knowledge of this and leads to issues being opened that are a non issue. The aim is not to show the users anything about clamping unless the key needs to be clamped as it was not clamped already. I belive it is key to remember that pfSense is not an end user application/tool and designed to be used by admins & network engineers so should be considered power users who are capable of being exposed to more information. Regards Tigger2014
Re: Certain private keys being mangled by wg on FreeBSD
On 6/7/21, Christian McDonald wrote: > One byproduct of this exercise was some code that I whipped > up that can at least detect a clamped vs unclamped key. This might > prove useful for informing a user of what is going on and thus > eliminating this class of erroneous bug report entirely. I'd recommend *not* introducing users to weird ideas like clamping or key transformation. While learning new concepts and bit masking in PHP is undoubtedly fun, those concerns shouldn't be user-facing. There's nothing wrong or dangerous about unclamped scalars passed to a proper 25519 implementation, because the implementation will clamp on input. Throwing an "X-vs-unX" distinction to users will just result in pointless fear mongering nonsense. Instead just communicate the identity of an interface by its public key, rather than its private key. If you're not willing to hide or mask private keys (which you really should), then at least deemphasize them?
Re: Certain private keys being mangled by wg on FreeBSD
Ah that makes sense. I spent some quality time playing with the bit arithmetic and I see what you mean now. Thanks for that snippet and direction. One byproduct of this exercise was some code that I whipped up that can at least detect a clamped vs unclamped key. This might prove useful for informing a user of what is going on and thus eliminating this class of erroneous bug report entirely. I'm really not sure hiding the private keys entirely in the UI is the right thing to do, especially if it seems that key generators should really be pre-clamping keys (thanks for reaching out to Mullvad btw) and not dishing out unclamped keys to begin with. On Sun, Jun 6, 2021 at 12:21 PM Jason A. Donenfeld wrote: > > On 6/6/21, Christian McDonald wrote: > > Would it not be better for wg to just fail outright instead of > > transforming a poorly generated key entered by a user, regardless of > > where the key came from? Especially if that problematic key passes the > > regex validation that was provided in another thread in this email > > list? > > No, it would not be better. There is nothing wrong with using those > keys. They're not "poorly generated" or "problematic" or dangerous in > the least. This is only a concern with your UI. > > The kernel is doing the correct thing -- clamping keys -- and > displaying an unambiguous identifier to the user: the key that it will > actually be using. > > I suspect the best thing to do for your UI would be to hide private > (and preshared) keys, and only show public keys, unless explicitly > exported into a config file. This not only reduces potential confusion > with this issue, but mitigates another potential footgun down the > line. It's also what wg(8)'s show command does by default (while > showconf will export all). -- R. Christian McDonald M: (616) 856-9291 E: rcmcdonal...@gmail.com
Re: Certain private keys being mangled by wg on FreeBSD
On 6/6/21, Christian McDonald wrote: > Would it not be better for wg to just fail outright instead of > transforming a poorly generated key entered by a user, regardless of > where the key came from? Especially if that problematic key passes the > regex validation that was provided in another thread in this email > list? No, it would not be better. There is nothing wrong with using those keys. They're not "poorly generated" or "problematic" or dangerous in the least. This is only a concern with your UI. The kernel is doing the correct thing -- clamping keys -- and displaying an unambiguous identifier to the user: the key that it will actually be using. I suspect the best thing to do for your UI would be to hide private (and preshared) keys, and only show public keys, unless explicitly exported into a config file. This not only reduces potential confusion with this issue, but mitigates another potential footgun down the line. It's also what wg(8)'s show command does by default (while showconf will export all).
Re: Certain private keys being mangled by wg on FreeBSD
Would it not be better for wg to just fail outright instead of transforming a poorly generated key entered by a user, regardless of where the key came from? Especially if that problematic key passes the regex validation that was provided in another thread in this email list? If not, what would be an appropriate solution to catch situations like this and in turn alert users? This seems like it could be a larger discussion on interoperability, especially when dealing with keys that are being generated by VPN providers. Granted, this certainly isn’t my area of expertise. Though, the behavior is just unexpected (and confusing) from an end user perspective. On Sun, Jun 6, 2021 at 11:09 AM Jason A. Donenfeld wrote: > > It looks like whatever is generating those private keys is not > clamping them. Specifically, all private keys should undergo this > transformation: > > key[0] &= 248; > key[31] = (key[31] & 127) | 64; > > In your case, your `Lm` prefix (first byte: 0x2c) is being anded with > 248, and thus turns into KG (first byte: 0x28). > > The kernel properly clamps the keys on input, though, in case > generators forget to clamp them. So what you're seeing is correct > behavior. -- R. Christian McDonald M: (616) 856-9291 E: rcmcdonal...@gmail.com
Re: Certain private keys being mangled by wg on FreeBSD
It looks like whatever is generating those private keys is not clamping them. Specifically, all private keys should undergo this transformation: key[0] &= 248; key[31] = (key[31] & 127) | 64; In your case, your `Lm` prefix (first byte: 0x2c) is being anded with 248, and thus turns into KG (first byte: 0x28). The kernel properly clamps the keys on input, though, in case generators forget to clamp them. So what you're seeing is correct behavior.
Certain private keys being mangled by wg on FreeBSD
Hi Jason, I've got an interesting case here. I've got what appears to be a few private keys provided by Mullvad that are being mangled by wg(8) on FreeBSD12.2 (pfSense 2.5.1+). Private key ```Lm0BA4UyPilHH5qnXCfr6Lw8ynecOPor88tljLy3ALk=``` ... becomes ```KG0BA4UyPilHH5qnXCfr6Lw8ynecOPor88tljLy3AHk=``` on wg showconf another example is ```zQwldrChCYP3P2fZeTtHmj3c/88hxM+TaLIq1mJ28ZY=``` turns into ```yAwldrChCYP3P2fZeTtHmj3c/88hxM+TaLIq1mJ28VY=``` I must have just gotten lucky with the keys Mullvad provided to me, as those worked and continue to work fine for me. ref : https://github.com/theonemcdonald/pfSense-pkg-WireGuard/issues/105 Best, -- R. Christian McDonald E: rcmcdonal...@gmail.com
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20210606 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20210606, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * netns: use `exit 0` for early exit * ci: add a Cirrus-CI config file to build + smoke test * if_wg: pass back result of selftests and enable in CI * netns: trim test to working parts and rework jail logic * netns: account for FreeBSD 12 quirks * netns: use massive datagrams * if_wg: do not crash if deiniting before vnet is up * ci: test on 12.1 and 12.2 We now have the vnet CI running on every commit on 12.1, 12.2, and 13.0, and so a decent amount of work went into enabling this and tidying up the tests we run. * global: destroy rwlocks and mtxs * global: replace rwlock with mtx if never rlocked This uses some fancy SmPL to simplify our lock types when possible. * compat: account for lack of CSUM_SND_TAG on ≤12.2 * if_wg: add braces for 12.1 compiler warning Usual compat work. This snapshot contains commits from: Jason A. Donenfeld and Ed Maste. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20210606.tar.xz SHA2-256: d7501666030619e4ee86e68d2b9fd6a718be6541be83bf726414b821c60383d0 Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmC8z7UQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrpTQD/9VenrT33AbdVf688t11dyKD6S/aZ0XLsF8 Qvjpv8tJuLOOHqSm1d7c6UKdsDrc9okbAvQvHoiX/VhU9CxBKzQoK74iTPiO+bL4 Y7Z+fe+25vzjyzJmW8KhPymgQzfhw2JW7KwjtwXqh4NnZ3zKFoaY/9hWBnZAIF2o pTrjBE6jL5W6smFDyHsa6L0f9hfOJ7PUvNRj9Tym9VQiGxY0Q3z7N845n2gvgwJD J2UKTugz12zdVt/sKSpqLaLrbGaonqY+10SdLVUNO19+6iIx7hRBPw+BKTakUVGl aqji3kM7nsszWIt9yEKzdmG62oIMfZg0524yU3ZW9f38TyN2x/5bNoaRcHxNvcA/ SJ+25mVsSKNiNvVho89e6NxSK9tqywkXCbVd62p6KyOKdJSOvsvuT5LhPzlA6RMz Oli8bpZHQucxNHFzNdnuojo44/G9RlKriCKqMtPY2BH+LJOswzFG071HcWn785hR 96wDkxPvL4QqD/7tHssyPna33Ees/IRSlnYTXaZ5L9wwBTilI0XnKO/X+whpm4jj 21rXfXEXlCbe9tA90H6tQS/sHcn9Ezj2+5MEgCxhZ6yRjeibDZvKR8kCuIcXJxAi /Y1iF5wyVvWSndNKruvUV7t4+w39WzzxaLNokZ3UZYysmLBvwbf+Ddl5yVUxtEkr HmA9Tocw9g== =eYnb -END PGP SIGNATURE-
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20210503 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20210503, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * if_wg: put event notifiers in main loop Should prevent connection stalls at sustained high speeds. * wg_noise: set handshake to dead before removing keypair Fixes an unlikely crash. * if_wg: destroy interfaces before uma zone Allows for removing the module while traffic is flowing. This snapshot contains commits from: Jason A. Donenfeld. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20210503.tar.xz SHA2-256: 96b2bceadc05dac2f026e203d1c0d25f114491afc8fbfc8d6f71c3de7f4a22f1 Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmCTtMEQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4Drtf0D/40Pz8jc7sNfLZYWYrQfra4gIgkScBkQFxU FqLnh3sG0Vc3lcLuhdXunowgTGXd7tTUW4HlQ5RcqtJimtogBI1bO8ZYo8lcYAJE wJPl7Jn5BEVDRKYUuXpDhtto5zpuWtyao/j1m3fyStnrZ19xqCMCsdCc5ULq2q7j m2Xo0z+ImqMM1l/PgnAM//aG1RvhJmzEQrMrYS6eBvTvphoDRjMztWT0UrkSM8EO Cn3EqH55KZZHz40654jLb8SxlDj8gJvFP3V0zzHvfcdDv/A3Shu1Mp8hSsmA0Da/ 280EZG9CjaAMsyEBBXnZGoFxgXq4xPGNVOC1N9TMnqUbh6q56yU/ABfYgC6yPD84 v//b6mJZ9DYITUq7t7h1TGgfAazD76C3O4v8JIHmRXg3xygsItWKO8QtoEWdtsBX Tqi45h5wq6Xn/dll2RkkeTkFa4lUKeSgpM3gWHNnBVHk8gNOHK1b9oujen59eOdK nN4vQ3DjXskDHMp4qabf9ffqD22C/BWQgBH8BTwmuAjSZq38SFBn/VGCzJwRR8OC VPYmlgo3aP6lbrIQqqSgZS8pGjR9m6w/aWuwQD+JOtEiiHU+VkivsKgeYCQAlctz /XvjxAqviEnxUEnJRDBmieGlLSEo+ED3F9Iu16n6E1WewdYhGh80G+KqlYTDQuvR c+Tmm2i3DA== =oVK+ -END PGP SIGNATURE-
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20210502 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20210502, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * if_wg: defragment mbufs early on * if_wg: ensure packet is not shared before writing These should improve performance and make sure everything works with functions like sendfile(2). * if_wg: return to m temporary variable style * if_wg: pad packets properly * if_wg: don't memcpy data for no reason * if_wg: don't double increment error counter * wg_cookie: zero before init in selftest for witness * wg_noise: cleanup counter algorithm Various small correctness fixes. This snapshot contains commits from: Jason A. Donenfeld. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20210502.tar.xz SHA2-256: cd59a0a711308083e40ac5e7e26abc63c8b1f4615a9d66f73f2398388b8e7b9b Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmCPrbAQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DroPhEADWbUA86iVDtnAdXgVLuaZfc+5gXYJQv8GV EzOECSES5iG4PAVXqZHjeTcY+mj5HxTRmNUz00ci1eVeZISFojr41084dm8/pv6y Er5LFxHXLDJRIT6o2ODadE1pxKE7ada/PgicLlUgX9gYg5VPGqoYAKrnrEBBOnmF /9PRSsHYTGdw7kPXBfOZDafjJfrnYe/NXg7QCjwl/Fi1T3+omtbCSsxIs11XGuzf 3EjXoEVgGRRRtY0ibvO13tqEb1Q8KUVe4SHFIe2Hbbc/X9ZI6RKADzIqaqTMsxHa sV2jTh4XpOZcy9tf2yi0M/r6pwO+GJ/DqqKqWeWLapHInQS/TIPmj0P0fpQ8aNtz nr0Z++DQXJxnVuiT226mpcAI8z6YMw5JNO/mGUhnNney7lz9vQA8i6MGBwZlC4po trP/QxtBgwT/zPy6lb7Bhgtl3M5OnuHzFB9nsSKfsfilsWqAmX4lbdJ6DfJ0hLtZ 1WNH7uWt623HpL3KmFgA9ACdpZGh5zBqF8E7pHl7F+XVWT4BSIrUdEb4vEPVi4PV VeQUPmob5lbhghvEOi8F+uzflflOMdO5Z0W0iVqIST1AZywnrKqBRId5l6+sGXCf ZxuMLlv/KM+3nF7Xol9iEuksg4NnPr+QVxrn1chDcUkDb8nZDtVxDLd627ojQBZD miKJ6+4Z/g== =rFq4 -END PGP SIGNATURE-
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20210428 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20210428, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * wg_noise: compile on 32-bit * if_wg: do not increment error counter when sc is null * if_wg: handle if_transmit and if_output properly * if_wg: do not assume that IP header is pulled up * wg_noise: fix remote refcount leak * if_wg: unify xmit error path * if_wg: pull up packet before checking aip on input * netns: enable debug logging * if_wg: simplify state setting flow * if_wg: use proper bool for is_retry * if_wg: do not block for memory when sending buffer * if_wg: write data header directly * if_wg: enter net epoch for isr dispatch * if_wg: do not double-free after m_pullup * if_wg: allocate entire mbuf all at once After the rather intense set of commits in the last snapshot, this new snapshot has been entirely bug fixes and cleanups, addressing regressions in the last snapshot and simplifying other things. It should be much more stable than last week. This snapshot contains commits from: Jason A. Donenfeld and Matt Dunwoodie. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20210428.tar.xz SHA2-256: cceebd8f3f21d522342b3629fa0350e4fbc64a00035d330f82301741e56def46 Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmCKEQIQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrhlkD/9p/xcF9I5i60Qmv6zLSL+Sd88OHx9LXybv RiLmLjIlKOYq7b5msI7Gxf7UQnHCMiu+UuC8d8ZtsW1SpaPXRx/PFP/J/RfB Hd8uYD0whosgdzgaxvfZm+p2FPtDssQW6YNnrmkB62sNqZVEZdtWpLQK0W8BSdSa DBcgTJe30g/bhFYKQm5GGf3FCR5wIGlcPSJG+h0uhYnRqIJxcDCJEQ9doFQLTHn/ nygIYE4itONeJnT8acmH9zQZ/zuzz4RAHXkE7ux9Ld9pyvFue7OViCYZXNUInVgz uVbRfL90PUbQz4rcRbRoY2fs+4O5A2HOxegKhf4DqQ6ZGqYYdONnvJ1fZsHhp0zh ldP6uzDsd1fStyoANEyG/IKRDjpNtRPKVPgtlYLoNdStJll8T6DkdKC2mKAqF4Rp iBvEXMk1CgsXucqUESogUASKGnuIFfVYp684uqXSIYdga3M0Nnj6ESBhj1GIqGaR eNkV5DCH3y6MVT0BMs+5UX/qHPLMwwjGYEhzZc4+cqZDjbi8fEMc6+B6SeGdb6I+ +Bl1QXJoD/uzV7OMvXrb+oVGeku59XhjxLe8hf+wilC88R4SKnf0FciiTD0/rAdD 4x+oxW1jRgDk45qN1ryI1d2smPqNaAXRFwbQaL5BpWz6tjEeF4dv9VVCrKVqc1dg eUDt3xFu0A== =a84y -END PGP SIGNATURE-
Re: Kernel Panic on FreeBSD 12.2 when downing an tunnel interface
Fixed here a few days ago: https://git.zx2c4.com/wireguard-freebsd/commit/?id=cb7cd32a7c47151c161603cdc8a1c21f48550a65 I'll have a new snapshot out likely tomorrow. Jason
Kernel Panic on FreeBSD 12.2 when downing an tunnel interface
Greetings, Here are my versions: wireguard-kmod 0.0.20210424_1 wireguard-tools 1.0.20210424 I'm running the latest snapshot code (see above) on FreeBSD 12.2. I am seeing kernel panics when tearing down tunnels using `wg-quick down` and even `ifconfig wg0 destroy` when the interface has IPv6 addresses assigned to it. Any ideas? All the best -- R. Christian McDonald M: (616) 856-9291 E: rcmcdonal...@gmail.com
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20210424 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20210424, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == This is our biggest snapshot ever, aside from the initial one, with a pretty crazy amount of code changed/rewritten/reworked: TODO.md | 14 +- src/compat.h | 15 +- src/crypto.c | 92 + src/crypto.h |9 + src/if_wg.c | 3076 +++-- src/selftest/allowedips.c | 625 +++ src/selftest/cookie.c | 291 src/selftest/counter.c| 97 ++ src/support.h | 62 +- src/version.h |2 +- src/wg_cookie.c | 430 +++-- src/wg_cookie.h | 92 +- src/wg_noise.c| 1447 ++-- src/wg_noise.h| 199 +-- 14 files changed, 3756 insertions(+), 2695 deletions(-) In addition to adding automated selftests for some of the more critical and finicky algorithms, we've squashed numerous bugs basically everywhere, added proper reference counting, epoch usage, and locking where it makes sense, and fixed up several critical state machines. Because of the size of this snapshot, I'm not going to annotate every commit like usual, but if you're curious, many of the commits in the git repo have quite a bit of commentary. Hopefully this is a massive step toward getting this kmod complete and stable. With all the churn, I would expect some whonkiness, at least in the short term, but I don't anticipate anything we won't be able to address quickly. This snapshot contains commits from: Matt Dunwoodie, Jason A. Donenfeld, and Frank Behrens. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20210424.tar.xz SHA2-256: bfa8d3c4854f802567db51a89fdea32e7bf98a3d54a525359bdb240f2e864735 Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmCEUjQQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrkhxD/9R/coAcfFsTW0pCLDCshVFXa3egquRQ8rm cdkzHqJ1gBELE7W77xWllKkZ1+zejMqzJg0KYOn4ybJv083CYij1htMD9KEr/aCb HKUbYVptuzGIBC0D+m39GFrkSstdoypRApUPixI5J69JJCCUQtraOJ5pMF8tKllw Yi5r/axfqZA5NhJKx0Z8HZeKCZWfUX7+ht9GuoYi6Xaw5ZCJF+YrG2g5tX8vebhO Mcnb6AgQx8AUeGDf6z2m3yEH8N3hqbNyoTd8lCSo2zztsbF52H8wV1b8tiVK2Ebd cwEpGM4XTCl+lWi+u9k5YhfTd5AyN4xIRReLbBdjt4EaC6abGe5G4/FpukrBgHOL lqhDfnT3gJxB5yc1p15tXgfnXlSYZiULnwTmOkQi5ZZH70cuypwNq8v9GfromOJJ PjNKrNH5Jpi9QeIQX2IVLyBeU8cfHldvulOvudfrwSN8FDw5qOzG4NUxS880LSSk HER8OvwV6PmMFskvhvVxKJfpJXqxm3bwnDjBhEwvQiNQYHLfNxSkg+EC/61ztqmg qO7gxTxzCRrX0Uz9F/27bVfqO02noeaZ907K54UvlPgAkYowKCBOrPbBxlZkiNA9 p8IU5d069yRbLnoPk/Iu3inw1HNmjUgwmc73wpzwO0M7rZKhY+lPJC1bkHYYGfvg lvLtFGbBKg== =MJeK -END PGP SIGNATURE-
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Stefan Haller writes: > On Mon, Apr 19, 2021 at 01:42:58PM -0600, Jason A. Donenfeld wrote: >> On Mon, Apr 19, 2021 at 1:42 PM Stefan Haller wrote: >> > >> > On Mon, Apr 19, 2021 at 08:25:46PM +0200, Toke Høiland-Jørgensen wrote: >> > > Stefan, any chance you could test this patch to Bird (*instead of* the >> > > previous one that removes the check from the Babel code)? >> > >> > The patch is working on FreeBSD 13.0. >> >> Just FYI, the previous patch was added to ports, so I wanted to double >> check that you removed that previous patch before adding this one... > > Yes, I did remove the old patch (in proto/babel/babel.c): > >> root@fbsd:/usr/ports/net/bird2 # git status . >> On branch main >> Changes not staged for commit: >> (use "git add/rm ..." to update what will be committed) >> (use "git restore ..." to discard changes in working directory) >> deleted:files/patch-proto_babel_babel.c >> >> Untracked files: >> (use "git add ..." to include in what will be committed) >> files/patch-sysdep_bsd_krt-sock.c Awesome! Thank you for testing! :) -Toke
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
On Mon, Apr 19, 2021 at 01:42:58PM -0600, Jason A. Donenfeld wrote: > On Mon, Apr 19, 2021 at 1:42 PM Stefan Haller wrote: > > > > On Mon, Apr 19, 2021 at 08:25:46PM +0200, Toke Høiland-Jørgensen wrote: > > > Stefan, any chance you could test this patch to Bird (*instead of* the > > > previous one that removes the check from the Babel code)? > > > > The patch is working on FreeBSD 13.0. > > Just FYI, the previous patch was added to ports, so I wanted to double > check that you removed that previous patch before adding this one... Yes, I did remove the old patch (in proto/babel/babel.c): > root@fbsd:/usr/ports/net/bird2 # git status . > On branch main > Changes not staged for commit: > (use "git add/rm ..." to update what will be committed) > (use "git restore ..." to discard changes in working directory) > deleted:files/patch-proto_babel_babel.c > > Untracked files: > (use "git add ..." to include in what will be committed) > files/patch-sysdep_bsd_krt-sock.c Kind regards, Stefan
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
On Mon, Apr 19, 2021 at 1:42 PM Stefan Haller wrote: > > On Mon, Apr 19, 2021 at 08:25:46PM +0200, Toke Høiland-Jørgensen wrote: > > Stefan, any chance you could test this patch to Bird (*instead of* the > > previous one that removes the check from the Babel code)? > > The patch is working on FreeBSD 13.0. Just FYI, the previous patch was added to ports, so I wanted to double check that you removed that previous patch before adding this one...
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
On Mon, Apr 19, 2021 at 08:25:46PM +0200, Toke Høiland-Jørgensen wrote: > Stefan, any chance you could test this patch to Bird (*instead of* the > previous one that removes the check from the Babel code)? The patch is working on FreeBSD 13.0. Kind regards, Stefan
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Toke Høiland-Jørgensen writes: > Stefan Haller writes: > >> Hi Jason, >> >> On Thu, Apr 15, 2021 at 06:05:03PM -0600, Jason A. Donenfeld wrote: >>> I spent the day playing around with bird and babel and sorted out >>> FreeBSD's v6 situation. Basically, ff00::/8 addresses are treated >>> differently, and they're blocked unless the interface sets >>> IFF_MULTICAST. So I've committed >>> https://git.zx2c4.com/wireguard-freebsd/commit/?id=a7a84a17faf784857f076e37aa4818f6b6c12a95 >>> to do this. >> >> That is also what I observed. Without IFF_MULTICAST I see the following >> error in bird's log: >> >> bird[8045]: babel1: Socket error: IPV6_MULTICAST_IF: Can't assign requested >> address >> bird[8045]: babel1: Cannot open socket for wg1 >> >>> Stefan - please let me know if those work for you. In my testing thus >>> far, things seem to work for me. >> >> After applying Toke's patch for bird and your Wireguard patch in >> a7a84a17faf784 everything is working as before (with minor config >> changes). >> >> Just for the record, my previous configuration looked like this (using >> POINTTOPOINT interfaces, I use ifconfig to set the peer address): >> >>> [Interface] >>> ... >>> Address = fe80::5/64 >>> PostUp = ifconfig %i inet 169.254.42.5/32 169.254.42.2 >> >> My new configuration without POINTTOPOINT, but only a single peer >> directly attached to other side of the wg tunnel: >> >>> [Interface] >>> ... >>> Address = 169.254.42.5/32, fe80::5/64 >>> PostUp = route add 169.254.42.2 -iface %i >> >> So for me everything works as expected again. A big thanks to all of you for >> figuring out what was going wrong and getting it fixed so quickly. > > Great! You're welcome :) Stefan, any chance you could test this patch to Bird (*instead of* the previous one that removes the check from the Babel code)? -Toke diff --git a/sysdep/bsd/krt-sock.c b/sysdep/bsd/krt-sock.c index c2faa23dd44f..cd89544063c7 100644 --- a/sysdep/bsd/krt-sock.c +++ b/sysdep/bsd/krt-sock.c @@ -665,6 +665,9 @@ krt_read_ifinfo(struct ks_msg *msg, int scan) else f.flags |= IF_MULTIACCESS; /* NBMA */ + if (fl & IFF_MULTICAST) +f.flags |= IF_MULTICAST; + iface = if_update(); if (!scan)
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi Frank, On Sat, Apr 17, 2021 at 9:23 AM Frank Behrens wrote: > > > Am 17.04.2021 um 17:00 schrieb Jason A. Donenfeld: > > Does this actually fix or change anything? Don't new sockets have > > fib==0 right out of the gate already? > > New sockets inherit the fib from the current process. If you create > the wg interface from a process with different fib, that fib will also > be used for this socket. Probably the difference in code is not very > important for the case of a system default boot. But that may vary > for jails/vnets with different default fibs. > > In my test case the sequence > > setfib 1 ifconfig wg0 create > > ifconfig wg0 tunnelfib 0 > failed. Ahh, interesting. I applied your patch. Thanks for the persistence in getting this feature working well. Jason
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Am 17.04.2021 um 17:00 schrieb Jason A. Donenfeld: Does this actually fix or change anything? Don't new sockets have fib==0 right out of the gate already? New sockets inherit the fib from the current process. If you create the wg interface from a process with different fib, that fib will also be used for this socket. Probably the difference in code is not very important for the case of a system default boot. But that may vary for jails/vnets with different default fibs. In my test case the sequence > setfib 1 ifconfig wg0 create > ifconfig wg0 tunnelfib 0 failed. Frank -- Frank Behrens Osterwieck, Germany
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi Frank, On 4/17/21, Frank Behrens wrote: > Hi Jason! > > Am 13.04.2021 um 04:57 schrieb Jason A. Donenfeld: >> Can you let me know if this fixes the issue? >> >> https://git.zx2c4.com/wireguard-freebsd/commit/?id=cdb18ebf44a5babb57cddccd6b33e9f19cfdf365 > > It looks better, but a little too much optimized. ;-) > > The fix is in https://git.zx2c4.com/wireguard-freebsd/log/?h=fb/fib3 Does this actually fix or change anything? Don't new sockets have fib==0 right out of the gate already? Jason
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi Jason! Am 13.04.2021 um 04:57 schrieb Jason A. Donenfeld: Can you let me know if this fixes the issue? https://git.zx2c4.com/wireguard-freebsd/commit/?id=cdb18ebf44a5babb57cddccd6b33e9f19cfdf365 It looks better, but a little too much optimized. ;-) The fix is in https://git.zx2c4.com/wireguard-freebsd/log/?h=fb/fib3 Best regards, Frank -- Frank Behrens Osterwieck, Germany
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
On Fri, Apr 16, 2021 at 9:17 AM Jason A. Donenfeld wrote: > > I set up a lab and will do some testing with version before and after > > the change, maybe for FRR it's enough to set NBMA or similar. > > Could you send a minimal bird config that's broken so that I can > reproduce the problem and debug? With the latest version -- https://lists.zx2c4.com/pipermail/wireguard/2021-April/006634.html -- wireguardd seems to be working out of the box with frr.
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Hi Michael, On 4/16/21, Muenz, Michael wrote: > Am 16.04.2021 um 10:57 schrieb Stefan Haller: >> After applying Toke's patch for bird and your Wireguard patch in >> a7a84a17faf784 everything is working as before (with minor config >> changes). >> >> Just for the record, my previous configuration looked like this (using >> POINTTOPOINT interfaces, I use ifconfig to set the peer address): > > > Hi, > > Just following the conversation and also had a quick chat with Jason via > IRC (mimugmail). > We had a couple of reports that with latest change of removing PTP which > breaks OSPF. > In our case (OPNsense) we rely on FRR so it would be nice to have a > generic solution without toucingh routing software itself. > > >> So for me everything works as expected again. A big thanks to all of you >> for >> figuring out what was going wrong and getting it fixed so quickly. >> >> > > I set up a lab and will do some testing with version before and after > the change, maybe for FRR it's enough to set NBMA or similar. Could you send a minimal bird config that's broken so that I can reproduce the problem and debug? Jason
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Am 16.04.2021 um 10:57 schrieb Stefan Haller: After applying Toke's patch for bird and your Wireguard patch in a7a84a17faf784 everything is working as before (with minor config changes). Just for the record, my previous configuration looked like this (using POINTTOPOINT interfaces, I use ifconfig to set the peer address): Hi, Just following the conversation and also had a quick chat with Jason via IRC (mimugmail). We had a couple of reports that with latest change of removing PTP which breaks OSPF. In our case (OPNsense) we rely on FRR so it would be nice to have a generic solution without toucingh routing software itself. So for me everything works as expected again. A big thanks to all of you for figuring out what was going wrong and getting it fixed so quickly. I set up a lab and will do some testing with version before and after the change, maybe for FRR it's enough to set NBMA or similar. Best, Michael
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Stefan Haller writes: > Hi Jason, > > On Thu, Apr 15, 2021 at 06:05:03PM -0600, Jason A. Donenfeld wrote: >> I spent the day playing around with bird and babel and sorted out >> FreeBSD's v6 situation. Basically, ff00::/8 addresses are treated >> differently, and they're blocked unless the interface sets >> IFF_MULTICAST. So I've committed >> https://git.zx2c4.com/wireguard-freebsd/commit/?id=a7a84a17faf784857f076e37aa4818f6b6c12a95 >> to do this. > > That is also what I observed. Without IFF_MULTICAST I see the following > error in bird's log: > > bird[8045]: babel1: Socket error: IPV6_MULTICAST_IF: Can't assign requested > address > bird[8045]: babel1: Cannot open socket for wg1 > >> Stefan - please let me know if those work for you. In my testing thus >> far, things seem to work for me. > > After applying Toke's patch for bird and your Wireguard patch in > a7a84a17faf784 everything is working as before (with minor config > changes). > > Just for the record, my previous configuration looked like this (using > POINTTOPOINT interfaces, I use ifconfig to set the peer address): > >> [Interface] >> ... >> Address = fe80::5/64 >> PostUp = ifconfig %i inet 169.254.42.5/32 169.254.42.2 > > My new configuration without POINTTOPOINT, but only a single peer > directly attached to other side of the wg tunnel: > >> [Interface] >> ... >> Address = 169.254.42.5/32, fe80::5/64 >> PostUp = route add 169.254.42.2 -iface %i > > So for me everything works as expected again. A big thanks to all of you for > figuring out what was going wrong and getting it fixed so quickly. Great! You're welcome :) -Toke
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Hi Jason, On Thu, Apr 15, 2021 at 06:05:03PM -0600, Jason A. Donenfeld wrote: > I spent the day playing around with bird and babel and sorted out > FreeBSD's v6 situation. Basically, ff00::/8 addresses are treated > differently, and they're blocked unless the interface sets > IFF_MULTICAST. So I've committed > https://git.zx2c4.com/wireguard-freebsd/commit/?id=a7a84a17faf784857f076e37aa4818f6b6c12a95 > to do this. That is also what I observed. Without IFF_MULTICAST I see the following error in bird's log: bird[8045]: babel1: Socket error: IPV6_MULTICAST_IF: Can't assign requested address bird[8045]: babel1: Cannot open socket for wg1 > Stefan - please let me know if those work for you. In my testing thus > far, things seem to work for me. After applying Toke's patch for bird and your Wireguard patch in a7a84a17faf784 everything is working as before (with minor config changes). Just for the record, my previous configuration looked like this (using POINTTOPOINT interfaces, I use ifconfig to set the peer address): > [Interface] > ... > Address = fe80::5/64 > PostUp = ifconfig %i inet 169.254.42.5/32 169.254.42.2 My new configuration without POINTTOPOINT, but only a single peer directly attached to other side of the wg tunnel: > [Interface] > ... > Address = 169.254.42.5/32, fe80::5/64 > PostUp = route add 169.254.42.2 -iface %i So for me everything works as expected again. A big thanks to all of you for figuring out what was going wrong and getting it fixed so quickly. Kind regards, Stefan
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20210415 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20210415, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * if_wg: remove peer marshalling from get request This is a pretty massive code cleanup that decreases memory usage on `wg show` and also simplifies the code considerably, replacing 312 lines with 94. * if_wg: allow debugging with `ifconfig wg0 debug` Users can now run `ifconfig wg0 debug` to see the usual debugging messages in dmesg, just like on Linux with dynamic_debug. * if_wg: don't check return value of WAITOK Tiny cleanup. * if_wg: do not allow ioctl to race with clone_destroy This works around some bugs in the core FreeBSD kernel networking stack, where clone_destroy races with ioctls and sometimes even packet transmission. There are upstream patches pending to fix this, but for now it looks like every driver works around it in its own way, so for now we go with an approach most similar to the if_tuntap.c driver. * if_wg: set multicast flag Following extensive discussion [1] with Stefan Haller and Toke Høiland- Jørgensen, the IFF_MULTICAST option is now set on the interface, so that bird can send packets using babel. It turns out that FreeBSD forbids v6 multicast address destinations, even when used in a unicast context, if this flag isn't set, which differs from Linux semantics. This patch combined with [2] from Toke to upstream bird will allow WireGuard to work with bird as it did when we previously used IFF_POINTTOPOINT (which had its own problems). I sent a patch to the FreeBSD port of bird here [3] so that hopefully if_wg is functional with bird and babel not before too long. [1] https://lore.kernel.org/wireguard/CAHmME9qerb3LhuJfQ2L=J9gz=vgxv47quawc3-lymtwvwnn...@mail.gmail.com/T/ [2] https://bird.network.cz/pipermail/bird-users/2021-April/015415.html [3] https://lists.freebsd.org/pipermail/freebsd-ports/2021-April/120867.html This snapshot contains commits from: Jason A. Donenfeld. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20210415.tar.xz SHA2-256: 40dae82e27b37e236f761a2e84f892fe10ee183227287e7affdd5be571a1e612 Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmB5HrUQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrhkhEACgF5svIroYhrH23L9/XU9ndBmOqM9kc+ke nx7w0x+jWdfOOvCRMbuS9LFLLTBfP+/y4igPpDVYi6Njl20YamcfmTa6mqNizpIg TA5OCYKqOZSPEQynPz4pNtqFhT+ZqxquDgnNQB6RL9PfmVzDT4Jvk4/8IFF5f3ls hjLa14cpL2MkrGvYbM5WPUo/3zXkHA6Ai1uzAAa8HmxZI8Dl3/L4EEoUXO6VeUK7 KI+EyelH7N8ZLZNKBTT7j0CcTtA26zbMY8VtNlAJBiYaSpktSaox3JcTtS9nRFU8 HOoherDDP4weHMcrr9En2VVkMGHK5F9EvgbpDGPGObLcXY8u/AU9xZTBxn7es7Go AYtkOcr6q1QvIEtnzQT8hYr8umhq31QnGUuA50LhuSCl5WOPJWAU0+Y9CnszNeyO KPgou/8zAE7VdUk4js3MXKXm5PbEEFMotqkluXHXYg8SRqyD+lYAi5G+wfc3iFgo U//8HkAPOr81O32Y+clLsTkmM270QRiQ90UTOBAZPDBjHR+ScUzUa6uZ8GKWasjO U8xMPw2t1DR1gYvPFQZ6O7VxVp4dL34GNmgrPFr9+pfsNyZ5nEMCYr2IBrHNwN2H zSWDYyw8ySs053IWZOMIlXGDvKp4xqI0AK8ioHQk3kY87UtNrM1BN1kvQAz7CIhS vvsCQq6Tdg== =bgfK -END PGP SIGNATURE-
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Hey Stefan, Toke, I spent the day playing around with bird and babel and sorted out FreeBSD's v6 situation. Basically, ff00::/8 addresses are treated differently, and they're blocked unless the interface sets IFF_MULTICAST. So I've committed https://git.zx2c4.com/wireguard-freebsd/commit/?id=a7a84a17faf784857f076e37aa4818f6b6c12a95 to do this. We _could_ also set IFF_BROADCAST, which would translate to babel enabling IF_MULTICAST, but so far I can't see how this would help anything real, and combined with Toke's patch -- https://bird.network.cz/pipermail/bird-users/2021-April/015415.html -- I think we're actually in a good situation. Seeing that this now works, I've also dropped the link1 hack and put that in a branch in case it becomes useful later. Bernhard (decke@) is CC'd here in case he'd like to get some of this into ports early for your use case. Specifically, this involves: 1) https://bird.network.cz/pipermail/bird-users/2021-April/015415.html for the bird2 package. 2) https://git.zx2c4.com/wireguard-freebsd/patch/?id=a7a84a17faf784857f076e37aa4818f6b6c12a95 for the wireguard-kmod package. Stefan - please let me know if those work for you. In my testing thus far, things seem to work for me. Long term, we'll certainly want to have Toke's planned support for direct WireGuard peering inside of bird. Regards, Jason
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
"Jason A. Donenfeld" writes:
> Hi Stefan,
>
> Sounds like Toke has come up with the optimal solution, so I think
> I'll drop the "link1" patch/hack.
>
> One thing I was wondering though, mostly for my own curiosity, is what
> config you're using that shows the problem, and how does the problem
> manifest?
>
> I just put this in bird.conf:
>
> router id 192.168.88.2;
> protocol babel {
>interface "wg0" {
>type wired;
>port 1234;
>};
> };
>
> And then I ran:
>
> # pkg install bird2
> # bird -d -c bird.conf
>
> And I didn't see any troubling error messages. But maybe it's more
> subtle than that?
I think it would just silently ignore the interface; does it say
anything about running on it? You could also see if there's any traffic,
there should be UDP packets with dest port 6696 appearing if it does
run...
-Toke
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Hi Stefan,
Sounds like Toke has come up with the optimal solution, so I think
I'll drop the "link1" patch/hack.
One thing I was wondering though, mostly for my own curiosity, is what
config you're using that shows the problem, and how does the problem
manifest?
I just put this in bird.conf:
router id 192.168.88.2;
protocol babel {
interface "wg0" {
type wired;
port 1234;
};
};
And then I ran:
# pkg install bird2
# bird -d -c bird.conf
And I didn't see any troubling error messages. But maybe it's more
subtle than that?
Jason
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Stefan Haller writes: > Hi Toke, > > On Thu, Apr 15, 2021 at 12:14:04AM +0200, Toke Høiland-Jørgensen wrote: >> That's because the babel protocol code is checking for Bird's internal >> MULTICAST flag, which is set like: >> >> else if (fl & IFF_POINTOPOINT)/* PtP */ >> f.flags |= IF_MULTICAST; >> else if (fl & IFF_BROADCAST) /* Broadcast */ >> f.flags |= IF_MULTIACCESS | IF_BROADCAST | IF_MULTICAST; >> >> so it needs either the OS-level POINTOPOINT or the BROADCAST flag set. >> Wireguard interfaces on Linux has POINTOPOINT which is enough for Bird. > > That explains a lot. I expected something like this, but did not have > time yet to look more closely. > >> And yeah, for now Babel only speaks multicast; the spec does allow for >> unicast communication, but the code in Bird doesn't implement that yet >> (I'm the author of the Babel implementation in Bird). Even for unicast, >> Babel still needs multicast for discovery, but in the case of Wireguard >> that could be replaced by reading the peers directly from the Wireguard >> kernel module. Add in updating of Wireguard AllowedIPs, and presto, >> there's you completely dynamic mesh requiring only a single wg interface >> on each peer :) > > Overall, this sounds like a great idea. Having to create so many > wireguard p2p tunnels to form a mesh is quite cumbersome. Using > Wireguards AllowedIPs as an alternative to the kernel routing table > sounds useful. The peer discovery could also be useful outside of the > babel protocol implementation (even though it will always be quite > non-standard). One could probably assume that the first configured > v6/128 and v4/32 IPs belong to the directly connected peer. > >> Quite happy to review Bird patches if someone wants to hack on this, >> BTW, but otherwise it's on my "eventually" list :P > > While I am interested and it sounds like a great opportunity to learn > cool new things I don't know a lot about yet, I have to see if I am > actually up to the task. :) > > Anyway, I think there is an agreement that it is better to add specific > support for Wireguard interfaces in bird instead of changing the > interface flags. Yeah; in the meantime, you can just patch Bird; just get rid of this check in proto/babel/babel.c (there are two of them): if (!(iface->flags & IF_MULTICAST)) continue; should have no ill effects. Actually I think we could just get rid of that check entirely, it's not strictly needed for anything other than maybe filtering a very wide glob of interfaces. I'll send a patch... -Toke
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Hi Toke, On Thu, Apr 15, 2021 at 12:14:04AM +0200, Toke Høiland-Jørgensen wrote: > That's because the babel protocol code is checking for Bird's internal > MULTICAST flag, which is set like: > > else if (fl & IFF_POINTOPOINT)/* PtP */ > f.flags |= IF_MULTICAST; > else if (fl & IFF_BROADCAST) /* Broadcast */ > f.flags |= IF_MULTIACCESS | IF_BROADCAST | IF_MULTICAST; > > so it needs either the OS-level POINTOPOINT or the BROADCAST flag set. > Wireguard interfaces on Linux has POINTOPOINT which is enough for Bird. That explains a lot. I expected something like this, but did not have time yet to look more closely. > And yeah, for now Babel only speaks multicast; the spec does allow for > unicast communication, but the code in Bird doesn't implement that yet > (I'm the author of the Babel implementation in Bird). Even for unicast, > Babel still needs multicast for discovery, but in the case of Wireguard > that could be replaced by reading the peers directly from the Wireguard > kernel module. Add in updating of Wireguard AllowedIPs, and presto, > there's you completely dynamic mesh requiring only a single wg interface > on each peer :) Overall, this sounds like a great idea. Having to create so many wireguard p2p tunnels to form a mesh is quite cumbersome. Using Wireguards AllowedIPs as an alternative to the kernel routing table sounds useful. The peer discovery could also be useful outside of the babel protocol implementation (even though it will always be quite non-standard). One could probably assume that the first configured v6/128 and v4/32 IPs belong to the directly connected peer. > Quite happy to review Bird patches if someone wants to hack on this, > BTW, but otherwise it's on my "eventually" list :P While I am interested and it sounds like a great opportunity to learn cool new things I don't know a lot about yet, I have to see if I am actually up to the task. :) Anyway, I think there is an agreement that it is better to add specific support for Wireguard interfaces in bird instead of changing the interface flags. Kind regards, Stefan
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
"Jason A. Donenfeld" writes:
> Hey Toke,
>
> Regarding POINTTOPOINT flag in Linux vs FreeBSD -- apparently FreeBSD
> routes everything differently simply by virtue of the interface having
> that flag, whereas on Linux, PTP routing mode is only switched on if
> you actually add an address with a dest peer. So for FreeBSD, the
> different routing seemed somewhat disruptive, so I stopped setting
> that flag in a recent snapshot release.
Ah, of course that would be different...
> Hey Stefan,
>
> Looking at bird's source code (iface.c), it looks like bird will only
> look at IFF flags if you fail to specify the interface type in the
> config file. Can you try *not changing to link1/ptp mode* and then
> setting the type flag in your bird config? Specifically:
>
> interface [instance ] {
> ...
> type [broadcast|bcast|pointopoint|ptp|
> nonbroadcast|nbma|pointomultipoint|ptmp];
This is from the OSPF protocol code, though, so it won't work for
Babel.
-Toke
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Hey Toke,
Regarding POINTTOPOINT flag in Linux vs FreeBSD -- apparently FreeBSD
routes everything differently simply by virtue of the interface having
that flag, whereas on Linux, PTP routing mode is only switched on if
you actually add an address with a dest peer. So for FreeBSD, the
different routing seemed somewhat disruptive, so I stopped setting
that flag in a recent snapshot release.
Hey Stefan,
Looking at bird's source code (iface.c), it looks like bird will only
look at IFF flags if you fail to specify the interface type in the
config file. Can you try *not changing to link1/ptp mode* and then
setting the type flag in your bird config? Specifically:
interface [instance ] {
...
type [broadcast|bcast|pointopoint|ptp|
nonbroadcast|nbma|pointomultipoint|ptmp];
If you set `type ptmp` or `type nbma` or something, and see if that
will do the trick for you? I'd like to _not_ release this link1
hack/trick if possible.
Thanks,
Jason
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Hi Jason, Thanks for your clarification. I understand that setting this flag would be a false promise to userspace, because generally Wireguard is point-to-multipoint and doesn't copy messages to multiple peers (which is not exactly necessary in my case, where only a single peer is configured on both sides). I just wanted to ensure that the introduced change was intentional before looking into other directions, hence my question. On Wed, Apr 14, 2021 at 02:24:20PM -0600, Jason A. Donenfeld wrote: > Does bird completely ignore interfaces without it? Is there no setting > to change that? At least a brief look at the code suggests this: [1] The Babel protocol seems to rely on well-known *link-local* IPv6 multicast addresses. I did not find anything related to unicast "hello" messages in the RFC or in the implementations. (OSPF is similar, but as far as I remember unicast hellos are explicitly allowed.) One odd thing I noticed: On Linux (5.11.13-arch1-1, so quite recent), the interface does not list the MULTICAST flag and the interface is still used by bird: # ip l show dev wg1 4: wg1: mtu 1400 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 I will have a closer look why it doesn't work on FreeBSD but the same thing works on Linux. I am probably missing something important. Kind regards, Stefan [1]: https://gitlab.nic.cz/labs/bird/-/blob/9c41e1ca3e93d4498eaa085139caf1545e08c1d8/proto/babel/babel.c#L1662
FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Hello everyone! Today I tried switching to the if_wg kernel module. I observed that the behaviour of the tunnel interface was changed to drop the POINTTOPOINT and MULTICAST flags (8801509656e9). For some reason the bird2 routing daemon is not picking up my interface if there is only a /32 address configured and I manually add host routes over the wg interface. This broke my wireguard mesh setup and I wanted to find out ways to get it back into a working state. Luckily, a look into the git history showed up change 0adab0e961c6e that I find really useful (and also quite smart). I can simply say `ifconfig wg0 link1` to get the POINTTOPOINT behaviour back. Unfortunately, most routing protocols seem to rely on multicast traffic (e.g. OSPF, Babel, at least with default settings). bird2 will not pick up my interface, because the MULTICAST flag is missing. I tested a simple change that you can also find at the end of this email. The link1 flag will not only toggle the POINTTOPOINT flag, but additionally also toggles the MULTICAST flag. I am not really experienced with kernel and network stack code, but to me it makes sense to mark the interface as multicast capable in a peer-to-peer setting (if you use this, you will most likely set AllowedIPs to 0.0.0.0/0, ::/0 anyway). Is such a change sensible? I tested the change for my specific use case and everything seems to be working again (without broader changes to the configuration otherwise necessary). I do not want to imply that the current behaviour is wrong, because I simply don't know much about the topic. If someone else is using dynamic routing protocols over p2p wireguard tunnels successfully, I appreciate pointers into the right direction :) Kind regards, Stefan diff --git a/src/if_wg.c b/src/if_wg.c index ca54476..414a641 100644 --- a/src/if_wg.c +++ b/src/if_wg.c @@ -2910,9 +2910,9 @@ wg_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data) case SIOCSIFFLAGS: if ((ifp->if_flags & IFF_LINK0) || !(ifp->if_flags & IFF_LINK1)) - ifp->if_flags &= ~IFF_POINTOPOINT; + ifp->if_flags &= ~IFF_POINTOPOINT & ~IFF_MULTICAST; else if (ifp->if_flags & IFF_LINK1) - ifp->if_flags |= IFF_POINTOPOINT; + ifp->if_flags |= IFF_POINTOPOINT | IFF_MULTICAST; ifp->if_flags &= ~(IFF_LINK0 | IFF_LINK1 | IFF_LINK2); if (ifp->if_flags & IFF_UP)
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Stefan Haller writes: > Hi Jason, > > Thanks for your clarification. I understand that setting this flag would > be a false promise to userspace, because generally Wireguard is > point-to-multipoint and doesn't copy messages to multiple peers (which > is not exactly necessary in my case, where only a single peer is > configured on both sides). > > I just wanted to ensure that the introduced change was intentional > before looking into other directions, hence my question. > > On Wed, Apr 14, 2021 at 02:24:20PM -0600, Jason A. Donenfeld wrote: >> Does bird completely ignore interfaces without it? Is there no setting >> to change that? > > At least a brief look at the code suggests this: [1] > > The Babel protocol seems to rely on well-known *link-local* IPv6 > multicast addresses. I did not find anything related to unicast "hello" > messages in the RFC or in the implementations. (OSPF is similar, but > as far as I remember unicast hellos are explicitly allowed.) > > One odd thing I noticed: On Linux (5.11.13-arch1-1, so quite recent), > the interface does not list the MULTICAST flag and the interface is > still used by bird: > > # ip l show dev wg1 > 4: wg1: mtu 1400 qdisc noqueue state UNKNOWN > mode DEFAULT group default qlen 1000 > > I will have a closer look why it doesn't work on FreeBSD but the same thing > works on Linux. I am probably missing something important. That's because the babel protocol code is checking for Bird's internal MULTICAST flag, which is set like: else if (fl & IFF_POINTOPOINT)/* PtP */ f.flags |= IF_MULTICAST; else if (fl & IFF_BROADCAST) /* Broadcast */ f.flags |= IF_MULTIACCESS | IF_BROADCAST | IF_MULTICAST; so it needs either the OS-level POINTOPOINT or the BROADCAST flag set. Wireguard interfaces on Linux has POINTOPOINT which is enough for Bird. And yeah, for now Babel only speaks multicast; the spec does allow for unicast communication, but the code in Bird doesn't implement that yet (I'm the author of the Babel implementation in Bird). Even for unicast, Babel still needs multicast for discovery, but in the case of Wireguard that could be replaced by reading the peers directly from the Wireguard kernel module. Add in updating of Wireguard AllowedIPs, and presto, there's you completely dynamic mesh requiring only a single wg interface on each peer :) Quite happy to review Bird patches if someone wants to hack on this, BTW, but otherwise it's on my "eventually" list :P -Toke
Re: FreeBSD if_wg POINTTOPOINT and MULTICAST behaviour
Hi Stefan, WireGuard does not do multicast, so we probably won't set that flag. You'll want to use babble over unicast anyway. As far as the `ifconfig wg0 link1` trick I added yesterday goes... I'm not totally convinced I'll keep that yet for the next snapshot. Does bird completely ignore interfaces without it? Is there no setting to change that? Ptp isn't quite a correct match for WireGuard, so having that flag to satisfy a misbehaving userspace seems like a bummer. Have you looked everywhere within bird first to see if there's another way? Jason
Re: freebsd - cpu consumption?
Hi Osku, We'll switch to using FreeBSD's opencrypto at some point, at which point the performance will improve. Right now we're using boring unoptimized reference implementations. See https://lists.freebsd.org/pipermail/freebsd-hackers/2021-March/057076.html for more info. So far nobody has stepped up to contribute code there. Jason
freebsd - cpu consumption?
Dear all, has anyone else noticed the CPU consumption on FreeBSD kernel Wireguard? For my test link, there is a net speed of 200 Mbit/s. On esxi based host, transferring inside Wireguard tunnel, full speed takes about 90% of xeon d-1541 host. Doing this on CentOS 7, takes less than 20% of the same host.
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20210412 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20210412, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * if_wg: set persistent keepalive even if interface is down * if_wg: remove old link state check * if_wg: allow getting tunnelfib to race * if_wg: set user cookie and fib through proper sockopts Various state fixes. * if_wg: simplify jail exit logic Fixing the jail exit logic here means that you can set jails up and down without rendering all wireguard tunnels useless. * TODO: initial dump There's now a pretty big TODO list of goals before switching into more intense bug fixing mode: https://git.zx2c4.com/wireguard-freebsd/about/TODO.md If you'd like to help out, please get in touch! This snapshot contains commits from: Jason A. Donenfeld. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20210412.tar.xz SHA2-256: eff7fdff89949ea294177fe9b18fde99a016fc4ad44aaa0f31ba6c2cff5ac506 Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmB1C0QQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrvE4EACNi7dCF+GUOh1FsEqLmovrcWTLk4+w9lm/ AFOB8X2mC3IYjDC1TNin7AnlG07uRE6W8ocLGywDdUYjh6ahozRqjbDIO/ZTuUD+ lc7IlisbDJ4QhJPbbuYpVnztKllptRvfuhDT28hX8T+3xhH0zdK7YfrotraEMMrt wiQSk4GNZvr+Z79h3Ev8vHv25EfzL8HNzg3k15kEPNsERvdcPdovSFYqAxS1+oss 6Z/L2OWlpCqVdOAwoQIxO1yWStxwEO8+x1SPHSDz8IS9lIrldPBgm277mEafo5u2 0jPd9D/9IS9tVgXFF9EcR/I5oPe1lUhpl193xEHgLnxlmD+H5ZhTi2Jud1s8MPmK XmqSTE2GA0clEwE0y4evBgDxP6B8o1LBcRsF62RR/KPASYAqoteQBAmv78MtL98S iluf4nGWwE+IrgSE1gfLbkHADo36GyX3a4jmEz5//IbjpnIzpuudqeH/p4d7QM4Y 4rS0egtkMF69cHtVdi98Xbi7vCDlXYEv1ikdi/mjAO5dr0sUoBisGSl0Uewje8Wl Lo3367BJOl7jwTUhRGtlxt8duR8FyYRvYAbMh+NJHCDjhwIWsqYZ+VW8/26YF2VX 6zg1Ge41kVgFvNaqrRXFKlWTJO0k8VIywbOWK6CnfB9LjborWhRSGF+hLXuGrxGx MxZAIgD/XA== =1CAc -END PGP SIGNATURE-
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi Frank, Can you let me know if this fixes the issue? https://git.zx2c4.com/wireguard-freebsd/commit/?id=cdb18ebf44a5babb57cddccd6b33e9f19cfdf365 Jason
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hello Jason! Am 31.03.2021 um 21:11 schrieb Jason A. Donenfeld: Thanks for the patch. Does the line `so4->so_fibnum = so6->so_fibnum = sc->sc_socket.so_fibnum;` also need to be changed too in initiation, or is that one fine? Thanks for the pointer. That part has to be changed as well. I updated my branch. Kind regards, Frank -- Frank Behrens Osterwieck, Germany
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hello Jason! Am 31.03.2021 um 21:11 schrieb Jason A. Donenfeld: Hi Frank, Thanks for the patch. Does the line `so4->so_fibnum = so6->so_fibnum = sc->sc_socket.so_fibnum;` also need to be changed too in initiation, or is that one fine? Good catch! I'll check this in the next few days (and update the branch if necessary). Frank -- Frank Behrens Osterwieck, Germany
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi Frank, Thanks for the patch. Does the line `so4->so_fibnum = so6->so_fibnum = sc->sc_socket.so_fibnum;` also need to be changed too in initiation, or is that one fine? Jason
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hello! Am 23.03.2021 um 06:51 schrieb Frank Behrens: Am 22.03.2021 um 19:14 schrieb Jason A. Donenfeld: Applied to git with some small modifications: https://git.zx2c4.com/wireguard-freebsd/commit/?id=0a5c6abdfaa1f4f09269a222c1720e2ff3b8aa02 Thanky you! That looks very good. I'm sorry, that I didn't test this in detail. Unfortunately it does not work, I made a patch in branch fb/fib2. Kind regards, Frank -- Frank Behrens Osterwieck, Germany
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20210323 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20210323, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * if_wg: do not check for null M_NOWAIT return * compat: backport callout_func_t to 12.1 * wg_noise: ensure non-zero'd handshakes have a valid local index * if_wg: fix decryption failures on jumbo ingress Numerous correctness fixes. * if_wg: implement selection of FIB (routing table) for tunneled packets It's now possible to run `ifconfig wg0 tunnelfib 7` or the like. * if_wg: drop pointtopoint flag * if_wg: disable llv6 Set better flags for the interface to reflect how its routing actually works. Note that this will need a patch to wg-quick(8), since we're leaving point-to-point mode. A new release of wireguard-go will be made to with the same change. This snapshot contains commits from: Jason A. Donenfeld, Matt Dunwoodie, and Frank Behrens. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20210323.tar.xz SHA2-256: 9e37f330cea71974d9ac0a7f177d7c4d3252faaee574eb896ff553c55790f46d Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmBaO1YQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4DrgwkEAC6yIjz759hctDkwnrpEaKXrYUNCSV7hjcN 1jIP6AtnV4cgDVQ0v2yvMtkWhqovdGlFK/2Db+3zeEc9cKRarvCPBfQ1TDTKb899 NkyNVuxgXduCha3wVJyWLQGO3ry5i0765H6Wxyo52Jjb8aBWqh8VRauF+hfljbfg BfiuLMMXXmPyhnuzlAc2p3G9SMV6NDMeGGs5ipjKk3fqBzIe4qvjVWA3Y41P4rjJ 89rXKWUrcRMgnb3/UOS/43+Q7b/9EeLfMZgHN4OUqpRcr9rJA5IED+JotcydpL5p ar6mcurltQFL8NzE0b8AXcO8x0fB6CYUCWJDISK2neXCsonltcnGmTuZAYA7HzL1 NqA5b5/7pTqNIym/gzdummYDqwTSjtTm4/EOdac1k0z9nAoo7kDtatZTD0d6a7sP jbM27DdwtEgIDp5aIaxkplI0x+5QRqfg/A+PjjSvRG6V8o/NpQWO2kJfu5Pcm4Ll mJ5c2qLFj0XCVSIWwnM7oQscolVk5VYKbtXIG5ZtBDPGCwuSuG4OPM4U1s/5hAwX 1DweYZvavjOxEOimvNS2FkVRUXII50r07SbrgAqoxIgTA7iNQ9IJ6aiqC9mbf+YD PZQbpFx8uGZUZDKHhkF9C6PROaYDgAqG6rPE01kWnWotTd8gjoVwGAGy3n0nn+Ph NV5wf4cIJw== =Hpzo -END PGP SIGNATURE-
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi Jason! Am 22.03.2021 um 19:14 schrieb Jason A. Donenfeld: Applied to git with some small modifications: https://git.zx2c4.com/wireguard-freebsd/commit/?id=0a5c6abdfaa1f4f09269a222c1720e2ff3b8aa02 Thanky you! That looks very good. -- Frank Behrens Osterwieck, Germany
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi Frank,
On Sat, Mar 20, 2021 at 11:15 AM Frank Behrens wrote:
>
> Hi Jason,
>
> thanks for your response.
>
> Am 19.03.2021 schrieb Jason A. Donenfeld:
> > In other words, you have push access to all branches beginning with fb/ .
> That works, thanks. Meanwhile I pushed my branch to fb/fib.
>
> > Right now we have the `wg set wg0 fwmark ...` mapped to
> > SO_USER_COOKIE, as I'm sure you saw there. But maybe FIB would be a
> > better thing to use for that? We could adjust wireguard-go to do the
> > same with the tuntap ioctl.
> I believe we have different, orthogonal things:
>
> 1. The selection of routing table (fib) for received, decrypted packets.
> -> Already implemented in wg_deliver_in() #2098 and controlled
> by "ifconfig wg0 fib 1"
>
> 2. The selection of routing table for outgoing, encrypted packets.
> -> That is addressed by my patch and controlled by
> "ifconfig wg0 tunnelfib 1". Maybe wg(8) should receive also
> an option for that purpose, if other OS use equivalent functions.
>
> 3. The setting of special marks, useable in packet filter/firewall
> processing. I guess, that is the meaning for "wg.. fwmark". I'm not
> sure, how best to implement that for FreeBSD. For ipfw(4) there is some
> functionality using socket cookies, as already implemented. For pf(4)
> packet filter the documentation mentions mbuf_tags(9). Apparently
> we need some input from a FreeBSD packet filter developer.
It looks like user_cookie is independently useful for use in ipfw, so
I'll keep fwmark mapped to that. But your tunnelfib patch is _also_
useful, so I'll apply this patch.
Some bugs/questions, though:
+static void
+wg_socket_setfib(struct wg_softc *sc)
+{
+ struct wg_socket *so = >sc_socket;
+ struct socket *so4, *so6;
+ struct sockopt sopt;
+ int rc;
+
+ so4 = atomic_load_ptr(>so_so4);
+ so6 = atomic_load_ptr(>so_so6);
These are epoch-protected members. So the line above that should have
NET_EPOCH_ASSERT().
+ if (so4) {
+ rc = sosetopt(so4, );
+ MPASS(rc == 0);
+ }
+ if (so6) {
+ rc = sosetopt(so6, );
+ MPASS(rc == 0);
+ }
Rather than MPASS, this error should be passed back to the caller and
reported back to userspace, in the same way that we account for errors
with socket binding.
+ case SIOCSTUNFIB:
+ if ((ret = priv_check(curthread, PRIV_NET_SETIFFIB)) != 0)
+ break;
This priv check should be relative to our stored creds that control
the socket right? There are some jail considerations to think about
here.
+ if (ifr->ifr_fib >= rt_numfibs) {
+ ret = EINVAL;
+ } else {
+ sc->sc_fibnum = ifr->ifr_fib;
+ wg_socket_setfib(sc);
+ }
There are two ways to implement this. Either we check for 0 <= ifr_fib
< rt_numfibs, and then directly assign it to so->so_fibnum, like we do
for user_cookie, and avoid sosockopt. This sounds simplest and I
wouldn't mind doing that. Or we use sosockopt, but the bounds testing
remains in sosockopt and not duplicated here, and we rely on
wg_socket_setfib bubbling the error back up. Let's pick one of these
approaches, rather than combining them.
struct wg_softc {
LIST_ENTRY(wg_softc) sc_entry;
struct ifnet *sc_ifp;
int sc_flags;
+ u_int sc_fibnum;
Shouldn't this be added to `struct wg_socket`, alongside the
so_user_cookie member there, and managed in the same way?
Jason
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi Frank, > On 20. Mar 2021, at 6:05 PM, Frank Behrens wrote: > > 3. The setting of special marks, useable in packet filter/firewall > processing. I guess, that is the meaning for "wg.. fwmark". I'm not > sure, how best to implement that for FreeBSD. For ipfw(4) there is some > functionality using socket cookies, as already implemented. For pf(4) > packet filter the documentation mentions mbuf_tags(9). Apparently > we need some input from a FreeBSD packet filter developer. In pf(4) the tags are stored using mtag and that's reachable through the kernel only for direct tagging (normally it matches through ruleset and applies tags to packets in fly-by), although it is difficult to look up the tag name to tag integer from static functions inside pf_ioctl.c and keeping the index in sync with the tags that could change when the ruleset changes, see pf_tag_packet() in pf.c for low level tagging using the tag integer translated from the tag name during the last ruleset apply. Cheers, Franco
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi Jason, thanks for your response. Am 19.03.2021 schrieb Jason A. Donenfeld: In other words, you have push access to all branches beginning with fb/ . That works, thanks. Meanwhile I pushed my branch to fb/fib. Right now we have the `wg set wg0 fwmark ...` mapped to SO_USER_COOKIE, as I'm sure you saw there. But maybe FIB would be a better thing to use for that? We could adjust wireguard-go to do the same with the tuntap ioctl. I believe we have different, orthogonal things: 1. The selection of routing table (fib) for received, decrypted packets. -> Already implemented in wg_deliver_in() #2098 and controlled by "ifconfig wg0 fib 1" 2. The selection of routing table for outgoing, encrypted packets. -> That is addressed by my patch and controlled by "ifconfig wg0 tunnelfib 1". Maybe wg(8) should receive also an option for that purpose, if other OS use equivalent functions. 3. The setting of special marks, useable in packet filter/firewall processing. I guess, that is the meaning for "wg.. fwmark". I'm not sure, how best to implement that for FreeBSD. For ipfw(4) there is some functionality using socket cookies, as already implemented. For pf(4) packet filter the documentation mentions mbuf_tags(9). Apparently we need some input from a FreeBSD packet filter developer. Kind regards, Frank -- Frank Behrens Osterwieck, Germany
Re: Unable to serve TCP connections over wireguard interface on FreeBSD 13.0-RC2
On 2021-03-19 22:18, Jason A. Donenfeld wrote: Hi Ashish, Fixed: https://git.zx2c4.com/wireguard-freebsd/commit/?id=bb59a61785322a086dfc437c51e7cbcd918a5241 Thanks for the report. Jason Seems to work as expected now, thank you. -- Ashish “There was truth and there was untruth, and if you clung to the truth even against the whole world, you were not mad.” -- George Orwell, "Nineteen Eighty-Four"
Re: [ANNOUNCE] wireguard-freebsd snapshot v0.0.20210319 is available
Good news! This is now in ports as net/wireguard-kmod. That means it should eventually be built into a package too: # pkg install wireguard-kmod
[ANNOUNCE] wireguard-freebsd snapshot v0.0.20210319 is available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, An experimental snapshot, v0.0.20210319, of WireGuard for FreeBSD has been been tagged in the git repository. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. == Changes == * if_wg: use our own taskqgroup This removes the dependency on iflib entirely. * compat: backport to FreeBSD 12.2 * compat: backport to FreeBSD 12.1 * compat: backport properly to 13 * if_wg: cleanup timeout_t usage This is now running on 13.0 RC3, 12.2, 12.1. * if_wg: fix malloc overflows These should have been fixed during our initial pass but somehow weren't. Good thing we have more time to work on this. * if_wg: dispatch packets using netisr * if_wg: mark as point to point These fix up some network particulars that bring the inbound packet flow a bit more inline with what other drivers, like GRE or tuntap do. In particular, it should now be possible to listen on incoming TCP sockets. The source repository is available at the usual location: git clone https://git.zx2c4.com/wireguard-freebsd This snapshot is available in compressed tarball form: https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20210319.tar.xz SHA2-256: 96836d90a8daad863ccd9503e3bad4ca6c4c25df4892fc1e55b7a8f7b03afbcd Thank you, Jason Donenfeld -BEGIN PGP SIGNATURE- iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmBU84AQHGphc29uQHp4 MmM0LmNvbQAKCRBJ/HASpd4Drvt3EAC9ZMmkCeePRV6m4cTxnGYzCsaaqhXBgMKP qqsB/YMe6d7gWgDSARBW+49Nql1EG+V1mYP8+Cn3hBbCNE76vR+EuDs8d7xA/lqE 2KA44gX8GTbfZJHRCtPsh/la96OARXPPT4OPyc86mY1GlJc3Xg5KxV24PH4QFqEA AFnEvE91/lc+aZxkhuZ2mLYhz9eHwdEIHUfGm4cK8D6UM1Jvt8q3I8TOYA+WTbTD 30sQz4ymMtfE89gH+ExCRpE5qrRxMm9AFP4N2A5CNf3tGLf3ptXW2prLo2u+UymP GSugEDHbyGjx7APtYCEv5uqfGBRpiv/lnmC5iv3Z1V8GMKqB4kPcxPJwQEvwOVhr Of7CefHdpyuMHr3jWH+dMc4ZWNvKhJ42wZbb/A+yojM4t3ZyEBirlYHJ0Ieb/8IY ZAvXEI16L4pIikcWbOR0u+faE4+YtO1F2H3eTxXEuyPCcMnPN7iatqXvwuPNjx/w /KaXtPXdi5gNIWLjohNx0nO/guC/hz7JiIs6A9N+0JjmyzElKwzDou4+Us99QNyh iwGi0OJmuSE1nEOZ4H9lwGx1CJlEB2aflVpqe8VBEFn1XpotQT42e84UpqHAAM7p L97KJkNFJcQpQIG29C2nu8cGKqvM2MguNwTjSUBYy3Q/eZqWwyxE3GFwbvcMMc1n Vs6DlYtxWA== =4RS0 -END PGP SIGNATURE-
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hey Frank, Thanks for the patch. That looks terrific. One question, though: Right now we have the `wg set wg0 fwmark ...` mapped to SO_USER_COOKIE, as I'm sure you saw there. But maybe FIB would be a better thing to use for that? We could adjust wireguard-go to do the same with the tuntap ioctl. Jason
Re: [PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hi again, As well, you can now do: git push g...@git.zx2c4.com:wireguard-freebsd master:fb/whatever-you-want In other words, you have push access to all branches beginning with fb/ . Poke me on IRC (I'm zx2c4) there if you want to chat about dev. Jason
Re: [PATCH] Fix build on FreeBSD 13 after removal from base
Thanks for the patch. I addressed this slightly differently here: https://git.zx2c4.com/wireguard-freebsd/commit/?id=519e70d5fe88f1ae3c6d2318b22ecf927953fd02 I also trimmed those structs down a bit, as the latter members differ between 13 and 14.
[PATCH] freebsd: Implement selection of FIB (routing table) for tunneled packets
Hello Jason, hello community, although I regret the recent removal of FreeBSD kernel driver I can live with the present development model. With the current sources in the external repository I could create a working VPN for the first time with my Android phone. It works well, thanks to all contributors. The setup was done only with ifconfig(8), I have the now removed parts still in place. :-) For my setup I needed a different route for the encapsulated packets and so I implemented the missing parts. With this message I give that part to the community (matching the existing BSD-2-Clause-FreeBSD license). For the sake of simplicity I created an own branch on github, the commit is https://github.com/frbehrens/wireguard-freebsd/commit/f0445be7b5b30a98da11bf2e209739a2155a59bb or use for direct patch download: https://github.com/frbehrens/wireguard-freebsd/commit/f0445be7b5b30a98da11bf2e209739a2155a59bb.patch Kind regards, Frank -- Frank Behrens Osterwieck, Germany
Re: Unable to serve TCP connections over wireguard interface on FreeBSD 13.0-RC2
Hi Ashish, Fixed: https://git.zx2c4.com/wireguard-freebsd/commit/?id=bb59a61785322a086dfc437c51e7cbcd918a5241 Thanks for the report. Jason
Re: Unable to serve TCP connections over wireguard interface on FreeBSD 13.0-RC2
Hi Ashish, On Fri, Mar 19, 2021 at 8:06 AM Ashish wrote: > I'm running if_wg kernel module (git revision: 5ef4d3efa691e71) on > FreeBSD 13.0-RC2. > > With 172.18.10.1 being my local host's wireguard interface's IP address, > I can receive SYN packets, but it does not seem to send any > corresponding SYN/ACK. Huh, that's curious. I'll look into it and play around. One peculiarity I noticed in this thing is that it calls the ip_input function directly, which other drivers don't seem to do. Regards, Jason
Re: Unable to serve TCP connections over wireguard interface on FreeBSD 13.0-RC2
Random thought, Have you tried adjusting MTU? Depending on the client/application, I have better luck with something around 1300. --- Matthew Poletiek 303.810.9082 matthew.polet...@gmail.com www.matthewpoletiek.com On Fri, Mar 19, 2021 at 9:09 AM Ashish wrote: > > > [apologies, in case you receive duplicate messages] > > Hi, > > I'm running if_wg kernel module (git revision: 5ef4d3efa691e71) on > FreeBSD 13.0-RC2. > > With 172.18.10.1 being my local host's wireguard interface's IP address, > I can receive SYN packets, but it does not seem to send any > corresponding SYN/ACK. > > = > 01:26:26.327484 IP 172.18[.10.3.34160 > 172.18.10.1.22: Flags [S], seq > 1278197331, win 64860, options [mss 1380,sackOK,TS val 223949166 ecr > 0,nop,wscale 7], length 0 > 01:26:42.708175 IP 172.18.10.3.34160 > 172.18.10.1.22: Flags [S], seq > 1278197331, win 64860, options [mss 1380,sackOK,TS val 223965550 ecr > 0,nop,wscale 7], length 0 > 01:27:14.964162 IP 172.18.10.3.34160 > 172.18.10.1.22: Flags [S], seq > 1278197331, win 64860, options [mss 1380,sackOK,TS val 223997806 ecr > 0,nop,wscale 7], length 0 > > > 01:28:34.035384 IP 172.18.10.2.41905 > 172.18.10.1.22: Flags [S], seq > 2569759726, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val > 3991744006 ecr 0], length 0 > 01:28:34.035392 IP 172.18.10.2.41905 > 172.18.10.1.22: Flags [S], seq > 2569759726, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val > 3991745042 ecr 0], length 0 > 01:28:34.036002 IP 172.18.10.2.41905 > 172.18.10.1.22: Flags [S], seq > 2569759726, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val > 3991747129 ecr 0], length 0 > = > > ICMP works fine: > > = > 01:53:15.638529 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id > 47881, seq 0, length 64 > 01:53:15.638535 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, > seq 0, length 64 > 01:53:16.624443 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id > 47881, seq 1, length 64 > 01:53:16.624448 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, > seq 1, length 64 > 01:53:17.672109 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id > 47881, seq 2, length 64 > 01:53:17.672115 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, > seq 2, length 64 > 01:53:18.676223 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id > 47881, seq 3, length 64 > 01:53:18.676230 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, > seq 3, length 64 > 01:53:19.682131 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id > 47881, seq 4, length 64 > 01:53:19.682136 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, > seq 4, length 64 > = > > And I can make outbound TCP connections: > > = > 01:50:43.267331 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [S], seq > 2119392003, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val > 1918472905 ecr 0], length 0 > 01:50:43.415524 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [S.], seq > 2602046635, ack 2119392004, win 65535, options [mss 1380,nop,wscale > 11,sackOK,TS val 1347987709 ecr 1918472905], length 0 > 01:50:43.415532 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [.], ack 1, > win 33, options [nop,nop,TS val 1918473053 ecr 1347987709], length 0 > 01:50:43.415613 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq > 1:31, ack 1, win 33, options [nop,nop,TS val 1918473053 ecr 1347987709], > length 30 > 01:50:43.614035 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [P.], seq > 1:39, ack 31, win 33, options [nop,nop,TS val 1347987870 ecr > 1918473053], length 38 > 01:50:43.653218 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [.], ack > 39, win 33, options [nop,nop,TS val 1918473291 ecr 1347987870], length 0 > 01:50:43.693420 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq > 31:1055, ack 39, win 33, options [nop,nop,TS val 1918473331 ecr > 1347987870], length 1024 > 01:50:43.693435 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq > 1055:1543, ack 39, win 33, options [nop,nop,TS val 1918473331 ecr > 1347987870], length 488 > 01:50:43.818391 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [P.], seq > 39:1119, ack 31, win 33, options [nop,nop,TS val 1347988093 ecr > 1918473291], length 1080 > 01:50:43.819870 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq > 1543:1591, ack 1119, win 33, options [nop,nop,TS val 1918473457 ecr > 1347988093], length 48 > 01:50:43.880995 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [.], ack > 1543, win 33, options [nop,nop,TS val 1347988163 ecr 1918473331], length 0 >
Unable to serve TCP connections over wireguard interface on FreeBSD 13.0-RC2
[apologies, in case you receive duplicate messages] Hi, I'm running if_wg kernel module (git revision: 5ef4d3efa691e71) on FreeBSD 13.0-RC2. With 172.18.10.1 being my local host's wireguard interface's IP address, I can receive SYN packets, but it does not seem to send any corresponding SYN/ACK. = 01:26:26.327484 IP 172.18[.10.3.34160 > 172.18.10.1.22: Flags [S], seq 1278197331, win 64860, options [mss 1380,sackOK,TS val 223949166 ecr 0,nop,wscale 7], length 0 01:26:42.708175 IP 172.18.10.3.34160 > 172.18.10.1.22: Flags [S], seq 1278197331, win 64860, options [mss 1380,sackOK,TS val 223965550 ecr 0,nop,wscale 7], length 0 01:27:14.964162 IP 172.18.10.3.34160 > 172.18.10.1.22: Flags [S], seq 1278197331, win 64860, options [mss 1380,sackOK,TS val 223997806 ecr 0,nop,wscale 7], length 0 01:28:34.035384 IP 172.18.10.2.41905 > 172.18.10.1.22: Flags [S], seq 2569759726, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 3991744006 ecr 0], length 0 01:28:34.035392 IP 172.18.10.2.41905 > 172.18.10.1.22: Flags [S], seq 2569759726, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 3991745042 ecr 0], length 0 01:28:34.036002 IP 172.18.10.2.41905 > 172.18.10.1.22: Flags [S], seq 2569759726, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 3991747129 ecr 0], length 0 = ICMP works fine: = 01:53:15.638529 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 0, length 64 01:53:15.638535 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 0, length 64 01:53:16.624443 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 1, length 64 01:53:16.624448 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 1, length 64 01:53:17.672109 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 2, length 64 01:53:17.672115 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 2, length 64 01:53:18.676223 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 3, length 64 01:53:18.676230 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 3, length 64 01:53:19.682131 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 4, length 64 01:53:19.682136 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 4, length 64 = And I can make outbound TCP connections: = 01:50:43.267331 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [S], seq 2119392003, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 1918472905 ecr 0], length 0 01:50:43.415524 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [S.], seq 2602046635, ack 2119392004, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 1347987709 ecr 1918472905], length 0 01:50:43.415532 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [.], ack 1, win 33, options [nop,nop,TS val 1918473053 ecr 1347987709], length 0 01:50:43.415613 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq 1:31, ack 1, win 33, options [nop,nop,TS val 1918473053 ecr 1347987709], length 30 01:50:43.614035 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [P.], seq 1:39, ack 31, win 33, options [nop,nop,TS val 1347987870 ecr 1918473053], length 38 01:50:43.653218 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [.], ack 39, win 33, options [nop,nop,TS val 1918473291 ecr 1347987870], length 0 01:50:43.693420 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq 31:1055, ack 39, win 33, options [nop,nop,TS val 1918473331 ecr 1347987870], length 1024 01:50:43.693435 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq 1055:1543, ack 39, win 33, options [nop,nop,TS val 1918473331 ecr 1347987870], length 488 01:50:43.818391 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [P.], seq 39:1119, ack 31, win 33, options [nop,nop,TS val 1347988093 ecr 1918473291], length 1080 01:50:43.819870 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq 1543:1591, ack 1119, win 33, options [nop,nop,TS val 1918473457 ecr 1347988093], length 48 01:50:43.880995 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [.], ack 1543, win 33, options [nop,nop,TS val 1347988163 ecr 1918473331], length 0 01:50:43.991756 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [P.], seq 1119:1571, ack 1591, win 33, options [nop,nop,TS val 1347988277 ecr 1918473457], length 452 = The tunnel is configured using `wg-quick'. The firewalls are unloaded for this testing. I have made sure to delete the if_wg.ko shipped with FreeBSD, and rebooted the host before trying this. And ofcourse, if I switch to userspace Go implementation, everything works as expected, keeping rest of the configuration same, and with firewalls enabled. Thanks! -- Ashish OpenPGP_signature Description: OpenPGP digital signature
Unable to serve TCP connections over wireguard interface on FreeBSD 13.0-RC2
Hi, I'm running if_wg kernel module (git revision: 5ef4d3efa691e71) on FreeBSD 13.0-RC2. With 172.18.10.1 being my local host's wireguard interface's IP address, I can receive SYN packets, but it does not seem to send any corresponding SYN/ACK. = 01:26:26.327484 IP 172.18[.10.3.34160 > 172.18.10.1.22: Flags [S], seq 1278197331, win 64860, options [mss 1380,sackOK,TS val 223949166 ecr 0,nop,wscale 7], length 0 01:26:42.708175 IP 172.18.10.3.34160 > 172.18.10.1.22: Flags [S], seq 1278197331, win 64860, options [mss 1380,sackOK,TS val 223965550 ecr 0,nop,wscale 7], length 0 01:27:14.964162 IP 172.18.10.3.34160 > 172.18.10.1.22: Flags [S], seq 1278197331, win 64860, options [mss 1380,sackOK,TS val 223997806 ecr 0,nop,wscale 7], length 0 01:28:34.035384 IP 172.18.10.2.41905 > 172.18.10.1.22: Flags [S], seq 2569759726, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 3991744006 ecr 0], length 0 01:28:34.035392 IP 172.18.10.2.41905 > 172.18.10.1.22: Flags [S], seq 2569759726, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 3991745042 ecr 0], length 0 01:28:34.036002 IP 172.18.10.2.41905 > 172.18.10.1.22: Flags [S], seq 2569759726, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 3991747129 ecr 0], length 0 = ICMP works fine: = 01:53:15.638529 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 0, length 64 01:53:15.638535 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 0, length 64 01:53:16.624443 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 1, length 64 01:53:16.624448 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 1, length 64 01:53:17.672109 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 2, length 64 01:53:17.672115 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 2, length 64 01:53:18.676223 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 3, length 64 01:53:18.676230 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 3, length 64 01:53:19.682131 IP 172.18.10.2 > 172.18.10.1: ICMP echo request, id 47881, seq 4, length 64 01:53:19.682136 IP 172.18.10.1 > 172.18.10.2: ICMP echo reply, id 47881, seq 4, length 64 = And I can make outbound TCP connections: = 01:50:43.267331 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [S], seq 2119392003, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 1918472905 ecr 0], length 0 01:50:43.415524 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [S.], seq 2602046635, ack 2119392004, win 65535, options [mss 1380,nop,wscale 11,sackOK,TS val 1347987709 ecr 1918472905], length 0 01:50:43.415532 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [.], ack 1, win 33, options [nop,nop,TS val 1918473053 ecr 1347987709], length 0 01:50:43.415613 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq 1:31, ack 1, win 33, options [nop,nop,TS val 1918473053 ecr 1347987709], length 30 01:50:43.614035 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [P.], seq 1:39, ack 31, win 33, options [nop,nop,TS val 1347987870 ecr 1918473053], length 38 01:50:43.653218 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [.], ack 39, win 33, options [nop,nop,TS val 1918473291 ecr 1347987870], length 0 01:50:43.693420 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq 31:1055, ack 39, win 33, options [nop,nop,TS val 1918473331 ecr 1347987870], length 1024 01:50:43.693435 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq 1055:1543, ack 39, win 33, options [nop,nop,TS val 1918473331 ecr 1347987870], length 488 01:50:43.818391 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [P.], seq 39:1119, ack 31, win 33, options [nop,nop,TS val 1347988093 ecr 1918473291], length 1080 01:50:43.819870 IP 172.18.10.1.55541 > 172.18.10.2.22: Flags [P.], seq 1543:1591, ack 1119, win 33, options [nop,nop,TS val 1918473457 ecr 1347988093], length 48 01:50:43.880995 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [.], ack 1543, win 33, options [nop,nop,TS val 1347988163 ecr 1918473331], length 0 01:50:43.991756 IP 172.18.10.2.22 > 172.18.10.1.55541: Flags [P.], seq 1119:1571, ack 1591, win 33, options [nop,nop,TS val 1347988277 ecr 1918473457], length 452 = The tunnel is configured using `wg-quick'. The firewalls are unloaded for this testing. I have made sure to delete the if_wg.ko shipped with FreeBSD, and rebooted the host before trying this. And ofcourse, if I switch to userspace Go implementation, everything works as expected, keeping rest of the configuration same, and with firewalls enabled. Thanks! -- Ashish OpenPGP_signature Description: OpenPGP digital signature
Re: Removing WireGuard Support From FreeBSD Base
On dv., març 19 2021, Gordon Bergling wrote:
On Wed, Mar 17, 2021 at 12:34:02PM -0600, Jason A. Donenfeld
wrote:
Hi Gordon,
On Wed, Mar 17, 2021 at 6:53 AM Gordon Bergling
wrote:
> I am not sure, if the removal is a great idea, a removal from
> releng/13 and stable/13 - possibly yes, but from main?
>
> This is still -CURRENT and -CURRENT should be central place
> for development,
> even if we have phabricator for review.
It looks like Kyle has gone ahead with the revert anyway, so
development is now happening at:
https://git.zx2c4.com/wireguard-freebsd/
And there are now regular snapshot releases:
https://lists.zx2c4.com/pipermail/wireguard/2021-March/006518.html
As for your objections, and the question of what -CURRENT
should or
shouldn't be used for, I really have no idea as a community
outsider.
But I do look forward to submitting it for proper inclusion in
-CURRENT after a few more cycles of development and refinement.
There's also the crypto question that I'd welcome some feedback
on:
https://lists.freebsd.org/pipermail/freebsd-hackers/2021-March/057076.html
> If the complete backout is happening, please don't forget the
> manual
> page. I have spend a lot of time on it, while OpenBSD made a
> good
> template.
Thanks for bringing this up; I had actually forgotten about
that. Do
you want to re-add it and keep that current as we develop? If
you
email me your SSH key, you can just commit it directly.
Jason
Thanks for the reply. I still think that the removal from main
was a mistake,
but it has happened.
I'll create a port for WireGuard tomorrow so that FreeBSD isn't
losing WireGuard
support at all, for whatever reason.
--Gordon
If you do that, please take following tiny patch into account
(missing from the git repo @zx2c4, posted to the WG ML awaiting
moderation):
This is due to the removal commit form stable/13 and, from what I
saw, didn't affect CURRENT or 12.
---
src/compat.h | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/src/compat.h b/src/compat.h
index 6126e26..bc29c01 100644
--- a/src/compat.h
+++ b/src/compat.h
@@ -7,6 +7,9 @@
*/
#include
+#if __FreeBSD_version < 140
+#include
+#include
#if __FreeBSD_version < 130
#define VIMAGE
@@ -18,8 +21,6 @@
#include
#include
#include
-#include
-#include
#include
#include
#include
@@ -39,6 +40,7 @@
#undef atomic_load_ptr
#define atomic_load_ptr(p) (*(volatile __typeof(*p) *)(p))
+#endif /* __FreeBSD_version < 130 */
struct taskqgroup_cpu {
LIST_HEAD(, grouptask) tgc_tasks;
@@ -67,7 +69,7 @@ static inline void taskqgroup_drain_all(struct
taskqgroup *tqg)
gtaskqueue_drain_all(q);
}
}
-#endif
+#endif /* __FreeBSD_version < 140 */
#if __FreeBSD_version < 1202000
static inline uint32_t arc4random_uniform(uint32_t bound)
--
2.30.1
[PATCH] Fix build on FreeBSD 13 after removal from base
\o Hello,
noticed that after if_wg removal from stable/13 following patch is
needed for compiling the module.
I'm currently lacking a 14/CURRENT machine to assert that the
module compiles with these changes, but from checking the branch,
it looks like these definitions are there already, I wouldn't
think versions pre-13 should be affected by this.
Cheers and thank you for the efforts and... handling of things.
---
src/compat.h | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/src/compat.h b/src/compat.h
index 6126e26..bc29c01 100644
--- a/src/compat.h
+++ b/src/compat.h
@@ -7,6 +7,9 @@
*/
#include
+#if __FreeBSD_version < 140
+#include
+#include
#if __FreeBSD_version < 130
#define VIMAGE
@@ -18,8 +21,6 @@
#include
#include
#include
-#include
-#include
#include
#include
#include
@@ -39,6 +40,7 @@
#undef atomic_load_ptr
#define atomic_load_ptr(p) (*(volatile __typeof(*p) *)(p))
+#endif /* __FreeBSD_version < 130 */
struct taskqgroup_cpu {
LIST_HEAD(, grouptask) tgc_tasks;
@@ -67,7 +69,7 @@ static inline void taskqgroup_drain_all(struct
taskqgroup *tqg)
gtaskqueue_drain_all(q);
}
}
-#endif
+#endif /* __FreeBSD_version < 140 */
#if __FreeBSD_version < 1202000
static inline uint32_t arc4random_uniform(uint32_t bound)
--
2.30.1
Re: Removing WireGuard Support From FreeBSD Base
Kyle, you (and others) have been put in a position you never should have been put in. You have earned respect from someone who could not be any more outside the community of core FreeBSD developers for behaving like an adult and publicly sharing your introspection. I’m sorry if this is seen as an inappropriate comment from an outsider, but I felt it important to say that the way you have conducted yourself over the past few days has been noticed, noted and appreciated. As a consumer of FreeBSD I move forward with my belief in the product stronger than ever.
Re: [ANNOUNCE] WireGuard for FreeBSD in development for 13.y – and a note of how we got here
On Fri, Mar 19, 2021 at 8:03 AM Kyle Evans wrote: > > On Mon, Mar 15, 2021 at 9:47 AM Jason A. Donenfeld wrote: > > > > Hi everybody, > > > > [... snip ...] > > > > The first step was assessing the current state of the code the previous > > developer had dumped into the tree. It was not pretty. I imagined > > strange Internet voices jeering, “this is what gives C a bad name!” > > This was a highly unnecessary jab. > > > There were random sleeps added to “fix” race conditions > > This is true. > > > validation functions that just returned true > > I looked back at the history here just now, and I count one, > wg_allowedip_valid, that pretty much got ripped out anyways. > > > catastrophic cryptographic vulnerabilities > > whole parts of the protocol unimplemented > > I'm not qualified to speak about these two, which are perhaps the > worst from the public's perspective. > > > kernel panics > > These are true. > > > security bypasses > > overflows > > I only recall one of each. > > > random printf statements deep in crypto code > > This is true. > > > the most spectacular buffer overflows > > English is a crappy language, but this sounds like you're advertising > more than one. There was one, and for the record to anyone watching to > dispel "the word on the street" that it's potentially an RCE: based on what's > happening I cannot imagine how you could usefully turn it into an RCE. > Local privileged execution, 100%, but looking at it further, you've got to be > pretty skilled to make it before killing the kernel elsewhere. > Ah, I have to clarify this one; I suspect if you're acting as a wg gateway, it's possible. It still doesn't look like an easy one to exploit. > > and the whole litany of awful things that go wrong when people aren’t > > careful when they write C. > > This is an exceedingly broad statement, and it would have been good to provide > some pointers to these. > > > [...] > > You know that I don't appreciate how you handled this initial communication, > as I've told you a number of times. Now that I look at the history again, I'm > even more disappointed in how you handled this because there are some > pretty broad statements in here and language that could go either way. > > I've additionally recommended, as a developer and not in any kind of official > capacity, that we can't include if_wg in any future version of base. We > cannot > have our users being put at the risk of this kind of publicity if we > can't get security > advisories issued in time. Ports is a fine place, where security issues can be > addressed expeditiously and more in line with how the WireGuard project > chooses > to handle them. > > Thanks, > > Kyle Evans
Re: [ANNOUNCE] WireGuard for FreeBSD in development for 13.y – and a note of how we got here
On Mon, Mar 15, 2021 at 9:47 AM Jason A. Donenfeld wrote: > > Hi everybody, > > [... snip ...] > > The first step was assessing the current state of the code the previous > developer had dumped into the tree. It was not pretty. I imagined > strange Internet voices jeering, “this is what gives C a bad name!” This was a highly unnecessary jab. > There were random sleeps added to “fix” race conditions This is true. > validation functions that just returned true I looked back at the history here just now, and I count one, wg_allowedip_valid, that pretty much got ripped out anyways. > catastrophic cryptographic vulnerabilities > whole parts of the protocol unimplemented I'm not qualified to speak about these two, which are perhaps the worst from the public's perspective. > kernel panics These are true. > security bypasses > overflows I only recall one of each. > random printf statements deep in crypto code This is true. > the most spectacular buffer overflows English is a crappy language, but this sounds like you're advertising more than one. There was one, and for the record to anyone watching to dispel "the word on the street" that it's potentially an RCE: based on what's happening I cannot imagine how you could usefully turn it into an RCE. Local privileged execution, 100%, but looking at it further, you've got to be pretty skilled to make it before killing the kernel elsewhere. > and the whole litany of awful things that go wrong when people aren’t > careful when they write C. This is an exceedingly broad statement, and it would have been good to provide some pointers to these. > [...] You know that I don't appreciate how you handled this initial communication, as I've told you a number of times. Now that I look at the history again, I'm even more disappointed in how you handled this because there are some pretty broad statements in here and language that could go either way. I've additionally recommended, as a developer and not in any kind of official capacity, that we can't include if_wg in any future version of base. We cannot have our users being put at the risk of this kind of publicity if we can't get security advisories issued in time. Ports is a fine place, where security issues can be addressed expeditiously and more in line with how the WireGuard project chooses to handle them. Thanks, Kyle Evans
Re: Removing WireGuard Support From FreeBSD Base
On Wed, Mar 17, 2021 at 12:34:02PM -0600, Jason A. Donenfeld wrote: > Hi Gordon, > > On Wed, Mar 17, 2021 at 6:53 AM Gordon Bergling wrote: > > I am not sure, if the removal is a great idea, a removal from > > releng/13 and stable/13 - possibly yes, but from main? > > > > This is still -CURRENT and -CURRENT should be central place for development, > > even if we have phabricator for review. > > It looks like Kyle has gone ahead with the revert anyway, so > development is now happening at: > > https://git.zx2c4.com/wireguard-freebsd/ > > And there are now regular snapshot releases: > > https://lists.zx2c4.com/pipermail/wireguard/2021-March/006518.html > > As for your objections, and the question of what -CURRENT should or > shouldn't be used for, I really have no idea as a community outsider. > But I do look forward to submitting it for proper inclusion in > -CURRENT after a few more cycles of development and refinement. > There's also the crypto question that I'd welcome some feedback on: > > https://lists.freebsd.org/pipermail/freebsd-hackers/2021-March/057076.html > > > If the complete backout is happening, please don't forget the manual > > page. I have spend a lot of time on it, while OpenBSD made a good > > template. > > Thanks for bringing this up; I had actually forgotten about that. Do > you want to re-add it and keep that current as we develop? If you > email me your SSH key, you can just commit it directly. > > Jason Thanks for the reply. I still think that the removal from main was a mistake, but it has happened. I'll create a port for WireGuard tomorrow so that FreeBSD isn't losing WireGuard support at all, for whatever reason. --Gordon signature.asc Description: PGP signature
Re: Removing WireGuard Support From FreeBSD Base
Hi Kyle, On Thu, Mar 18, 2021 at 10:53 AM Kyle Evans wrote: > involved with this announcement that I'm leaving it for now. There's > been too much press surrounding this, and it's distracting me from the > work that I like to do and what I'm typically known for. Makes sense and is understandable. It's been pretty miserable for all of us. It looks like we'll eventually find somebody on the FreeBSD side of things to take over where you left off, but hopefully for now in the coming weeks things can just level out to some tranquility, so we can get back to distraction-free coding without all the drama. Jason
Re: Removing WireGuard Support From FreeBSD Base
On Tue, Mar 16, 2021 at 11:48 AM Kyle Evans wrote: > > Hi, > > You may have recently noticed some chatter around the internet about > FreeBSD's in-kernel WireGuard implementation, and the work we've done > on it in the last week. You may have also noticed additional chatter > afterwards with regards to the original implementation. I'd like to give > some context and information with regards to the current situation, as > well as provide some insight into the future as one of the developers > involved. > I'm afraid I must follow this one up with an announcement that I'm stepping back from the wireguard-freebsd efforts. You'll likely hear some things about me in the coming days. I'm sure I've said more things in the past week and a half in anger that I regret, and I'll have to live with that. I'd like to set the record straight. Netgate personnel were involved in part with my announcement of removal. I did not take a number of suggestions, because I wrote what I believed in and I continue to do so. Netgate is in no way involved with this announcement that I'm leaving it for now. There's been too much press surrounding this, and it's distracting me from the work that I like to do and what I'm typically known for. The wireguard-freebsd project will need a new maintainer. Thanks, Kyle Evans
