Ramiro Aceves <[email protected]> writes: > #!/bin/sh > set -x > ifconfig wg0 create mtu 1380 > #ifconfig wg0 create mtu 1280 > ifconfig wg0 inet 44.27.132.76/32 > ifconfig wg0 inet6 fe80::644d:cf7a:c00:bae9/128 > wgconfig wg0 set private-key /etc/wg/wg0.priv > wgconfig wg0 add peer A \ > asdfasdfasdfasdfasdfasdf= \ > --allowed-ips=0.0.0.0/0,::/0 \ > --endpoint=44.27.227.1:44000 > ifconfig wg0 up
You should add a comment explaining why you chose 1380. Any straightforward tunneling will take the inner packet (from wg0 in your case), process it, perhaps rounding up length depending on crypto algorithm, perhaps insert some kind of application-layer header, and then place it in another packet, with an IP header, and often UDP. Based on those sizes, there is a calculated max feasible MTU for the inner interface that results in not exceeding the MTU of the outer interface. Beyond that, there is the question of the minimum MTU along the path. If a packet exceeds MTU, it can either be fragmented (v4 only) or dropped. Increasingly the world has moved to a don't-fragemnt approach which causes an ICMP reply that the packet is too big, causing the source send smaller packets. With firewalls, tunneling, etc. this can go badly if those packets are not both allowed and translated properly. > netbsd-raspaZeroW# cat cambia_rutas.sh > #!/bin/sh > set -x > route add 44.27.227.1 192.168.1.1 > route delete default > route add default 44.27.132.76 This is more straightforward than 0/1 and 128/1. > As man page says, only 0 or 1 values seem to be valid. > > tcp.mss_ifmtu > If set to 1, TCP calculates the outgoing maximum segment > size based on the MTU of the appropriate interface. If > set to 0, it is calculated based on the greater of the > MTU of the interface, and the largest (non-loopback) > interface MTU on the system. You've shown the code, and it's quite clear it is 0 or not zero, and the man page documents 0 and 1. Thus it is a configuration error to set any other value; don't do that! > I can run now successful ssh sessions from outside. This is a huge clue, almost proof, that your setup has Path MTU discovery problems. I would suggest reading: https://en.wikipedia.org/wiki/Path_MTU_Discovery https://datatracker.ietf.org/doc/html/rfc1191 maybe https://datatracker.ietf.org/doc/html/rfc4821 I looked at your FreeBSD tcpdump but the wg0/real don't cover the same time period, so it really isn't helpful. I was trying to suggest that you run them both at once, -w to a file, -s1500 with *no filters*, and then after the experiment analyze them with -r and then using filters to omit things that you have decided don't matter. Probably you should redo tcpdump and see if you can see the ICMP Fragmentation Needed replies. >From your earlier tcpdump > 07:13:37.531243 IP 90.167.219.193.44127 > 44.27.132.76.22: Flags [S], seq > 145184924, win 65535, options [mss 1410,sackOK,TS val 851808448 ecr > 0,nop,wscale 10], length 0 > 07:13:37.531257 IP 44.27.132.76.22 > 90.167.219.193.44127: Flags [S.], seq > 1482509222, ack 145184925, win 32768, options [mss 1460,nop,wscale > 3,sackOK,TS val 1 ecr 851808448], length 0 > 07:13:37.701029 IP 90.167.219.193.44127 > 44.27.132.76.22: Flags [.], ack 1, > win 86, options [nop,nop,TS val 851808627 ecr 1], length 0 connection opened. Trace is clearly at the 44 side because the timestamps from SYN to SYNACK are only 14 us apart, whereas the SYNACK to ACK is 169 ms ish. > 07:13:37.708854 IP 90.167.219.193.44127 > 44.27.132.76.22: Flags [P.], seq > 1:23, ack 1, win 86, options [nop,nop,TS val 851808631 ecr 1], length 22 > 07:13:37.745163 IP 44.27.132.76.22 > 90.167.219.193.44127: Flags [P.], seq > 1:64, ack 23, win 4194, options [nop,nop,TS val 1 ecr 851808631], length 63 > 07:13:37.920768 IP 90.167.219.193.44127 > 44.27.132.76.22: Flags [.], ack 64, > win 86, options [nop,nop,TS val 851808841 ecr 1], length 0 > 07:13:37.920794 IP 44.27.132.76.22 > 90.167.219.193.44127: Flags [P.], seq > 64:1184, ack 23, win 4197, options [nop,nop,TS val 2 ecr 851808841], length > 1120 This seems ok > 07:13:37.929014 IP 90.167.219.193.44127 > 44.27.132.76.22: Flags [P.], seq > 1421:1591, ack 64, win 86, options [nop,nop,TS val 851808847 ecr 1], length > 170 This is a packet covering 1421:1591, but the previous seq was 1:23. so 24:1420 is missing. Surely that was sent from 90. and dropped. So you need to tcpdump (again, -wFILE and -s1500, no filters, as always) on the remote machine, and on the wg client's real interface. > 07:13:37.929029 IP 44.27.132.76.22 > 90.167.219.193.44127: Flags [.], ack 23, > win 4197, options [nop,nop,TS val 2 ecr 851808841,nop,nop,sack 1 > {1421:1591}], length 0 you are acking 23 again with a sack block for the out-of-order data. > 07:13:38.139063 IP 90.167.219.193.44127 > 44.27.132.76.22: Flags [.], ack > 1184, win 89, options [nop,nop,TS val 851809069 ecr 2], length 0 This acks the 64:1184 segment. The 90. host should arguably be doing PMTU-D with blackhole detection (search and read on that). > 07:13:39.080829 IP 20.102.117.55.51803 > 44.27.132.76.143: Flags [S], seq > 2992133723, win 65535, options [mss 1440,nop,wscale 8,nop,nop,sackOK], length > 0 > 07:13:39.080840 IP 44.27.132.76.143 > 20.102.117.55.51803: Flags [R.], seq 0, > ack 2992133724, win 0, length 0 Looks like you are being probed! > Also think that sending a ping from the Rpi to the 44.27.132.76 IP is > mandatory to mantain the tunnel alive. If not, when time passes, It > becomes a bit lazy until it responds to the external requests. Have to > experiment that subject. You will have to understand from your tunnel provider what the timeout/keepalive rules are. It is not really surprising to have tunnels time out, but it's also sort of broken, and definitely broken if not documented clearly or the times are short (sub hour IMHO). This seems a separate issue from PMTU-D.
