Hello all, one issue we have encountered with encapsulated packets is the IP fragment packets that are created when the would be encapsulated packet would exceed the MTU of an underlay interface. on non natted networks with firewalls that behave them selves the tunnels may work. however across the internet more often than not there will be some problematic nat / firewall implementations that block these IP fragments.
a good way of diagnosing the issue with tunnels is to use the ping -s and gradually increase the ping packet size (inside the tunnel and do a packet capture on the Underlay interface that is transmitting the VPN Encapsulated Ping Packets , if this packet capture is done on the physical interface of the devices at both ends of the tunnel one can see if the IP fragments are getting through... I found that this exercise was a good way of understanding how the packets are encapsulated in a given vpn protocol. and discover the conditions where IP fragments were being generated. if IP fragments are blocked or dropped or routed asymmetrically (packet ordering issues) then they are likely to break a great article on IP fragments and why they suck so bad is outlined here https://blog.cloudflare.com/ip-fragmentation-is-broken/ avoiding IP fragments: IP fragments on internet are avoided generally through PMTU discovery (mtu path discovery) but PMTU does not work beyond a Nat (if a smaller MTU interface exists behind a NAT then the smaller MTU will not be discovered. PMTU cannot properly account for underlay restrictions Inside a VPN the TCP MSS resizing eliminates fragments for tunneled TCP Packets, but the problem remains for non TCP IP payloads, by reducing MSS (and therefore the required MTU to support a connection) the need to generate IP Fragments is reduced, Restricting the MTU of interfaces of internal devices that generate alot of non TCP traffic can reduce fragmentation across the VPN (its horrible I know) the alternative is to setup the VPN so that it fragments gracefully Openvpn for instance mitigates this IP fragment issue for UDP vpns by introducing a UDP fragments, so (packets less than half the udp frag limit are sent as one packet) larger packets are broken into 2 equal sized encapsulated packets and transmitted, these UDP fragmented vpn encapsulated packets have the benefit, of having same source / destination, IPs, Ports and protocol (and are more likely to be accepted in Firewall State tables (and have the same Hash so they will be routed / switched along the same path (reducing packet ordering issues) (there is a disadvantage of doubling PPS requirements of your hardeare) bottom line captures on physical interfaces sending and receiving encapsulated vpn Packets at both sides of the vpn can help identify the IP fragment issue, and then steps to avoid it can be taken. on my own network we try to avoid fragments by increasing the Physical Interfaces MTU (underlay) to ensure the overlay VPNS can send full sized packets . (but this is difficult to achieve across the internet) I hope this helps, On Sun, 15 May 2022 at 07:03, Jason McIntyre <j...@kerhand.co.uk> wrote: > > On Sat, May 14, 2022 at 09:14:36PM -0000, Stuart Henderson wrote: > > On 2022-05-14, Georg Pfuetzenreuter <ge...@syscid.com> wrote: > > > pppoe(4) already has a section on this, possibly this could be used as a > > > start. > > > > It's not a great start really. Mixes up information about a method to > > set the pppoe MTU to 1500 (RFC4638) and using scrub, doesn't describe > > the problem (says "causing conflict" but this isn't very meaningful > > or really correct), and points at nonexistent "more information on MTU, > > MSS and NAT" as this isn't in pf.conf(5). > > > > > > hi. > > if there are issues in that text, feel free to suggest how to improve > it. > > - mixing mtu to 1500 and scrub: well, both concern issues with mtu. why > wouldn;t they be together in there? > > - "causing conflict": feel free to be more specific. it's not something > i have knowledge of > > - "more information in pf.conf": yes there is information in pf.conf on > mtu, mss, and nat, including the syntax for using them. again, why > wouldn;t we point people there? > > i'm happy to try and rework the text if you think it can be improved. > > jmc > -- Kindest regards, Tom Smyth.
