Re: TLS suddenly not working over IKED site-to-site - SOLVED?
Den tors 14 mars 2019 kl 21:51 skrev Zhi-Qiang Lei : > Mine is resolved by applying a smaller max-mss in pf and disabling ipcomp. > Only disabling ipcomp didn’t work. > > > On Thu, Dec 20, 2018 at 6:54 PM Theodore Wynnychenko > wrote: > >> Then, I took the advice above, and disable ipcomp on the tunnel, and, > BAHM, https (and imaps) were working without an issue from openbsd, Windows > 7, and Macs! > I ran into something similar a while ago, and even if "fixing" https/imaps works with mss clamping, it will still cause issues with fragmented UDP and large icmp, since those will not care about mss, only TCP does. The problem is still there, its just a tcp-only workaround to lower mss in-flight for a problem that is mostly visible when doing *s services since they ship long lists of preferred algorithms which causes large packets to be sent, whereas simple ldap lookups or ntp/dns/http get by with less info sent and hence send smaller packets. Still, large non-tcp ip will see unexpected drops in such scenarios where you only lower mss and not the MTU on some in-between L3 interface so it correctly fragments when needed. -- May the most significant bit of your life be positive.
Re: TLS suddenly not working over IKED site-to-site - SOLVED?
Mine is resolved by applying a smaller max-mss in pf and disabling ipcomp. Only disabling ipcomp didn’t work. > On Mar 15, 2019, at 3:15 AM, Andrew Daugherity > wrote: > > On Thu, Dec 20, 2018 at 6:54 PM Theodore Wynnychenko > wrote: >> Then, I took the advice above, and disable ipcomp on the tunnel, and, BAHM, >> https (and imaps) were working without an issue from openbsd, Windows 7, and >> Macs! >> >> Just to be sure, I updated this am to the 12/19 amd64 snapshot. >> >> When I turn on ipcomp, https/imaps hangs for most connections; when I turn >> ipcomp off, https/imaps works. > > I can confirm this behavior. I've set up a simple RSA key VPN as > described at http://www.openbsd.org/faq/faq17.html#site2site, which > does not include ipcomp by default, and everything works fine, > including https. After reading this I decided to test enabling > ipcomp, and sure enough, loading an https page across the VPN fails. > With ipcomp I also see some "unprotected" packets when running tcpdump > on enc0, e.g.: > 13:32:19.600062 (authentic,confidential): SPI 0xee345270: > 10.95.10.236.57254 > 10.95.0.233.443: P 273:518(245) ack 5604 win 455 > (DF) (encap) > 13:32:19.614996 (unprotected): SPI 0x5a04: 10.95.0.233.443 > > 10.95.10.236.57254: . 5604:7052(1448) ack 518 win 252 timestamp 61011950 1069884950> (DF) (encap) > > I don't know why that is happening, but as everything seems to work > well and perform decently without ipcomp, I'll be leaving it disabled. > >> I noticed that the last change to sys/netinet/ip_ipcomp.c (I am guessing >> this is the code that is involved) in the log (I think) was about 3 months >> ago, and at this point, I can't recall if my last updated (prior to the one >> where the instability began) was before or after that change. >> >> I was going to try to recompile it with the change undone, but am not sure >> how to do that, or even if it can be done for just that one part of sys. > > Yes, just use git or cvs (whatever you checked out the code with) to > fetch an earlier revision of that file (not the whole repo) and then > build a new kernel. Sometimes you'd need to also revert other related > changes, but that does not appear to be the case here, assuming you're > referring to [1]. Note that some previous commits did touch multiple > files. > >> And, after removing ipcomp from iked.conf, my subjective observation is that >> things load a lot faster than they seemed to in the past with ipcomp on; so, >> I am happy with where I am. >> >> I was just posting my observations in case anyone else has a similar issue. > > Thank you for sharing. I had (I think) been using ipcomp in my old > ikev1 (ipsec.conf/isakmpd) setup but had not yet gotten around to > enabling it in the ikev2 setup. Based on this, I won't bother. > > > -Andrew > > [1] https://github.com/openbsd/src/commit/4b5fa55 >
Re: TLS suddenly not working over IKED site-to-site - SOLVED?
On Thu, Dec 20, 2018 at 6:54 PM Theodore Wynnychenko wrote: > Then, I took the advice above, and disable ipcomp on the tunnel, and, BAHM, > https (and imaps) were working without an issue from openbsd, Windows 7, and > Macs! > > Just to be sure, I updated this am to the 12/19 amd64 snapshot. > > When I turn on ipcomp, https/imaps hangs for most connections; when I turn > ipcomp off, https/imaps works. I can confirm this behavior. I've set up a simple RSA key VPN as described at http://www.openbsd.org/faq/faq17.html#site2site, which does not include ipcomp by default, and everything works fine, including https. After reading this I decided to test enabling ipcomp, and sure enough, loading an https page across the VPN fails. With ipcomp I also see some "unprotected" packets when running tcpdump on enc0, e.g.: 13:32:19.600062 (authentic,confidential): SPI 0xee345270: 10.95.10.236.57254 > 10.95.0.233.443: P 273:518(245) ack 5604 win 455 (DF) (encap) 13:32:19.614996 (unprotected): SPI 0x5a04: 10.95.0.233.443 > 10.95.10.236.57254: . 5604:7052(1448) ack 518 win 252 (DF) (encap) I don't know why that is happening, but as everything seems to work well and perform decently without ipcomp, I'll be leaving it disabled. > I noticed that the last change to sys/netinet/ip_ipcomp.c (I am guessing this > is the code that is involved) in the log (I think) was about 3 months ago, > and at this point, I can't recall if my last updated (prior to the one where > the instability began) was before or after that change. > > I was going to try to recompile it with the change undone, but am not sure > how to do that, or even if it can be done for just that one part of sys. Yes, just use git or cvs (whatever you checked out the code with) to fetch an earlier revision of that file (not the whole repo) and then build a new kernel. Sometimes you'd need to also revert other related changes, but that does not appear to be the case here, assuming you're referring to [1]. Note that some previous commits did touch multiple files. > And, after removing ipcomp from iked.conf, my subjective observation is that > things load a lot faster than they seemed to in the past with ipcomp on; so, > I am happy with where I am. > > I was just posting my observations in case anyone else has a similar issue. Thank you for sharing. I had (I think) been using ipcomp in my old ikev1 (ipsec.conf/isakmpd) setup but had not yet gotten around to enabling it in the ikev2 setup. Based on this, I won't bother. -Andrew [1] https://github.com/openbsd/src/commit/4b5fa55
Re: TLS suddenly not working over IKED site-to-site - SOLVED?
> -Original Message- > From: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] On Behalf > Of William Ahern > Sent: Monday, December 17, 2018 1:11 PM > To: Theodore Wynnychenko > Cc: misc@openbsd.org > Subject: Re: TLS suddenly not working over IKED site-to-site > . . . > I'm not well versed in these issues but if I were in your shoes I would > > 1) Figure out why those packets were unprotected. (Could be normal for > all I > know, but in a quick test on my enc0 I didn't see any packets like > that.) > > 2) Make sure the tunnel handles fragmentation correctly. Are fragments > being > dropped? Are reassembled fragments being dropped? > > 2.a) Relatedly, make sure the tunnel handles pMTU discovery. Do ICMP > Fragmentation Needed packets make it back through the tunnel? pMTU and > ICMP > issues are very common with IPSec tunnels. IME most people "fix" these > issues by forcing a lower MSS or setting a lower MTU at the ingress > point > rather than properly configuring routing so ICMP errors are properly > routed. > I've experienced this issue myself and had to learn the hard way. > > 3) From an earlier post it looks like you're using ipcomp. You should > remove > this complication while debugging. It's possible ipcomp is hiding MTU > issues. Thank you so much for the suggestions. To summarize, I have noticed that in the last month, SSL/TLS connections were failing when traversing an ipsec tunnel created by iked. This had worked stably for over a year, with no changes to iked.conf or pf.conf. In trying to find the issue, I had added "max-mss" to pf and tried decreasing MTU values on the adapters. This did not seem to make a difference. In addition, the problem was very sporadic, and seemed to "evolve" over the last few weeks. In the last few days, I was able to establish https connections over the tunnel when that connection was initiated by the gateway openbsd machine or a Mac on the "local" network; but connections from another openbsd machine "behind" the gateway, and a Windows 7 machine kept hanging. Anyway, I decided to revert everything to the way it was. I removed all "max-mss" entries and reset MTU values to 1500. Then, I took the advice above, and disable ipcomp on the tunnel, and, BAHM, https (and imaps) were working without an issue from openbsd, Windows 7, and Macs! Just to be sure, I updated this am to the 12/19 amd64 snapshot. When I turn on ipcomp, https/imaps hangs for most connections; when I turn ipcomp off, https/imaps works. I noticed that the last change to sys/netinet/ip_ipcomp.c (I am guessing this is the code that is involved) in the log (I think) was about 3 months ago, and at this point, I can't recall if my last updated (prior to the one where the instability began) was before or after that change. I was going to try to recompile it with the change undone, but am not sure how to do that, or even if it can be done for just that one part of sys. And, after removing ipcomp from iked.conf, my subjective observation is that things load a lot faster than they seemed to in the past with ipcomp on; so, I am happy with where I am. I was just posting my observations in case anyone else has a similar issue. Ted
