Hi Jim, > On May 14, 2017, at 6:38 PM, Jim Thompson <j...@netgate.com> wrote: >> 3. Create one or more P2s to make selectors for traffic inclusion and >> exclusion. Note there's a bug in PFSense, where if you do IPSec from not-LAN >> interfaces, the traffic to the firewall's IP gets stolen unless you manually >> hack the PHP file that generates the IPSec traffic selector configuration, >> to >> whitelist more interfaces. This prevents being able to do Ping, DNS, NTP, >> etc. >> all other services against the firewall on non-LAN if IPSec is on. Bad times >> right there. > > Additional details would be great here. Even better would be to open a bug > on redmine.pfsense.org with these additional details. >
I did discuss this problem previously in the mailing list and forum. I was seeking some views on the topic, before I went forward with filing a defect, as IPSec traffic selectors are an area I don't profess to understand incredibly well, and I wasn't 100% sure I didn't miss something in my analysis, and didn't want to generate a bogus bug if so. I found this when creating a restricted LAN on the OPT1 port that was allowed to use IPSec when the LAN connected to the LAN port was not supposed to use IPSec. Basically, it's a DMZ network inside a house, walled off from the normal network, with a separate wireless SSID and separate Ethernet ports, which is then IPSec connected to a colo facility, w/ the PFSense IPSec on both ends. The issue happens here: https://github.com/pfsense/pfsense/blob/e470f72139ed54972465e653e27536687ce58b23/src/etc/inc/vpn.inc#L807 If you look at this code, it doesn't exempt all of the firewall's own IPs or at least Internal IPs from getting captured by the IPSec selectors for any tunnels. So management / admin traffic / other helpers to and from the firewall, like Ping, DNS, NTP, DHCP / SLAAC, etc. don't get through or don't get replies because only the LAN IP gets exempted when the UI Checkbox for bypass is checked and not all of the other interfaces (or specifically chosen interfaces... it only has a checkbox for LAN and not for any others). Also it's only exempting IPv4 so IPv6 will get broken even more than IPv4 will, if you're doing IKEv2 w/ IPv6 tunnels on top, which I use heavily in my case. I "fixed it" by hand-editing this file that generates the VPN setup to make more bypass exemptions, and promptly watching the issues stop happening after I added this hack. > Don’t know what you mean by “broad”, but it’s all (multiple) /24s here. It took quite some time, for example, to get ::/0 in IPv6 routing across my tunnel w/o malfunctions. And the same would apply using 0.0.0.0/0, and it was very critical to read and follow this document, and the logical equivalent behavior for IPv6, to the letter for it to work. Regarding when the issues hit exactly... it can happen if you aren't really careful to make sure that the selectors grabbing big swaths of IP space on one tunnel end, aren't carefully restricted to a narrow range of IP space on the other tunnel end. It's not saying PFSense did something wrong, but more that with the IPSec stuff, you have to be extremely judicious with the setup of the selectors, to prevent them from stealing unexpected traffic and sending it in the tunnel. If you make typos here or mess up, you can make your firewall unreachable (especially w/ the bypass issues I wrote about above), and have to come in from the console to roll things back if they aren't set up 100% perfectly. >> 10. Instead of the MOBIKE and DPD crap, keep the tunnel up, by using valid >> IPs >> on PFSense on other end of tunnel in the P2 auto-ping host entry. This will >> keep the IPSec up all the time and keep it from getting foobarred, unless >> the >> link itself has a gnarly outage, in which case you're down regardless. >> >> 11. On both the P1 and P2, lock down the list of KEX, Enc, and Auth >> algorithms >> to a single solid algorithm. If the negotiation screws up, it causes weird >> connection problems which you will damage your brain trying to debug. > > All of this is of interest, and deeply appreciated, but I’ve got an IPsec > connection between home and work that has been stable since … a couple years > ago. I have very reliably working connections now as well, but only after hacking on vpn.inc as described above, and only when MOBIKE, DPD, Rekey, are all disabled. Otherwise, in my case, when the tunnel times out due to rekey, MOBIKE, DPD, detecting drops or relocation of anything, the renegotiation that gets triggered seems to get stuck, and all the traffic going through the tunnel is getting blocked. It's surely possible the UVerse router was causing the brokenness. I can play with the settings some more on the cleaner Comcast connection to see if I can reproduce it again. I forgot to mention another item, in the last go-around, which seemed to prevent me from getting as many IPSec tunnels stuck, timeouts, etc.: I always set System -> Advanced -> Firewall & NAT -> Firewall Optimization Options to Conservative on both ends of the tunnel. It seems to prevent the various sockets for IKE, IPSec, NAT-T, etc. from getting closed as aggressively if traffic is quiet for a while. In my case, I couldn't really reliably configure my tunnels to the Initiator + Responder mode, because most of my endpoints are Dynamic on Client side and Static only on Colo Provider side. Matthew. _______________________________________________ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
