Hi Jim,

> On May 14, 2017, at 6:38 PM, Jim Thompson <j...@netgate.com> wrote:
>> 3. Create one or more P2s to make selectors for traffic inclusion and 
>> exclusion. Note there's a bug in PFSense, where if you do IPSec from not-LAN 
>> interfaces, the traffic to the firewall's IP gets stolen unless you manually 
>> hack the PHP file that generates the IPSec traffic selector configuration, 
>> to 
>> whitelist more interfaces. This prevents being able to do Ping, DNS, NTP, 
>> etc. 
>> all other services against the firewall on non-LAN if IPSec is on. Bad times 
>> right there.
> 
> Additional details would be great here.  Even better would be to open a bug 
> on redmine.pfsense.org with these additional details.
> 

I did discuss this problem previously in the mailing list and forum. I was 
seeking some views on the topic, before I went forward with filing a defect, as 
IPSec traffic selectors are an area I don't profess to understand incredibly 
well, and I wasn't 100% sure I didn't miss something in my analysis, and didn't 
want to generate a bogus bug if so.

I found this when creating a restricted LAN on the OPT1 port that was allowed 
to use IPSec when the LAN connected to the LAN port was not supposed to use 
IPSec. Basically, it's a DMZ network inside a house, walled off from the normal 
network, with a separate wireless SSID and separate Ethernet ports, which is 
then IPSec connected to a colo facility, w/ the PFSense IPSec on both ends.

The issue happens here:

https://github.com/pfsense/pfsense/blob/e470f72139ed54972465e653e27536687ce58b23/src/etc/inc/vpn.inc#L807

If you look at this code, it doesn't exempt all of the firewall's own IPs or at 
least Internal IPs from getting captured by the IPSec selectors for any 
tunnels. So management / admin traffic / other helpers to and from the 
firewall, like Ping, DNS, NTP, DHCP / SLAAC, etc. don't get through or don't 
get replies because only the LAN IP gets exempted when the UI Checkbox for 
bypass is checked and not all of the other interfaces (or specifically chosen 
interfaces... it only has a checkbox for LAN and not for any others). Also it's 
only exempting IPv4 so IPv6 will get broken even more than IPv4 will, if you're 
doing IKEv2 w/ IPv6 tunnels on top, which I use heavily in my case.

I "fixed it" by hand-editing this file that generates the VPN setup to make 
more bypass exemptions, and promptly watching the issues stop happening after I 
added this hack.

> Don’t know what you mean by “broad”, but it’s all (multiple) /24s here.

It took quite some time, for example, to get ::/0 in IPv6 routing across my 
tunnel w/o malfunctions. And the same would apply using 0.0.0.0/0, and it was 
very critical to read and follow this document, and the logical equivalent 
behavior for IPv6, to the letter for it to work.

Regarding when the issues hit exactly... it can happen if you aren't really 
careful to make sure that the selectors grabbing big swaths of IP space on one 
tunnel end, aren't carefully restricted to a narrow range of IP space on the 
other tunnel end. It's not saying PFSense did something wrong, but more that 
with the IPSec stuff, you have to be extremely judicious with the setup of the 
selectors, to prevent them from stealing unexpected traffic and sending it in 
the tunnel. If you make typos here or mess up, you can make your firewall 
unreachable (especially w/ the bypass issues I wrote about above), and have to 
come in from the console to roll things back if they aren't set up 100% 
perfectly.

>> 10. Instead of the MOBIKE and DPD crap, keep the tunnel up, by using valid 
>> IPs 
>> on PFSense on other end of tunnel in the P2 auto-ping host entry. This will 
>> keep the IPSec up all the time and keep it from getting foobarred, unless 
>> the 
>> link itself has a gnarly outage, in which case you're down regardless.
>> 
>> 11. On both the P1 and P2, lock down the list of KEX, Enc, and Auth 
>> algorithms 
>> to a single solid algorithm. If the negotiation screws up, it causes weird 
>> connection problems which you will damage your brain trying to debug.
> 
> All of this is of interest, and deeply appreciated, but I’ve got an IPsec 
> connection between home and work that has been stable since … a couple years 
> ago.

I have very reliably working connections now as well, but only after hacking on 
vpn.inc as described above, and only when MOBIKE, DPD, Rekey, are all disabled. 
Otherwise, in my case, when the tunnel times out due to rekey, MOBIKE, DPD, 
detecting drops or relocation of anything, the renegotiation that gets 
triggered seems to get stuck, and all the traffic going through the tunnel is 
getting blocked.

It's surely possible the UVerse router was causing the brokenness. I can play 
with the settings some more on the cleaner Comcast connection to see if I can 
reproduce it again.

I forgot to mention another item, in the last go-around, which seemed to 
prevent me from getting as many IPSec tunnels stuck, timeouts, etc.: I always 
set System -> Advanced -> Firewall & NAT -> Firewall Optimization Options to 
Conservative on both ends of the tunnel. It seems to prevent the various 
sockets for IKE, IPSec, NAT-T, etc. from getting closed as aggressively if 
traffic is quiet for a while.

In my case, I couldn't really reliably configure my tunnels to the Initiator + 
Responder mode, because most of my endpoints are Dynamic on Client side and 
Static only on Colo Provider side.

Matthew.
_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

Reply via email to