Hello, I have been doing a lot of IPSec stuff from PFSense to PFSense (all with latest stable code versions) lately. I have run into several different weird issues, where I could use some input how these could be fixed, and/or worked around. I tried to describe some of the issues in a forum post also, but at the time I hadn't gotten as far with debugging this issue: https://forum.pfsense.org/index.php?topic=127136.0 . Perhaps some defaults could be adjusted in PFSense to make things more reliable for others as well?
1. charon's default packet retransmission timer is way too slow. Your IPsec connection will lock up for many, many seconds and kill off TCPs, etc. riding above it, and make you lose SSH and such. I would love to be able to change these nice settings to fix it, but I can't find a way to put overrides into the UI. Can I stash them in *.d directories somewhere in the filesystem where PFSense wouldn't clobber the content instead? https://wiki.strongswan.org/projects/1/wiki/Retransmission charon.retransmit_tries charon.retransmit_timeout charon.retransmit_base I think something like this would help a lot: charon.retransmit_tries = 5 charon.retransmit_timeout = 2.0 charon.retransmit_base = 1.6 2. I have some questions about this magical IPSec setting: "Auto-exclude LAN address" aka "autoexcludelanaddress" aka "noshuntlaninterfaces". I have found several issues here. a) The setting doesn't work right if you use OPT1 to run your special IPSec network, which is completely walled off from the normal network, because it can only does one specific interface (LAN). I can't figure out how to work around it from the UI. Same as above... is there a *.d directory available or other method? b) The setting doesn't work right if you use IPv6, because it's hardcoded to IPv4 only. To me that seems like a bug. This is especially problematic if you use IKEv2 to run IPv4 and IPv6 P2s on your P1 for adding IPv6 to networks stuck inside of IPv4. Any services like NTP, default DNS Resolver / Forwarder, etc. are broken because the firewall's replies get sent into the enc0 device and not back onto the LAN as expected. Suspect code is located here: https://github.com/pfsense/pfsense/blob/e470f72139ed54972465e653e27536687ce58b23/src/etc/inc/vpn.inc Here is the part which is missing important parameters: conn bypasslan leftsubnet = {$lansa}/{$lansn} rightsubnet = {$lansa}/{$lansn} authby = never type = passthrough auto = route 3. I am getting problems where my IPSec traffic quits transmitting right after socket session timeouts and rekeys. I had to set the Firewall Optimization to Conservative in order to make the sockets stay open longer so the IPSec sockets on 500/udp and 4500/udp would not get broken and lose traffic. It did seem to help somewhat but now it's blowing up every time it rekeys the tunnel. a) Since these VPN ports are secretly opened up to the world using special auto-generated policies in the firewall rulechains, it seems like I can't make custom rules with longer timeouts to steal back control of the traffic, so I could apply a longer, effectively infinite custom session timeout on these sockets. b) For the loss of traffic after rekeys, at the moment I am still kind stumped. I tried disabling MOBIKE and DPD as described in https://forum.pfsense.org/index.php/topic,41617.0.html which made some improvement despite how one would expect the opposite result. I also tried the option to "Initiate IKEv2 reauthentication with a make-before-break" but it didn't help. On the Initiator side I get really weird stuff going on like this log below, and the timestamps are mere seconds before my traffic quits working on the tunnel. There is a lot of packet timeouts, right around the rekeying time, and no traffic can get through properly after that without power-cycling the tunnel. I wanted to see if I should change some other IPSec Advanced settings to take care of this, or if there was a way to disable rekeying temporarily to see if the issue will stop happening, as that apparently used to have an option but doesn't have one now. (Log below signature.) Thanks, Matthew. Mar 14 23:06:16 fw-01 charon: 14[KNL] creating rekey job for CHILD_SA ESP/0xc66be176/TARGET_IP Mar 14 23:06:16 fw-01 charon: 14[KNL] creating rekey job for CHILD_SA ESP/0xc66be176/TARGET_IP Mar 14 23:06:16 fw-01 charon: 15[IKE] establishing CHILD_SA con1{1} Mar 14 23:06:16 fw-01 charon: 15[IKE] <con1|24> establishing CHILD_SA con1{1} Mar 14 23:06:16 fw-01 charon: 15[ENC] generating CREATE_CHILD_SA request 12 [ N(REKEY_SA) N(ESP_TFC_PAD_N) SA No TSi TSr ] Mar 14 23:06:16 fw-01 charon: 15[ENC] <con1|24> generating CREATE_CHILD_SA request 12 [ N(REKEY_SA) N(ESP_TFC_PAD_N) SA No TSi TSr ] Mar 14 23:06:16 fw-01 charon: 15[NET] sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:06:16 fw-01 charon: 15[NET] <con1|24> sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:06:20 fw-01 charon: 14[IKE] retransmit 1 of request with message ID 12 Mar 14 23:06:20 fw-01 charon: 14[IKE] <con1|24> retransmit 1 of request with message ID 12 Mar 14 23:06:20 fw-01 charon: 14[NET] sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:06:20 fw-01 charon: 14[NET] <con1|24> sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:06:28 fw-01 charon: 09[IKE] retransmit 2 of request with message ID 12 Mar 14 23:06:28 fw-01 charon: 09[IKE] <con1|24> retransmit 2 of request with message ID 12 Mar 14 23:06:28 fw-01 charon: 09[NET] sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:06:28 fw-01 charon: 09[NET] <con1|24> sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:06:41 fw-01 charon: 14[IKE] retransmit 3 of request with message ID 12 Mar 14 23:06:41 fw-01 charon: 14[IKE] <con1|24> retransmit 3 of request with message ID 12 Mar 14 23:06:41 fw-01 charon: 14[NET] sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:06:41 fw-01 charon: 14[NET] <con1|24> sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:07:04 fw-01 charon: 09[IKE] retransmit 4 of request with message ID 12 Mar 14 23:07:04 fw-01 charon: 09[IKE] <con1|24> retransmit 4 of request with message ID 12 Mar 14 23:07:04 fw-01 charon: 09[NET] sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:07:04 fw-01 charon: 09[NET] <con1|24> sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:07:46 fw-01 charon: 14[IKE] retransmit 5 of request with message ID 12 Mar 14 23:07:46 fw-01 charon: 14[IKE] <con1|24> retransmit 5 of request with message ID 12 Mar 14 23:07:46 fw-01 charon: 14[NET] sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:07:46 fw-01 charon: 14[NET] <con1|24> sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (300 bytes) Mar 14 23:07:57 fw-01 charon: 09[KNL] creating rekey job for CHILD_SA ESP/0xccfdaec4/INITIATOR_IP Mar 14 23:07:57 fw-01 charon: 09[KNL] creating rekey job for CHILD_SA ESP/0xccfdaec4/INITIATOR_IP Mar 14 23:08:07 fw-01 charon: 02[CFG] received stroke: terminate 'con1' Mar 14 23:08:07 fw-01 charon: 02[CFG] received stroke: terminate 'con1' Mar 14 23:08:07 fw-01 charon: 08[IKE] queueing IKE_DELETE task Mar 14 23:08:07 fw-01 charon: 08[IKE] <con1|24> queueing IKE_DELETE task Mar 14 23:08:07 fw-01 charon: 08[IKE] delaying task initiation, CREATE_CHILD_SA exchange in progress Mar 14 23:08:07 fw-01 charon: 08[IKE] <con1|24> delaying task initiation, CREATE_CHILD_SA exchange in progress Mar 14 23:08:19 fw-01 charon: 12[CFG] received stroke: terminate 'con1' Mar 14 23:08:19 fw-01 charon: 12[CFG] received stroke: terminate 'con1' Mar 14 23:08:19 fw-01 charon: 07[IKE] queueing IKE_DELETE task Mar 14 23:08:19 fw-01 charon: 07[IKE] <con1|24> queueing IKE_DELETE task Mar 14 23:08:19 fw-01 charon: 07[IKE] delaying task initiation, CREATE_CHILD_SA exchange in progress Mar 14 23:08:19 fw-01 charon: 07[IKE] <con1|24> delaying task initiation, CREATE_CHILD_SA exchange in progress Mar 14 23:08:27 fw-01 charon: 09[CFG] received stroke: terminate 'con1' Mar 14 23:08:27 fw-01 charon: 09[CFG] received stroke: terminate 'con1' Mar 14 23:08:27 fw-01 charon: 05[IKE] queueing IKE_DELETE task Mar 14 23:08:27 fw-01 charon: 05[IKE] <con1|24> queueing IKE_DELETE task Mar 14 23:08:27 fw-01 charon: 05[IKE] delaying task initiation, CREATE_CHILD_SA exchange in progress Mar 14 23:08:27 fw-01 charon: 05[IKE] <con1|24> delaying task initiation, CREATE_CHILD_SA exchange in progress Mar 14 23:08:29 fw-01 charon: 14[CFG] received stroke: terminate 'con1' Mar 14 23:08:29 fw-01 charon: 14[CFG] received stroke: terminate 'con1' Mar 14 23:08:29 fw-01 charon: 08[IKE] queueing IKE_DELETE task Mar 14 23:08:29 fw-01 charon: 08[IKE] <con1|24> queueing IKE_DELETE task Mar 14 23:08:29 fw-01 charon: 08[IKE] delaying task initiation, CREATE_CHILD_SA exchange in progress Mar 14 23:08:29 fw-01 charon: 08[IKE] <con1|24> delaying task initiation, CREATE_CHILD_SA exchange in progress Mar 14 23:09:02 fw-01 charon: 10[IKE] giving up after 5 retransmits Mar 14 23:09:02 fw-01 charon: 10[IKE] <con1|24> giving up after 5 retransmits Mar 14 23:09:02 fw-01 charon: 10[IKE] IKE_SA con1[24] state change: ESTABLISHED => DESTROYING Mar 14 23:09:02 fw-01 charon: 10[IKE] <con1|24> IKE_SA con1[24] state change: ESTABLISHED => DESTROYING Mar 14 23:09:02 fw-01 charon: 10[KNL] unable to delete SAD entry with SPI c97bd6b5: No such file or directory (2) Mar 14 23:09:02 fw-01 charon: 10[KNL] <con1|24> unable to delete SAD entry with SPI c97bd6b5: No such file or directory (2) Mar 14 23:09:02 fw-01 charon: 14[CFG] received stroke: terminate 'con1{73}' Mar 14 23:09:02 fw-01 charon: 14[CFG] received stroke: terminate 'con1{73}' Mar 14 23:09:02 fw-01 charon: 10[IKE] unable to terminate, CHILD_SA with ID 73 not found Mar 14 23:09:02 fw-01 charon: 10[IKE] unable to terminate, CHILD_SA with ID 73 not found Mar 14 23:09:02 fw-01 charon: 02[CFG] received stroke: terminate 'con1' Mar 14 23:09:02 fw-01 charon: 02[CFG] received stroke: terminate 'con1' Mar 14 23:09:02 fw-01 charon: 02[CFG] no IKE_SA named 'con1' found Mar 14 23:09:02 fw-01 charon: 02[CFG] no IKE_SA named 'con1' found Mar 14 23:09:12 fw-01 charon: 12[KNL] creating acquire job for policy INITIATOR_IP/32|/0 === TARGET_IP/32|/0 with reqid {1} Mar 14 23:09:12 fw-01 charon: 12[KNL] creating acquire job for policy INITIATOR_IP/32|/0 === TARGET_IP/32|/0 with reqid {1} Mar 14 23:09:12 fw-01 charon: 07[IKE] initiating IKE_SA con1[25] to TARGET_IP Mar 14 23:09:12 fw-01 charon: 07[IKE] <con1|25> initiating IKE_SA con1[25] to TARGET_IP Mar 14 23:09:12 fw-01 charon: 07[ENC] generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ] Mar 14 23:09:12 fw-01 charon: 07[ENC] <con1|25> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ] Mar 14 23:09:12 fw-01 charon: 07[NET] sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (272 bytes) Mar 14 23:09:12 fw-01 charon: 07[NET] <con1|25> sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (272 bytes) Mar 14 23:09:12 fw-01 charon: 12[NET] received packet: from TARGET_IP[500] to INITIATOR_IP[500] (272 bytes) Mar 14 23:09:12 fw-01 charon: 12[NET] <con1|25> received packet: from TARGET_IP[500] to INITIATOR_IP[500] (272 bytes) Mar 14 23:09:12 fw-01 charon: 12[ENC] parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(MULT_AUTH) ] Mar 14 23:09:12 fw-01 charon: 12[ENC] <con1|25> parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(MULT_AUTH) ] Mar 14 23:09:12 fw-01 charon: 12[IKE] authentication of 'INITIATOR_IP' (myself) with pre-shared key Mar 14 23:09:12 fw-01 charon: 12[IKE] <con1|25> authentication of 'INITIATOR_IP' (myself) with pre-shared key Mar 14 23:09:12 fw-01 charon: 12[IKE] establishing CHILD_SA con1{1} Mar 14 23:09:12 fw-01 charon: 12[IKE] <con1|25> establishing CHILD_SA con1{1} Mar 14 23:09:12 fw-01 charon: 12[ENC] generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) ] Mar 14 23:09:12 fw-01 charon: 12[ENC] <con1|25> generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) ] Mar 14 23:09:12 fw-01 charon: 12[NET] sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (332 bytes) Mar 14 23:09:12 fw-01 charon: 12[NET] <con1|25> sending packet: from INITIATOR_IP[500] to TARGET_IP[500] (332 bytes) Mar 14 23:09:12 fw-01 charon: 07[NET] received packet: from TARGET_IP[500] to INITIATOR_IP[500] (300 bytes) Mar 14 23:09:12 fw-01 charon: 07[NET] <con1|25> received packet: from TARGET_IP[500] to INITIATOR_IP[500] (300 bytes) Mar 14 23:09:12 fw-01 charon: 07[ENC] parsed IKE_AUTH response 1 [ IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(AUTH_LFT) ] Mar 14 23:09:12 fw-01 charon: 07[ENC] <con1|25> parsed IKE_AUTH response 1 [ IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(AUTH_LFT) ] Mar 14 23:09:12 fw-01 charon: 07[IKE] authentication of 'TARGET_IP' with pre-shared key successful Mar 14 23:09:12 fw-01 charon: 07[IKE] <con1|25> authentication of 'TARGET_IP' with pre-shared key successful Mar 14 23:09:12 fw-01 charon: 07[IKE] IKE_SA con1[25] established between INITIATOR_IP[INITIATOR_IP]...TARGET_IP[TARGET_IP] Mar 14 23:09:12 fw-01 charon: 07[IKE] <con1|25> IKE_SA con1[25] established between INITIATOR_IP[INITIATOR_IP]...TARGET_IP[TARGET_IP] Mar 14 23:09:12 fw-01 charon: 07[IKE] scheduling reauthentication in 27801s Mar 14 23:09:12 fw-01 charon: 07[IKE] <con1|25> scheduling reauthentication in 27801s Mar 14 23:09:12 fw-01 charon: 07[IKE] maximum IKE_SA lifetime 28341s Mar 14 23:09:12 fw-01 charon: 07[IKE] <con1|25> maximum IKE_SA lifetime 28341s Mar 14 23:09:12 fw-01 charon: 07[IKE] received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding Mar 14 23:09:12 fw-01 charon: 07[IKE] <con1|25> received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding Mar 14 23:09:12 fw-01 charon: 07[IKE] CHILD_SA con1{75} established with SPIs cfbacf5c_i ca0bee43_o and TS 10.20.0.0/16|/0 IPV6_SUBNET::/56|/0 === 10.200.0.0/16|/0 ::/0|/0 Mar 14 23:09:12 fw-01 charon: 07[IKE] <con1|25> CHILD_SA con1{75} established with SPIs cfbacf5c_i ca0bee43_o and TS 10.20.0.0/16|/0 IPV6_SUBNET::/56|/0 === 10.200.0.0/16|/0 ::/0|/0 Mar 14 23:09:12 fw-01 charon: 07[IKE] received AUTH_LIFETIME of 28210s, scheduling reauthentication in 27670s Mar 14 23:09:12 fw-01 charon: 07[IKE] <con1|25> received AUTH_LIFETIME of 28210s, scheduling reauthentication in 27670s _______________________________________________ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
