I just installed pFsense over Endian Firewall after becoming very unhappy
with the reliability of the VPN. I previously used m0n0wall where in after
thought it was quite reliable. After having a few minutes of trouble setting
up the VPN to my PIX I was able to make a connection. But, it does not last.
By this I mean the IPSEC stays, but no traffic goes through. Then in about a
minute or so I can ping a device on the other side.
What concerns me is the error message "racoon: WARNING:
setsockopt(UDP_ENCAP_ESPINUDP_NON_IKE): Invalid argument" for which finding
any information on has been difficult.
This is the pfsense config:
interface:wan
local-subnet
address: 192.168.2.0/24
remote-subnet: 192.168.1.0/24
remote-gateway: 6x.xxx.xxx.xx<
mode: main
myident
address: 7x.xx.xxx.xxx
encryption-algorithm: 3des
hash-algorithm: md5
dhgroup: 2
lifetime: 3600
pre-shared-key: XXXXXXXXXXXXXXXXX
authentication_method: pre_shared_key
protocol: esp
encryption-algorithm-option: des
encryption-algorithm-option: 3des
encryption-algorithm-option: blowfish
encryption-algorithm-option: rijndael
encryption-algorithm-option: rijndael 256
hash-algorithm-option: hmac_md5
pfsgroup: 2
lifetime: 3600
pinghost>192.168.1.4
Here is the log output:
Apr
25 04:44:50
racoon: INFO: 7x.xx.xxx.xxx[500] used as isakmp port
(fd=13)
Apr 25 04:44:50
racoon: WARNING: setsockopt(UDP_ENCAP_ESPINUDP_NON_IKE):
Invalid argument
Apr 25 04:44:50
racoon: INFO: fe80::201:2ff:fe65:5554%xl1[500] used as
isakmp port (fd=14)
Apr 25 04:44:50
racoon: INFO: fe80::250:4ff:fed8:c909%xl0[500] used as
isakmp port (fd=15)
Apr 25 04:44:50
racoon: INFO: 192.168.2.1[500] used as isakmp port
(fd=16)
Apr 25 04:44:50
racoon: WARNING: setsockopt(UDP_ENCAP_ESPINUDP_NON_IKE):
Invalid argument
Apr 25 04:44:50
racoon: INFO: initiate new phase 2 negotiation:
7x.xx.xxx.xxx[500]<=>6x.xxx.xxx.xx[500]
Apr 25 04:44:51
racoon: WARNING: ignore RESPONDER-LIFETIME
notification.
Apr 25 04:44:51
racoon: WARNING: transform number has been modified.
Apr 25 04:44:51
racoon: WARNING: attribute has been modified.
Apr 25 04:44:51
racoon: WARNING: trns_id mismatched: my:DES peer:AES
Apr 25 04:44:51
racoon: WARNING: trns_id mismatched: my:3DES peer:AES
Apr 25 04:44:51
racoon: WARNING: trns_id mismatched: my:BLOWFISH
peer:AES
Apr 25 04:44:51
racoon: WARNING: less key length proposed, mine:128
peer:256. Use initiaotr's one.
Apr 25 04:44:51
racoon: INFO: IPsec-SA established: ESP/Tunnel
6x.xxx.xxx.xx[0]->7x.xx.xxx.xxx[0] spi=52797788(0x325a15c)
Apr 25 04:44:51
racoon: INFO: IPsec-SA established: ESP/Tunnel
7x.xx.xxx.xxx[0]->6x.xxx.xxx.xx[0] spi=2863459557(0xaaacece5)Here is the PIX
config:
sysopt connection permit-ipsec
sysopt connection permit-pptp
crypto ipsec transform-set ESP-3DES-MD5 esp-3des esp-md5-hmac
crypto ipsec transform-set ESP-DES-MD5 esp-des esp-md5-hmac
crypto ipsec transform-set ESP-DES-SHA esp-des esp-sha-hmac
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto ipsec transform-set ESP-AES-128-SHA esp-aes esp-sha-hmac
crypto ipsec transform-set ESP-AES-128-MD5 esp-aes esp-md5-hmac
crypto ipsec transform-set ESP-AES-192-SHA esp-aes-192 esp-sha-hmac
crypto ipsec transform-set ESP-AES-192-MD5 esp-aes-192 esp-md5-hmac
crypto ipsec transform-set ESP-AES-256-SHA esp-aes-256 esp-sha-hmac
crypto ipsec transform-set ESP-AES-256-MD5 esp-aes-256 esp-md5-hmac
crypto ipsec transform-set Cisco esp-des esp-md5-hmac
crypto ipsec transform-set SNet esp-aes-256 esp-md5-hmac
crypto ipsec transform-set pFsense ah-md5-hmac esp-3des esp-md5-hmac
crypto dynamic-map cisco 1 set peer 7x.xxx.xx.xxx
crypto dynamic-map cisco 1 set transform-set Cisco ESP-DES-SHA ESP-DES-MD5
ESP-3 DES-SHA ESP-3DES-MD5
crypto dynamic-map outside_dyn_map 1 set pfs group2
crypto dynamic-map outside_dyn_map 1 set peer 7x.xx.xxx.xxx
crypto dynamic-map outside_dyn_map 1 set transform-set SNet
crypto dynamic-map outside_dyn_map 1 set security-association lifetime
seconds 3600 kilobytes 4608000
crypto map dyn-map 65535 ipsec-isakmp dynamic outside_dyn_map
crypto map dyn-map interface outside
isakmp enable outside
isakmp enable inside
isakmp key ******** address 7x.xx.xxx.xxx netmask 255.255.255.255 no-xauth
no-config-mode
isakmp identity address
isakmp keepalive 60
isakmp policy 10 authentication pre-share
isakmp policy 10 encryption 3des
isakmp policy 10 hash md5
isakmp policy 10 group 2
isakmp policy 10 lifetime 3600
