On Fri, Feb 22, 2013 at 2:29 PM, sangdrax8 <[email protected]> wrote:
> I am new to OpenBSD, but would like to take advantage of a redundant > setup with ipsec/carp/sasyncd. I have run into a situation which seems > to be a bug, and was directed to post to tech with config files. > > I believe my problem is that the phase 1 of an ipsec negotiation is not > being synced with sasyncd, causing a repeatable condition where tunnels > die for extended periods of time. I have tried the following with all > three machines running 5.1-stable, 5.2-stable, and 5.2-stable with a > snapshot kernel from 2/17/2013. My main problem exists across all three > setup types. I am running 5.2 with the snapshot kernel now as it > provides the lifetime setting in ipsec phase 2 to make the testing > faster. > > > ####### Setup Description ###### > > 172.16.10.0/24 behind the carp devices on vlan 2 > 172.16.20.0/24 the other side of the tunnel no vlan > 1.1.1.0/24 is used for the internet > > vlan 3 is tagged on Fw's, untagged to the lab1 box connected with a > switch between them > > fw boxes use trunk ports as follows > em0 + em1 = trunk0 > em2 + em3 = trunk1 > > > ####### Setup Drawing ###### > > > 172.16.10.0/24 > ................ > . . Vlan 2 > . .3 . .7 > *****.**** ****.***** > * fw1 * * fw2 * > * * * * > *****.**** ****.***** > . 1.1.1.2 . 1.1.1.3 > . . > ................ Vlan 3 to switch > . 1.1.1.1 > . > . > . > . > . > . 1.1.1.5 > ******.****** > * Lab1 * > * * > ******.****** > . > 172.16.20.0/24 > > > > ###### How to re-create the problem ##### > > Bring all machines up, and allow ipsec to come up (ensuring the fw1 is > the master) > > start ping from 172.16.10.0/24 net to 172.16.20.0/24 net > > tcpdump on vlan3 on both fw1 and fw2 (only fw1 should show active esp > traffic). Note the spi's seen. this is spi set 1 > > carp demote fw1 'ifconfg -g carp carpdemote 128'. > > tcpdump on fw2 should now show the esp (same spi's as before, spi set > 1), and a large increase in sequence numbers > > soon after transfer fw2 will do a full phase1 and phase2 re-negotiation > (can be seen on the tcpdump). Spi's will change (referring to this as > spi set 2), sequence numbers will reset, and no pings are lost. This is > where I believe the phase 1 is now renegotiated between fw2 and lab1 > because it was not synced from fw1. > > recover fw1 as carp master 'ifconf -g carp -carpdemote 128'. > > tcpdump on fw1 should now show the esp packets (spi's now from set 2), > and a large increase in sequence numbers > > sometimes soon after transfer fw1 will attempt a phase 2 re-key and be > denied. even if it doesn't do it quickly, when the phase 2 begins to > time out it will attempt to re-key and be denied at that time. I have > reduced phase 2 to 5 minutes in my tests to allow this to happen more > quickly. > > when phase 2 times out, the pings through the tunnel fail and the tunnel > is down. > > You can fail back to fw2, and a new phase 2 negotiation will take place > to resume traffic, otherwise fw1 will not be able to re-build the tunnel > until the phase 1 times out (I believe 8 hours default) > > As a note, if you fail a firewall by actually rebooting it, this problem > goes undetected as this clears the SA's. > > I know this is a long e-mail, but I have tried to provide all the > details and configurations that could be needed to re-create this. I > have been able to consistently re-create this issue every time across > multiple versions. If there is anything I have left off, please let me > know. > > > ####################################################### > ############## Configuration Files Below ############## > ####################################################### > > > ##### fw1 configs ##### > > ==> sasyncd.conf <== > interface carp3 > group carp > peer 172.16.10.7 > sharedkey > 0xf04c0d7fada85a2c0f3fec1db4e52e6d6cbd360936b163133df4917566308bd3 > > > ==> hostname.carp2 <== > up > inet 172.16.10.1 255.255.255.0 172.16.10.255 vhid 2 pass password > carpdev vlan2 > > ==> hostname.carp3 <== > up > inet 1.1.1.1 255.255.255.0 1.1.1.255 vhid 3 pass password carpdev vlan3 > > ==> hostname.em0 <== > up > > ==> hostname.em1 <== > up > > ==> hostname.em2 <== > up > > ==> hostname.em3 <== > up > > ==> hostname.enc0 <== > up > > ==> hostname.gif1 <== > create > tunnel 172.16.10.1 172.16.20.1 > 10.10.10.1 10.10.20.1 netmask 255.255.255.252 > mtu 1426 > up > !route add 172.16.20.0/24 10.10.20.1 > > ==> hostname.pfsync0 <== > up syncdev vlan2 syncpeer 172.16.10.7 > > ==> hostname.trunk0 <== > up > trunkproto failover trunkport em0 trunkport em1 > > ==> hostname.trunk1 <== > up > trunkproto failover trunkport em2 trunkport em3 > > ==> hostname.vlan2 <== > up > inet 172.16.10.3 255.255.255.0 NONE vlan 2 vlandev trunk0 > > ==> hostname.vlan3 <== > up > inet 1.1.1.2 255.255.255.0 NONE vlan 3 vlandev trunk1 > > ==> ipsec.conf <== > fw_gw = "1.1.1.1" > fw_gif = "172.16.10.1" > fw_net = "172.16.10.0/24" > > lab_gw = "1.1.1.5" > lab_gif = "172.16.20.1" > lab_net = "172.16.20.0/24" > > ike esp from $fw_gif to $lab_gif \ > local $fw_gw peer $lab_gw \ > main auth hmac-sha1 enc aes-256 group modp1024 \ > quick auth hmac-sha1 enc aes-256 group modp1024 lifetime 5m \ > psk "password" > > > > > ##### fw2 configs ##### > > ==> sasyncd.conf <== > interface carp3 > group carp > peer 172.16.10.3 > sharedkey > 0xf04c0d7fada85a2c0f3fec1db4e52e6d6cbd360936b163133df4917566308bd3 > > ==> hostname.carp2 <== > up > inet 172.16.10.1 255.255.255.0 172.16.10.255 vhid 2 pass password > carpdev vlan2 advskew 128 > > ==> hostname.carp3 <== > up > inet 1.1.1.1 255.255.255.0 1.1.1.255 vhid 3 pass password carpdev vlan3 > advskew 128 > > ==> hostname.em0 <== > up > > ==> hostname.em1 <== > up > > ==> hostname.em2 <== > up > > ==> hostname.em3 <== > up > > ==> hostname.enc0 <== > up > > ==> hostname.gif1 <== > create > tunnel 172.16.10.1 172.16.20.1 > 10.10.10.1 10.10.20.1 netmask 255.255.255.252 > mtu 1426 > up > !route add 172.16.20.0/24 10.10.20.1 > > ==> hostname.pfsync0 <== > up syncdev vlan2 syncpeer 172.16.10.3 > > ==> hostname.trunk0 <== > up > trunkproto failover trunkport em0 trunkport em1 > > ==> hostname.trunk1 <== > up > trunkproto failover trunkport em2 trunkport em3 > > ==> hostname.vlan2 <== > up > inet 172.16.10.7 255.255.255.0 NONE vlan 2 vlandev trunk0 > > ==> hostname.vlan3 <== > up > inet 1.1.1.3 255.255.255.0 NONE vlan 3 vlandev trunk1 > > ==> ipsec.conf <== > fw_gw = "1.1.1.1" > fw_gif = "172.16.10.1" > fw_net = "172.16.10.0/24" > > lab_gw = "1.1.1.5" > lab_gif = "172.16.20.1" > lab_net = "172.16.20.0/24" > > ike esp from $fw_gif to $lab_gif \ > local $fw_gw peer $lab_gw \ > main auth hmac-sha1 enc aes-256 group modp1024 \ > quick auth hmac-sha1 enc aes-256 group modp1024 lifetime 5m \ > psk "password" > > ###### lab1 ###### > > ==> hostname.em0 <== > up > inet 1.1.1.5 255.255.255.0 > > ==> hostname.em2 <== > up > inet 172.16.20.1 255.255.255.0 > > ==> hostname.enc0 <== > up > > ==> hostname.gif0 <== > create > tunnel 172.16.20.1 172.16.10.1 > 10.10.20.1 10.10.10.1 netmask 255.255.255.252 > mtu 1426 > up > !route add 172.16.10.0/24 10.10.10.1 > > So fw1 is not ready if you manually turn it off but come back if you reboot it ! i guess a MASTER that fail need maintenance ;-) MAybe it is a missbehavior, but does it actually happen in real use scenarii ? -- --------------------------------------------------------------------------------------------------------------------- () ascii ribbon campaign - against html e-mail /\
