Jason Stubbs wrote:
From what I understand of the theory, it should work but I was hoping
to get a "yes, I'm doing it" from somebody. Unless there's a reason it
won't work, I'll be having a go and getting it set up in the first week
of March and will write back with the results.
Ok, I had troubles and then looked at the supported solutions, but am
still having the same problems. For reference, I now have a testing
setup that is the same as the "something bigger" on the following page:
http://www.countersiege.com/doc/pfsync-carp/#big
However, pfsync is syncing too slow causing problems with state
handling. With one client, one server and the two arp-balanced firewalls
in between, essentially what's happening is:
* Client sends SYN packet to server public ip via firewall 1
* Firewall 1 switches the destination to the server's private ip and
forwards the packet
* Server receives the packet and sends a SYN/ACK packet back to the
client via firewall 2
* Firewall 2 forwards the packet as is to the client
* Client sends a RST to the server's private IP (which gets forwarded
elsewhere due to the private ip being unroutable)
* Firewall 2 receives the state update from firewall 1
* Client sends another SYN packet to server public ip via firewall 1
* Firewall 1 NATs it, sends it to the server which replies with a
SYN/ACK going via firewall 2 as before
* Firewall 2 now has the state and so un-NAT's it and sends it back to
the client.
* Client ACK's the SYN/ACK and the connection is set up
I don't fully understand the reasons, but even though the connection is
set up, the state on each firewall is now out of sync. On firewall 1 the
state is CLOSED:SYN_SENT and on firewall 2 the state is
SYN_SENT:ESTABLISHED.
If I turn off arp-balance, only the one firewall is used and the states
are correctly synced on the other firewall. Connections are maintained
when rebooting either firewall and fully synced again after booting.
Configuration on firewall 1 is as follows:
/etc/hostname.carp0
inet 192.168.1.201 255.255.255.0 192.168.1.255 vhid 1 pass carp0dev
carpdev fxp0 advskew 0
/etc/hostname.carp1
inet 192.168.1.201 255.255.255.0 192.168.1.255 vhid 2 pass carp0dev
carpdev fxp0 advskew 50
/etc/hostname.carp2
inet 10.0.0.1 255.255.0.0 10.0.255.255 vhid 3 pass carp2dev carpdev em1
advskew 0
/etc/hostname.carp3
inet 10.0.0.1 255.255.0.0 10.0.255.255 vhid 4 pass carp2dev carpdev em1
advskew 50
/etc/hostname.em0
inet 10.255.255.2 255.255.255.0 NONE
/etc/hostname.em1
inet 10.0.255.1 255.255.0.0 NONE
/etc/hostname.fxp0
inet 192.168.1.203 255.255.255.0 NONE
/etc/hostname.pfsync0
up syncdev em0
/etc/pf.conf
ext_if="fxp0"
syn_if="em0"
int_if="em1"
srv_ip="192.168.1.201"
table <fw> { 192.168.1.201 192.168.1.203 192.168.1.204 10.0.0.1 }
rdr on $ext_if proto tcp from any to 192.168.2.1 port ssh -> 10.0.1.1
pass quick on { $int_if $ext_if } proto carp
pass quick on { $syn_if } proto pfsync
pass in on $ext_if from !<fw> to 10.0.1.1 keep state
/etc/sysctl.conf
net.inet.ip.forwarding=1
net.inet.carp.preempt=1
net.inet.carp.arpbalance=1
Configuration on firewall 2 is almost identical. The 0 and 50 are
toggled on the carp devices. em0, em1 and fxp0 have their IPs
incremented by 1. The firewall is wide open for testing. Closing
everything but that which is absolutely necessary produces the same
out-of-sync issue.
While I would ordinarily assume that the above is expected behaviour,
the countersiege example indicates that it should be doable. Is there
some step that I'm missing?
--
Jason Stubbs
