I have a pair of OpenBSD firewall/routers in a reasonably vanilla pf + pfsync + CARP configuration, each straddling two routed networks. The CARP interface on the internal network is the default gateway for that subnet. The CARP interface on the external network is the default destination for traffic aimed at the internal network.
It all works splendidly, with one exception. In order for our firewall to operate effectively, we use 'keep state' pf rules. We empirically determined that we must have CARP preemption enabled, otherwise pf cannot properly establish state for new TCP connections. If pfsync could be told to synchronize incomplete states, this issue might go away. Example: firewall1 is the master on the carp1 interface, and firewall2 is the master on the carp2 interface. Inbound traffic to an internal host arrives via the carp1 interface, and return traffic arrives via the carp2 interface. pf will not establish state for this new connection since the inbound and return traffic are not handled by the same firewall host. We thus use CARP preemption to force one of the firewalls to always come up as the master for both CARP interfaces. This is not so unresaonable, though it might be nice if the documentation presented this use-case (or similar) as a rationale for needing CARP preemption. Where this presents a problem is if the current CARP master loses a single network interface (cable unplugged, isolated hardware failure, sysadmin failure, etc.), as opposed to the CARP master failing entirely. The slave will appropriately assume the master role for one CARP interface, but will *not* do so for the second. Beyond the basic asynchronous routing + state creation issue described above, this specific failure mode would still result in a complete inability to pass traffic in a single direction, since packets would come into the half-dead master via its good interface, but could not be forwarded to the network associated with the failed interface. We would like our otherwise nicely redundant firewall configuration to be resilient against this type of failure. Short of running a cron job every sixty seconds to check the interface state, is there some way we can automatically force the promotion of a CARP slave if a second CARP interface flips from slave to master? Suggestions are most appreciated. I apologize if the CARPish-flavor of this question is not entirely appropriate for the pf mailing list; if there is another list that would be more suited for this question, please let me know. --Kyle
