On Sat, Jul 12, 2008 at 04:12:14PM -0500, Karl O. Pinc wrote: > I'm assuming that the pfsync mechanism (with syncdev) does > more than expose changes, that it actually merges the > existing state tables of the two machines.
When a pfsync'd firewall boots, pfsync requests a bulk update of all states from the other firewall(s). Until this update is complete, it increases the carp demotion counter, preventing carp from taking over the virtual IP address. When the bulk update completes or times out, the demotion counter is decreased again. (The demotion counter is also twiddled in /etc/rc, and daemons such as bgpd and ospfd for similar reasons) > considered the alternatives until just now. In my case > I'd want the master's state table copied to the standby, > but how would it know which is the master? It seems > safest to just merge the state of the two boxes > so that's what I assume pf does.) In the case of the system booting, we know that WE are not the master, but in general you are correct. The firewalls all synchronize their state tables whether they are master or not. > Aside from dumping state tables on the master and standby > boxes and comparing them, is there a way to ask > if the state tables are synchronized? Not really, because pfsync makes no attempt to verify that the state tables are synchronized itself; for performance reasons the protocol makes no attempts at reliability, but rather tends towards complete synchronization. (For example, if the backup firewall somehow misses a state insertion message and sees an update for a state it doesn't know about, it will make an explicit request for that state. > Knowing would help prevent shutting down the master when the standby > is not yet synchronized. Don't shut your "master" down until all it's carp interfaces are in the MASTER state. > What happened next was also a bit perturbing. While > running the firewall on the standby box one of the > "disconnected" applications was logged back in. > Booting the master _again_ disconnected the > application and returned it to the login screen. This should probably not be happening. Have you verified that pfsync messages are being recieved correctly? Try looking at 'netstat -sp pfsync', and try comparing the output of pfctl -ss; you can also tcpdump on the pfsync interface or the syncdev interface with -s 1518 to see the states being synchronized. > (Which brings me to a slightly off-topic question about the demotion > counter. I can't find where to RTFM. Just what's the relationship > between the demotion counter value and advskew when it comes to > deciding the state of an interface? You're right, this should probably be better documented in carp(4). The demotion counter applies to a whole GROUP of interfaces, while the advskew applies to each interface individually, and it does in fact override the advskew. > I see this in the 4.2->4.3 changelogs: > Changed rc(8) and netstart(8) so pfsync(4) is not brought up before the > working ruleset has been loaded I don't believe this is critical, but it means that if your rulesets are identical across firewalls the synchronized states are more likely to get associated with the correct rule.
