Short question: does pfsync currently support fluent failover of a pf
established 'route-to' state, when a CARP failover happens?
Reason for the question: CARP, pfsync, and route-to all seem to work
nicely in our OpenBSD load balancer (LB) setup, except: fluent failover
of established TCP connections doesn't work for us.
When an external client establishes a TCP connection, via our primary
LB, to one of our servers, and then we induce a CARP failover to our
secondary LB, and then the external client sends the next packet on the
established TCP connection, the new LB doesn't foward (route) that
packet to our server like the original LB would do; instead, the new LB
sends a TCP RST back to the client.
====
Sketch of our setup:
# Load balancer A:
ifconfig pcn0 192.168.1.50 netmask 255.255.255.0
ifconfig carp1 vhid 1 carpdev pcn0 inet 192.168.1.100 netmask 255.255.255.0
ifconfig pfsync0 syncdev pcn1
relayd.conf:
table <servers> { 192.168.1.231 192.168.1.232 }
redirect "server" {
listen on 192.168.1.100 port 1234 interface pcn0
route to <servers> mode roundrobin
}
# Load balancer B:
ifconfig pcn0 192.168.1.51 netmask 255.255.255.0
Rest identical to Firewall B, except higher advskew
# Server C:
ifconfig pcn0 192.168.1.231 netmask 255.255.255.0
ifconfig lo0 alias 192.168.1.100
In our test, the load balancers, servers. and test client are all on the
same 192.168.1.0/24 network on the same Ethernet segment.
====
Note: "Why not 'rdr-to' or 'nat-to'?" Because our servers need to see
the real destination IP to know which SSL certificate to present to the
client, and because our servers need to see the real source IP to check
it against the client's SSL Subject CN.
Note: "Isn't 'sloppy' keep state on the 'route-to' rule of the load
balancers a security risk?" We think that risk is tolerable, because the
(also OpenBSD) servers already have strict pf firewalls. The load
balancers are not used as firewalls, they are not expected to add security.
Note: "Why are the servers on the same Internet-facing Ethernet segment
as the load balancers?" Part of the reason is to allow direct access to
individual servers even when the load balancers are not active.
====
Sketch of our test:
# Disable all servers expect Server C
# On server C:
nc -l 192.168.1.100 1234
# On test client:
ifconfig pcn0 192.168.1.10
nc -v 192.168.1.100 1234 # Result: connected, can exchange messages
# On BACKUP load balancer:
pfctl -s state | egrep 1234 # Result: shows the ESTABLISHED connection
# On MASTER load balancer:
ifconfig -g carp carpdemote 128 # Result: CARP failover
# On test client:
type next line in 'nc'. # Result: disconnect.
# 'tcpdump' shows TCP RST from load balancer to test client.
+++chefren
