-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
We have two firewalls running OpenBSD 4.2 with PF and CARP. We have two
web servers that we want to load balance traffic between and have
clients stay connected to one server as long as that server is up.
Our /etc/pf.conf and /etc/hoststated.conf files are configured per the
examples in "The Book of PF" on pages 51-53. The problem is that a web
client is not sticking to one server or the other. Each time we refresh
the page in the browser, we connect to the other server. The servers
each have a web page whose contents identify which server it is.
We have been testing this from clients with IP addresses on the
10.1.16.0/24 subnet which is attached to the trunk0 interface on the
firewalls. The web servers are on the 10.1.48.0/24 subnet connected to
trunk2 on the firewalls.
Here are the contents of the config files:
/etc/pf.conf
- ------------
int_if="trunk0"
build_if="trunk2"
webserver="10.1.48.200"
webports = "{ http, https }"
table <webpool> persist { 10.1.48.100, 10.1.48.101 }
set timeout src.track 60
rdr-anchor "hoststated/*"
rdr on $int_if proto tcp from any to $webserver port $webports -> \
~ <webpool> round-robin sticky-address
pass in on $int_if proto tcp from any to <webpool> port $webports
pass out on $build_if proto tcp from any to <webpool> port $webports
/etc/hoststated.conf
- --------------------
interval 30
timeout 2000
site1_public = "10.1.48.200"
site1_web1a = "10.1.48.100"
site1_web1b = "10.1.48.101"
# Port 80
table webpool {
~ real port http
~ check http "/up.txt" code 200
~ host $site1_web1a
~ host $site1_web1b
}
service site1 {
~ virtual host $site1_public port http
~ tag HOSTSTATED
~ table webpool
}
/etc/rc.conf
- ------------
hoststated_flags="" # for normal use: ""
pf=YES # Packet filter / NAT
pf_rules=/etc/pf.conf # Packet filter rules file
pflogd_flags= # add more flags, ie. "-s 256"
Any obvious ideas about why our client is bouncing between the web servers?
Could this be an issue with using trunk interfaces rather than the
individual Ethernet devices? Do we need to refer to the CARP interfaces?
Here is the results of running "pfctl -s all" after hitting refresh in
the browser several times and hitting both web servers.
PF State Info
- -------------
TRANSLATION RULES:
rdr-anchor "hoststated/*" all
rdr on trunk0 inet proto tcp from any to 10.1.48.200 port = www ->
<webpool> round-robin stic
ky-address
rdr on trunk0 inet proto tcp from any to 10.1.48.200 port = https ->
<webpool> round-robin st
icky-address
FILTER RULES:
pass in on trunk0 proto tcp from any to <webpool> port = www flags S/SA
keep state
pass in on trunk0 proto tcp from any to <webpool> port = https flags
S/SA keep state
pass out on trunk2 proto tcp from any to <webpool> port = www flags S/SA
keep state
pass out on trunk2 proto tcp from any to <webpool> port = https flags
S/SA keep state
No queue in use
STATES:
all carp 10.1.16.252 -> 224.0.0.18 SINGLE:NO_TRAFFIC
all carp 10.1.32.2 -> 224.0.0.18 SINGLE:NO_TRAFFIC
all carp 10.1.48.2 -> 224.0.0.18 SINGLE:NO_TRAFFIC
all carp 224.0.0.18 <- 10.1.16.2 NO_TRAFFIC:SINGLE
all tcp 10.1.48.2:1091 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:46783 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:6985 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:43486 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:14244 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:8754 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.101:80 <- 10.1.48.200:80 <- 10.1.16.60:2216
FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.16.60:2216 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.100:80 <- 10.1.48.200:80 <- 10.1.16.60:2217
FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.16.60:2217 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.101:80 <- 10.1.48.200:80 <- 10.1.16.60:2218
FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.16.60:2218 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.100:80 <- 10.1.48.200:80 <- 10.1.16.60:2219
FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.16.60:2219 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.101:80 <- 10.1.48.200:80 <- 10.1.16.60:2220
FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.16.60:2220 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.100:80 <- 10.1.48.200:80 <- 10.1.16.60:2221
FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.16.60:2221 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:26880 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:11005 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:26283 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:13055 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:43776 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:30663 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:39595 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:3984 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:18393 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:48273 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:16513 -> 10.1.48.101:80 FIN_WAIT_2:FIN_WAIT_2
all tcp 10.1.48.2:32860 -> 10.1.48.100:80 FIN_WAIT_2:FIN_WAIT_2
INFO:
Status: Enabled for 1 days 22:03:01 Debug: Urgent
State Table Total Rate
~ current entries 34
~ searches 3579299 21.6/s
~ inserts 27497 0.2/s
~ removals 27463 0.2/s
Counters
~ match 2519631 15.2/s
~ bad-offset 0 0.0/s
~ fragment 0 0.0/s
~ short 0 0.0/s
~ normalize 0 0.0/s
~ memory 0 0.0/s
~ bad-timestamp 0 0.0/s
~ congestion 0 0.0/s
~ ip-option 0 0.0/s
~ proto-cksum 0 0.0/s
~ state-mismatch 2 0.0/s
~ state-insert 0 0.0/s
~ state-limit 0 0.0/s
~ src-limit 0 0.0/s
~ synproxy 0 0.0/s
TIMEOUTS:
tcp.first 120s
tcp.opening 30s
tcp.established 86400s
tcp.closing 900s
tcp.finwait 45s
tcp.closed 90s
tcp.tsdiff 30s
udp.first 60s
udp.single 30s
udp.multiple 60s
icmp.first 20s
icmp.error 10s
other.first 60s
other.single 30s
other.multiple 60s
frag 30s
interval 10s
adaptive.start 6000 states
adaptive.end 12000 states
src.track 60s
LIMITS:
states hard limit 10000
src-nodes hard limit 10000
frags hard limit 5000
tables hard limit 1000
table-entries hard limit 100000
TABLES:
webpool
OS FINGERPRINTS:
696 fingerprints loaded
=====
Thanks
- --
David Goldsmith,
SANS NOC
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHmilZ417vU8/9QfkRAtyeAJ49P6AmhbJhGMtQ8vbBHy+rj1zhUQCfSayh
2fsJx0IHcNWuP0aSF0dM6do=
=IcyZ
-----END PGP SIGNATURE-----
