Hallo!
I use ip-based load balancing with carp on two-sided firewall, no nat,
just routing and it works like this
internet ---> router --> 172.16.5.118:firewall:192.168.222.189 --->
web server
I tried this setup with two and more firewalls, where 5.118 is ip
address assigned to outer carp0 interface and router uses it for routing
subnet 192.168.222/24 behind the firewall. Web server uses .222.189 as
their default gateway and it is inner carp1 interface's address.
carp interfaces are configured like this in different firewalls (for
example this is for one side, two-firewall setup)
inet 172.16.5.118 255.255.255.0 195.222.5.0 carpnodes 1:100,2:0
balancing ip-stealth pass xxx carpdev em0
inet 172.16.5.118 255.255.255.0 195.222.5.0 carpnodes 1:0,2:100
balancing ip-stealth pass xxx carpdev em0
To my mind everything works like a charm, i see packages on every
interface and only one of them processes them. They get out at the other
side and 'magically' routing is symmetrical, i.e. each firewall accepts
from webserver answers for the packets it sent out. PF is enabled.
And here follows the problem part.
With this described setup i have one doubt which seems to be remedied
easy in my case since i can ask person responsible for the router to do
load balancing on incoming traffic. Namely, i guess in case traffic
flood comes in, every firewall gets incoming packets, although every
firewall but one ignores each of them. Still i suspect that under heavy
network load it leaves its mark on firewalls' performance.
In the router it is possible to set up so to say src and dst address
based hashed load-balancing (it is Juniper device) saying like this
(5.116, 5.117 being firewalls' real outer ip addresses)
route 192.168.222.0/24 next-hop [ 172.16.5.116 172.16.5.117 ]
Now outer side of the firewalls doesnt have carp device configured any
more and each physical firewall gets only its share of incoming packets,
just as router decides to send them.
And inner side of the firewall still has carp1 device configured as
webserver's default gateway.
The problem is that with this setup arises asymmetrical routing which i
followed with tcpdump. If firewalls work with pf disabled (essentially
as routers), traffic gets thru; but with it pf enabled it doesnt (in
fact some wget's get answered but thats not obviously not enough). I
believe it is possible to set up such a kind of pf.conf that works with
asymmetrical traffic but the whole setup is then less balanced.
I am sorry for the long description but i wanted to be clear of what i
succeeded to configure and where i sumbled onto the obsticle. Is there
a way to say to kernel that it does some kind of so to say stickyness to
returning packets in a way packets get back to the very firewall they
left? I.e. the the whole picture would be similar to the working case
described in the beginning except Juniper does the load balancing and
OpenBSD working with one carp interface on the inner side offering
webserver a default gateway.
Best regards,
Imre
PS As a somehow useable workaround would be not to use carp devices at
all and to have packetes going to webserver nat'ed, but it has a
drawback that webserver doesnt see any more their 'real' src addresses.
PPS OpenBSD is needed between router and webserver to filter and analyze
traffic.
