(all the following NICs are in a single bridge)
NIC_A: IP 123.123.0.1, connected to the big bad internet
NIC_B: IP 192.168.0.1, internal network (desktops etc)
NIC_C: IP 10.0.0.1, internal servers (development and staging area)
NIC_D: NO IP, DMZ 1 (a collection of operational www and mail servers)
NIC_E: NO IP, DMZ 2 (a collection of operational DB and backend servers)
With this setup, why are you bridging and not routing?
took out NIC_B and NIC_C from the bridge. this was a stupid mistake to put them in, these interfaces should serve as gateways to subnets while the others should only bridge separate segments.
using NAT on NIC_A i map all the outbound connections from the internal network (192.168.0.0/24) to the IP of the bridge/firewall (123.123.0.1).
using BINAT on NIC_A i map further IP aliases (eg. 123.123.0.2) to internal development or staging servers (eg. 10.0.0.2).
then there are a bunch of PF rules (on each interface) to controlthe access of each of these segments to each other.
2. if all the NATing happens on NIC_A, why do i get such entries in my state table when an internal desktop tries to reach a server in DMZ 1:
192.168.0.13 -> 123.123.0.1 -> 123.123.0.13
(ie. the private address is translated to the external bridge IP!)
What is the default gateway for the internal desktop? Where does this gateway reside?
the default gateway for the internal desktop is 192.168.0.1 which is NIC_B. the netmasks are correct. even after taking NIC_B and NIC_C out of the bridge, i still get these entries. i don't even understand how such a sequence of packet traversal is possible in the first place.
Some of this is guesswork, as I can't completely follow the code yet, but here we go.
Keep in mind the bridge isn't a single entity, and there are two distinct layers at work here. The IP layer has no knowledge of the bridge; the bridge itself is just a bunch of ties between interfaces at the ethernet level.
The initial packet from the internal desktop hits the gateway in IP routing mode. The only available IP interface for forwarding in this case is NIC_A, so the packet gets sent out (IP perspective) from there. Just underneath IP, the bridge realizes the real destination is on one of the other interfaces, so it transits the packet there before physically sending it out.
3. my understanding is that a packet from an internal desktop (ie. 192.168.0.13) to an internal server (ie. 10.0.0.13) would PASS IN ON NIC_B and then PASS OUT ON NIC_C but it doesn't seem to behave that way. did i get something wrong?
You could verify this by adding "log-all" to all of your pass rules, and using tcpdump (with -e) on pflog0. It will show you where pf is seeing the packets.
ok, i'm passing in and out all on NIC_B and NIC_C. i see the packet come in on NIC_B and disappear there. is being translated to the IP of NIC_A (123.123.0.1)? how can i verify which NAT rules are being applied?
I can't think of a way to check NAT rules, but I'll bet the packet will show up as matching a pass rule on NIC_A.
4. equally, a server on DMZ 1 trying to reach a service on DMZ 2 should PASS IN ON NIC_D and PASS OUT ON NIC_E but the packets seem to be going through NIC_A as well. does this make any sense or do i have a terribly bad setup?
IP routing can't be ignored. I would guess it is what's causing the behavior you're seeing.
sorry if i'm just not getting this but why does anything need to get routed here? isn't the bridge supposed to know which MACs are on which interfaces and just forward the packet without any modfications to it?
The bridge knows MACs, yes, but the internal desktop doesn't. At the IP layer, the DMZ machines are on a different subnet, and the desktop needs a gateway to get there. When the packet from the desktop hits the OpenBSD machine, the dest MAC is for the gateway, not the DMZ machine. The bridge can't act until the IP layer does its forwarding thing to get a new MAC.
pf itself is in the IP layer, with a hook for packets directly forwarded by the bridge from input to output. The internal transits between layers don't hit pf. tcpdump will show which physical interfaces the packets arrive and leave on, because it taps them from the drivers.
Does that make sense?
