Well, I have more information for you regarding topology, and what
appears to fix it.
router->bridge->network (4 consecutive class C's)
The router doesn't do a 255.255.252.0 netmask, but rather aliases out
x.x.12.1, x.x.13.1, etc. The reason for this is that some client
routers are so crappy that they cannot handle a non-255.255.255.0
netmask and this fixed the problem (ugh). So a machine on x.x.12.x was
having trouble reaching a machine on x.x.13.x.
Traffic flow:
x.x.12.x -> bridge -> router -> bridge -> x.x.13.x
When state was kept on this flow, it would exhibit the weird 96k
stoppage mentioned earlier, regardless of protocol (http, ssh, ftp,
etc)
Now, I implemented queueing and assigned localnet to localnet traffic
into an 8.5M queue without keeping state and shoved everything else
into default 1.5M queue and kept state. This solved the problem
handily, so the problem is apparently keeping state on a network with
that exact setup when traffic bounces off an external router.
Relevant bits of pf.conf below:
.
# This contains localnet traffic that does not go out a T (shouldn't be
limited to 1.5 or state tracked)
table <localnet> persist { x.x.12.0/24 x.x.13.0/24 x.x.14.0/24
x.x.15.0/24 }
.
# External interface
altq on $ext_if cbq bandwidth 10Mb queue { ext int }
queue ext on $ext_if bandwidth 1.5Mb { hogs customer }
queue hogs on $ext_if bandwidth 100Kb
queue customer on $ext_if bandwidth 1.4Mb cbq(borrow default)
queue int on $ext_if bandwidth 8.5Mb cbq(borrow)
# Internal interface
altq on $int_if cbq bandwidth 10Mb queue { ext int }
queue ext on $int_if bandwidth 1.5Mb { hogs customer }
queue hogs on $int_if bandwidth 100Kb
queue customer on $int_if bandwidth 1.4Mb cbq(borrow default)
queue int on $int_if bandwidth 8.5Mb cbq(borrow)
# Pass quick all of our local traffic in the int queue
pass quick on { $ext_if $int_if } from <localnet> to <localnet> queue
int
# Modulate state on all connections
pass on $ext_if modulate state
<end>
If state is kept on the localnet line, then it breaks. (luckily, I
don't really care much about state there anyway, and it's probably
because I'm trying to do something silly anyway, that happens often)
On a somewhat related note, the localnet rule applies if I specify each
class C but the rule ceases to match if I use a CIDR x.x.12.0/22.
Kelley Reynolds
President
Inside Systems, Inc.
