On Thu, Apr 06, 2006 at 09:52:34AM -0400, Peter wrote:
> > Do you know if there is something going on to make this possible?
> > And today the only way is a rule for each customer IP in pf.conf
> > then....?
> > Or are there maybe other tools except labels in PF to make this
> > statistics to
> > work in an easy way?
>
> I'm just a poor user like yourself. To me, it doesn't sound like a big
> change. Maybe Daniel can let us know.
Obviously, if you want per-IP counters, the kernel needs to allocate
memory for each counter per IP. It should be clear that if you want to
have individual counters for 100,000 addresses, you need to allocate a
memory for those 100,000 counters. Since we're talking about a
non-trivial amount of memory there, there's no way pf will automatically
keep such counters by default, on the off-chance that some users will
actually query some of them.
There's two ways to get such counters already. Adding individual rules
per IP is the first one. This requires more memory (which any solution
will), but also makes ruleset evaluation slower. I agree that it's not
an elegant solution for thousands of IP addresses.
The other is address tables. If you change your ruleset to
table <coloc_ips_1> const { 65.45.128.128, 65.45.128.129, \
65.45.128.130, ..., 65.45.128.254, 65.45.128.255 }
pass in quick on $EXTERNAL_INT inet from any to <COLOC_IPS_1> \
keep state
you get counters allocated and updated for every address in the table,
and you can query them with
# pfctl -t COLOC_IPS_1 -vTs
65.45.128.128/32
Cleared: Tue Mar 14 14:22:32 2006
In/Block: [ Packets: 0 Bytes: 0 ]
In/Pass: [ Packets: 0 Bytes: 0 ]
Out/Block: [ Packets: 0 Bytes: 0 ]
Out/Pass: [ Packets: 0 Bytes: 0 ]
65.45.128.129/32
...
If you want multiple counters for each IP address, like
per-IP-and-protocol counters, you'll need to duplicate the table for
each protocol, i.e. multiple tables containing the same addresses, for
the sake of allocating multiple counters per address:
table <coloc_ips_1_http> const { 65.45.128.128, 65.45.128.129, \
65.45.128.130, ..., 65.45.128.254, 65.45.128.255 }
table <coloc_ips_1_smtp> const { 65.45.128.128, 65.45.128.129, \
65.45.128.130, ..., 65.45.128.254, 65.45.128.255 }
...
That's pretty optimal with regards to memory usage. The address itself
is small compared to all the counters associated with it. If you want
separate counters per protocol, you HAVE to allocate the counters per
protocol and address. Allocating the address itself twice is a
relatively small waste.
So, I consider this a sufficiently elegant (existing!) solution, the
only annoying thing is that you have to manually enumerate all IPs
within the netblock.
This could be improved by adding a little syntactic sugar to pfctl,
introducing some optional syntax for table additions, like
table ... { 10.1.1.1, 10.2.2/24*, 10.2.3/24 }
where the '*' means 'don't add the netblock itself, but instead generate
and insert all individual address within that netblock), i.e. the above
table would then contain the entries
10.1.1.1
10.2.2.0
10.2.2.1
10.2.2.2
...
10.2.2.255
10.2.3/24
This doesn't HAVE to be done by pfctl itself, you can generate the list
automatically with jot(1) or similar, but I guess it might be nice.
Anything beyond this, like 'I want to track a whole /8, but I don't have
the memory to pre-allocate 2^24 counters, I want counters allocated on
demand for those addresses actually seen', is NOT a simple change.
Daniel
