Hi Paolo:
On Mon, Jul 28, 2008 at 10:05:25PM +0100, Paolo Lucente wrote:
> Hi John,
>
> thanks very much for your interesting email and for your interest
> into the pmacct project. Please follow me inline:
>
> On Sun, Jul 27, 2008 at 08:07:45AM +0000, John Rouillard wrote:
>
> > Just started using pmacct (0.11.5) this evening on a Centos 4.5
> > machine. I wanted to aggregate a number of internal networks together
> > while retaining individual host stats for hosts that aren't in one of
> > those networks.
>
> This is not supported by the networks_file directive. [...]
>
> An alternate way to accomplish what you would like to, is to
> use the pre_tag_map directive along with a filter, in libpcap
> style. Pre-Tagging allows to stick a tag to the packet or flow
> being considered, which can be used internally (ie. to split
> traffic among different plugins) or externally (ie. shown in
> the backend to the user).
>
> Applying Pre-Tagging to your specific case means you can split
> the traffic: specific traffic can be delivered to a plugin for
> subsequent aggregation by the networks_file map, like you are
> doing currently; the remaining is delivered to another plugin
> and can be even aggregated differently (ie. with more details
> to get insights) compared to the one of interest.
>
> The only issue with this approach is that currently the lines
> in a Pre-Tag Map are limited to slightly more than 256 chars,
> meaning there is space for around 10 prefixes at the most in
> the same filter; i can easily extend this by introducing a new
> directive to let the user to choose the length of the Pre-Tag
> Map line as this already happens for the number of entries the
> map can contain, by default 384). I really have this item on
> the todo list for quite a while now. Splitting prefixes onto
> different filters/tag is doable at the moment but would result
> in a less efficient approach. Will let you know as soon as this
> feature is committed in the CVS.
>
> Following there is an example of the config:
>
> pmacctd.conf:
> ...
> plugins: memory[in], memory[in2]
> !
> pre_tag_map: /path/to/pretag.map
> !
> aggregate[in]: dst_net
> pre_tag_filter[in]: 1,2
I assume I would also specify:
networks_file[in]: pmnet.lst
> !
> aggregate[in2]: dst_host
> pre_tag_filter[in2]: 0
> ...
>
> pretag.map:
> id=1 filter='dst net 192.168.2.0/24 or dst net 192.168.3.0/24 or dst net
> 192.168.5.0'
> id=2 filter='dst net 192.168.7.0/24 or dst net 192.168.9.0/24 or dst net
> 192.168.12.0'
Hmm, ok. I see how that works. Nice. I would have to query two
separate plugins and merge/sort their results manually to get an
overall view of traffic, but it does make it easier to query:
the top traffic generators on my networks
the top hosts not on my networks generating traffic
One other idea on the filters may be:
id=1 filter='dst net 192.168.2.0/24 or dst net 192.168.3.0/24 or dst net
192.168.5.0'
id=1 filter='dst net 192.168.7.0/24 or dst net 192.168.9.0/24 or dst net
192.168.12.0'
That is allow defining multiple filters for the same ID rather than
making the filters longer. This handles the case where the hosts you
want to analyze are:
spec1 or spec2 or spec3 or spec4 ...
It could be specified as:
id=1 spec1
id=1 spec2
id=1 spec3
Currently you map multiple filter tag id's to a single plugin
pre_tag_filter[in]: 1,2
Defining multiple specs for a single id allows mapping multiple filter
specs to a single ID. Internally you could still turn it into:
(spec1) or (spec2) or (spec3) -> tag with id = 1
to pass it to pcap or whatever.
> > How does the "networks_mask" keyword work?
> >
> > KEY: networks_mask
> > DESC: specifies the network mask - in bits - to apply to IP address
> > values in L3 header. The mask is applied sistematically and
> > (systematically btw)
> > before evaluating the 'networks_file' content (if any is
> > specified).
> >
> > This doesn't really tell me how it is applied, and there are no
> > matches for networks_mask in the docs directory. If I was to set that
> > to 32, would I get what I want? Does it set the default "network" for
> > a host using the networks_mask and a more specific mask can be set in
> > the networks_file? Hmm, just tried setting that to /24 to see if I got
> > a 192.168.10.0 entry in the output above and I didn't so I am truly
> > confused as to what this keyword does.
>
> You need to make sure you are using src_net or dst_net primitives (and
> not src_host or dst_host) in the aggregate directive. This appears to
> not be properly specified in the docs, will take action on this. And
> yes, it it set the default "network" for a host using the networks_mask
> and a more specific mask can be set in the networks_file.
Ahh, so I need:
aggregate[network_in]: src_net, dst_net
aggregate_filter[network_in]: dst net 192.168.7.0/24
imt_path[network_in]: /tmp/collect.out.network_in
networks_mask[network_in]: 255.255.255.0
and not just:
networks_mask: 255.255.255.0
My original idea wouldn't work as I was specifying netmasks of
192.168.n.0/24
which are less specific than the /32.
> > And while I am at it, it would be nice to allow:
> >
> > pmacct -c src_net -N '192.168.0.0/22'
> >
> > so that CIDR or netmasks could be used to select networks to report
> > on.
>
> Might spend a thought at this.
Cool.
Also I have a need to gather stats on traceroute packets across the
wan from our monitoring network. This is the approach I was thinking
of (after reading the tagging explanation above)
pretag.map:
id=1 filter='(udp or ip) and ip[8] < 10'
This should tag udp packets with a ttl of less than 10.
then I can create a plugin definition:
aggregate[traceroute]: src_host, dst_net
pre_tag_filter[traceroute]: 1
networks_mask[traceroute]: 255.255.255.0
to track all the bytes being sent out. Does this look right?
--
-- rouilj
John Rouillard
System Administrator
Renesys Corporation
603-244-9084 (cell)
603-643-9300 x 111
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
