Thanks a lot! aggregate_filter seems more appropriate for me - IX networks list changes not so often.
PS Thanks you for really good and flexible product - pmacctd. On Tue, Sep 25, 2012 at 10:31 AM, Paolo Lucente <[email protected]> wrote: > Hi Evgeniy, > > This is great to know. aggregate_filter is quicker for the simple > reason it is a giant filter whereas pre_tag_map makes you specify > a collection of filters. I would add to the equation: don't under > estimate maintainability, which is much better with pre_tag_map: > ie. you can reload pre_tag_map content at runtime whereas you have > to restart pmacct in order to reload an aggregate_filter. > > Cheers, > Paolo > > > On Tue, Sep 25, 2012 at 12:27:22AM +0300, Evgeniy Kozhuhovskiy wrote: > > I've solved this issue using aggregate_filter[]. > > I wrote simple script that generates pcap filter from IX networks > > list: http://download.datahata.by/parse.php.txt > > > > But the only one question: is there any difference between > > aggregate_filter and pre_tag_map > > inside pmacctd? Which one is faster? Or it doesn't matter in my case? > > > > On Fri, Sep 21, 2012 at 4:32 PM, Evgeniy Kozhuhovskiy > > <[email protected]> wrote: > > > I have a list of networks in our IX: http://noc.datahata.by/free.txt > > > > > > Our customer networks are 31.130.200.0/21 and 178.172.181.0/24 > > > > > > I wrote a script that generates from IX networks list filter like this: > > > > > > debian:~# ./parse | head > > > id=1 ip=31.130.xxx.xxx filter='dst net 31.24.88.0/21 and not dst net > > > 31.130.200.0/21' > > > id=1 ip=31.130.xxx.xxx filter='dst net 31.24.88.0/21 and not dst net > > > 178.172.181.0/24' > > > id=1 ip=31.130.xxx.xxx filter='dst net 31.130.200.0/21 and not dst net > > > 31.130.200.0/21' > > > id=1 ip=31.130.xxx.xxx filter='dst net 31.130.200.0/21 and not dst net > > > 178.172.181.0/24' > > > id=1 ip=31.130.xxx.xxx filter='dst net 46.28.96.0/21 and not dst net > > > 31.130.200.0/21' > > > id=1 ip=31.130.xxx.xxx filter='dst net 46.28.96.0/21 and not dst net > > > 178.172.181.0/24' > > > id=1 ip=31.130.xxx.xxx filter='dst net 46.53.128.0/17 and not dst net > > > 31.130.200.0/21' > > > id=1 ip=31.130.xxx.xxx filter='dst net 46.53.128.0/17 and not dst net > > > 178.172.181.0/24' > > > id=1 ip=31.130.xxx.xxx filter='dst net 46.56.0.0/16 and not dst net > > > 31.130.200.0/21' > > > id=1 ip=31.130.xxx.xxx filter='dst net 46.56.0.0/16 and not dst net > > > 178.172.181.0/24' > > > > > > Is this correct? > > > > > > On Wed, Sep 19, 2012 at 11:01 AM, Paolo Lucente <[email protected]> > wrote: > > >> Hi Evgeniy, > > >> > > >> It might be better (more maintanable) to use a different approach > > >> for your task. Remove the aggregate_filter and go for pre-tagging: > > >> > > >> pre_tag_map: /path/to/pretag.map > > >> pre_tag_filter[bla]: 0 > > >> refresh_maps: true > > >> pre_tag_map_entries: <say 1000? it's an upper bound anyway> > > >> > > >> It means filter in traffic tagged as 0 (no tag effectively). Then > > >> build /path/to/pretag.map as follows (with the idea what is being > > >> tagged with id == 1 within the map is then being discarded): > > >> > > >> id=1 ip=<NetFlow exporter IP address> filter='dst net <IX net, ie. > 192.168.0.0/16> and not dst net <local net, ie. 192.168.100.0/24>' > > >> id=1 ip=<NetFlow exporter IP address> filter='src net <IX net, ie. > 192.168.0.0/16> and not src net <local net, ie. 192.168.100.0/24>' > > >> ... > > >> > > >> You can indeed merge the two rules above in a single one, i did > > >> not do it just for readability. My current understanding of your > > >> goal is that you want to bill customer/customer traffic but not > > >> customer/IX and vice-versa. > > >> > > >> Cheers, > > >> Paolo > > >> > > >> On Wed, Sep 19, 2012 at 12:19:45AM +0300, Evgeniy Kozhuhovskiy wrote: > > >>> Thanks a lot, it works for now. > > >>> > > >>> Also i have another question, related to pcap filter. > > >>> > > >>> I have a list of networks in our IX. It has some big networks that > > >>> includes small networks > > >>> that are announced from our AS. > > >>> For example, we have 192.168.0.0/16 and 172.16.0.0/16 in IX, > > >>> but 192.168.100.0/24 and 172.16.100.0/24 are our clients networks. > > >>> > > >>> Of course, we need to bill traffic from/to our clients > > >>> (192.168.100/24, 172.16.100.0/24), but we dont need to bill > > >>> traffic to 192.168.0.0/16 and 172.16.0.0/16. > > >>> > > >>> I wrote such rule: > > >>> aggregate_filter[dsmgr]: (net not 192.168.0.0/16 and net not > > >>> 172.16.0.0/16) and (net 192.168.100/24 or net 172.16.100.0/24) > > >>> > > >>> but this filter returns no traffic at all (because it removes our > > >>> customers networks from result). Where I'm wrong in filter? > > >>> > > >>> On Tue, Sep 18, 2012 at 7:05 PM, Paolo Lucente <[email protected]> > wrote: > > >>> > > > >>> > Hi Evgeniy, > > >>> > > > >>> > You are doing all good except aggregate_filter need to be bound to > > >>> > a specific plugin, can't be global. The daemon is surely spitting > an > > >>> > error out about that. Please rewrite the two following lines and > try > > >>> > again: > > >>> > > > >>> > plugins: print[bla] > > >>> > aggregate_filter[bla]: net not 178.120.0.0/13 > > >>> > > > >>> > It should be that the daemon is logging errors somewhere, you might > > >>> > want to add a 'logfile' directive for catching errors, etc. > > >>> > > > >>> > Cheers, > > >>> > Paolo > > >>> > > > >>> > On Tue, Sep 18, 2012 at 04:38:47PM +0300, Evgeniy Kozhuhovskiy > wrote: > > >>> > > Hello. > > >>> > > > > >>> > > We're using some proprietary software (dsmgr by ispsystems), > that uses > > >>> > > nfacctd as it's part to calculate traffic usage (by > netflow/sflow). > > >>> > > > > >>> > > By default dsmgr generate such nfacctd config: > > >>> > > daemonize: true > > >>> > > plugins: print > > >>> > > aggregate: src_host,dst_host > > >>> > > nfacctd_port: 9995 > > >>> > > print_refresh_time: 900 > > >>> > > print_output: csv > > >>> > > print_output_file: /var/flowstat/%Y:%m:%d-%H:%M > > >>> > > > > >>> > > and csv file is processed via proprietary utility runned from > cron. > > >>> > > > > >>> > > We need to exclude some networks (in fact, our local IX) from > billing. > > >>> > > I've added this line: > > >>> > > aggregate_filter: net not 178.120.0.0/13 > > >>> > > to config. (178.120.0.0/13 is one of networks that we dont need > to bill) > > >>> > > > > >>> > > But traffic from this network still goes to csv file: > > >>> > > > > >>> > > debian:/var/flowstat# cat 2012\:09\:18-16\:23 | grep 178.120 | > head -4 > > >>> > > > > >>> > > > 0,0,unknown,00:00:00:00:00:00,00:00:00:00:00:00,0,0,0,0,0,0,,0,0,0,0,,,0,0,0:0:0,178.120.88.201,178.172.181.110,0,0,0,0,0,ip,0,1072,0,75723 > > >>> > > > > >>> > > > 0,0,unknown,00:00:00:00:00:00,00:00:00:00:00:00,0,0,0,0,0,0,,0,0,0,0,,,0,0,0:0:0,178.172.181.128,178.120.217.213,0,0,0,0,0,ip,0,22085,0,16089127 > > >>> > > > > >>> > > > 0,0,unknown,00:00:00:00:00:00,00:00:00:00:00:00,0,0,0,0,0,0,,0,0,0,0,,,0,0,0:0:0,178.172.181.95,178.120.60.190,0,0,0,0,0,ip,0,1695,0,159303 > > >>> > > > > >>> > > > 0,0,unknown,00:00:00:00:00:00,00:00:00:00:00:00,0,0,0,0,0,0,,0,0,0,0,,,0,0,0:0:0,178.120.112.234,178.172.181.64,0,0,0,0,0,ip,0,1326,0,120156 > > >>> > > > > >>> > > Where i'm wrong? > > >>> > > >>> > > >>> -- > > >>> With best regards, Evgeniy Kozhuhovskiy > > > > > > > > > > > > -- > > > With best regards, Evgeniy Kozhuhovskiy > > > > > > > > -- > > With best regards, Evgeniy Kozhuhovskiy > -- With best regards, Evgeniy Kozhuhovskiy
_______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
