I've solved this issue using aggregate_filter[]. I wrote simple script that generates pcap filter from IX networks list: http://download.datahata.by/parse.php.txt
But the only one question: is there any difference between aggregate_filter and pre_tag_map inside pmacctd? Which one is faster? Or it doesn't matter in my case? On Fri, Sep 21, 2012 at 4:32 PM, Evgeniy Kozhuhovskiy <[email protected]> wrote: > I have a list of networks in our IX: http://noc.datahata.by/free.txt > > Our customer networks are 31.130.200.0/21 and 178.172.181.0/24 > > I wrote a script that generates from IX networks list filter like this: > > debian:~# ./parse | head > id=1 ip=31.130.xxx.xxx filter='dst net 31.24.88.0/21 and not dst net > 31.130.200.0/21' > id=1 ip=31.130.xxx.xxx filter='dst net 31.24.88.0/21 and not dst net > 178.172.181.0/24' > id=1 ip=31.130.xxx.xxx filter='dst net 31.130.200.0/21 and not dst net > 31.130.200.0/21' > id=1 ip=31.130.xxx.xxx filter='dst net 31.130.200.0/21 and not dst net > 178.172.181.0/24' > id=1 ip=31.130.xxx.xxx filter='dst net 46.28.96.0/21 and not dst net > 31.130.200.0/21' > id=1 ip=31.130.xxx.xxx filter='dst net 46.28.96.0/21 and not dst net > 178.172.181.0/24' > id=1 ip=31.130.xxx.xxx filter='dst net 46.53.128.0/17 and not dst net > 31.130.200.0/21' > id=1 ip=31.130.xxx.xxx filter='dst net 46.53.128.0/17 and not dst net > 178.172.181.0/24' > id=1 ip=31.130.xxx.xxx filter='dst net 46.56.0.0/16 and not dst net > 31.130.200.0/21' > id=1 ip=31.130.xxx.xxx filter='dst net 46.56.0.0/16 and not dst net > 178.172.181.0/24' > > Is this correct? > > On Wed, Sep 19, 2012 at 11:01 AM, Paolo Lucente <[email protected]> wrote: >> Hi Evgeniy, >> >> It might be better (more maintanable) to use a different approach >> for your task. Remove the aggregate_filter and go for pre-tagging: >> >> pre_tag_map: /path/to/pretag.map >> pre_tag_filter[bla]: 0 >> refresh_maps: true >> pre_tag_map_entries: <say 1000? it's an upper bound anyway> >> >> It means filter in traffic tagged as 0 (no tag effectively). Then >> build /path/to/pretag.map as follows (with the idea what is being >> tagged with id == 1 within the map is then being discarded): >> >> id=1 ip=<NetFlow exporter IP address> filter='dst net <IX net, ie. >> 192.168.0.0/16> and not dst net <local net, ie. 192.168.100.0/24>' >> id=1 ip=<NetFlow exporter IP address> filter='src net <IX net, ie. >> 192.168.0.0/16> and not src net <local net, ie. 192.168.100.0/24>' >> ... >> >> You can indeed merge the two rules above in a single one, i did >> not do it just for readability. My current understanding of your >> goal is that you want to bill customer/customer traffic but not >> customer/IX and vice-versa. >> >> Cheers, >> Paolo >> >> On Wed, Sep 19, 2012 at 12:19:45AM +0300, Evgeniy Kozhuhovskiy wrote: >>> Thanks a lot, it works for now. >>> >>> Also i have another question, related to pcap filter. >>> >>> I have a list of networks in our IX. It has some big networks that >>> includes small networks >>> that are announced from our AS. >>> For example, we have 192.168.0.0/16 and 172.16.0.0/16 in IX, >>> but 192.168.100.0/24 and 172.16.100.0/24 are our clients networks. >>> >>> Of course, we need to bill traffic from/to our clients >>> (192.168.100/24, 172.16.100.0/24), but we dont need to bill >>> traffic to 192.168.0.0/16 and 172.16.0.0/16. >>> >>> I wrote such rule: >>> aggregate_filter[dsmgr]: (net not 192.168.0.0/16 and net not >>> 172.16.0.0/16) and (net 192.168.100/24 or net 172.16.100.0/24) >>> >>> but this filter returns no traffic at all (because it removes our >>> customers networks from result). Where I'm wrong in filter? >>> >>> On Tue, Sep 18, 2012 at 7:05 PM, Paolo Lucente <[email protected]> wrote: >>> > >>> > Hi Evgeniy, >>> > >>> > You are doing all good except aggregate_filter need to be bound to >>> > a specific plugin, can't be global. The daemon is surely spitting an >>> > error out about that. Please rewrite the two following lines and try >>> > again: >>> > >>> > plugins: print[bla] >>> > aggregate_filter[bla]: net not 178.120.0.0/13 >>> > >>> > It should be that the daemon is logging errors somewhere, you might >>> > want to add a 'logfile' directive for catching errors, etc. >>> > >>> > Cheers, >>> > Paolo >>> > >>> > On Tue, Sep 18, 2012 at 04:38:47PM +0300, Evgeniy Kozhuhovskiy wrote: >>> > > Hello. >>> > > >>> > > We're using some proprietary software (dsmgr by ispsystems), that uses >>> > > nfacctd as it's part to calculate traffic usage (by netflow/sflow). >>> > > >>> > > By default dsmgr generate such nfacctd config: >>> > > daemonize: true >>> > > plugins: print >>> > > aggregate: src_host,dst_host >>> > > nfacctd_port: 9995 >>> > > print_refresh_time: 900 >>> > > print_output: csv >>> > > print_output_file: /var/flowstat/%Y:%m:%d-%H:%M >>> > > >>> > > and csv file is processed via proprietary utility runned from cron. >>> > > >>> > > We need to exclude some networks (in fact, our local IX) from billing. >>> > > I've added this line: >>> > > aggregate_filter: net not 178.120.0.0/13 >>> > > to config. (178.120.0.0/13 is one of networks that we dont need to bill) >>> > > >>> > > But traffic from this network still goes to csv file: >>> > > >>> > > debian:/var/flowstat# cat 2012\:09\:18-16\:23 | grep 178.120 | head -4 >>> > > >>> > > 0,0,unknown,00:00:00:00:00:00,00:00:00:00:00:00,0,0,0,0,0,0,,0,0,0,0,,,0,0,0:0:0,178.120.88.201,178.172.181.110,0,0,0,0,0,ip,0,1072,0,75723 >>> > > >>> > > 0,0,unknown,00:00:00:00:00:00,00:00:00:00:00:00,0,0,0,0,0,0,,0,0,0,0,,,0,0,0:0:0,178.172.181.128,178.120.217.213,0,0,0,0,0,ip,0,22085,0,16089127 >>> > > >>> > > 0,0,unknown,00:00:00:00:00:00,00:00:00:00:00:00,0,0,0,0,0,0,,0,0,0,0,,,0,0,0:0:0,178.172.181.95,178.120.60.190,0,0,0,0,0,ip,0,1695,0,159303 >>> > > >>> > > 0,0,unknown,00:00:00:00:00:00,00:00:00:00:00:00,0,0,0,0,0,0,,0,0,0,0,,,0,0,0:0:0,178.120.112.234,178.172.181.64,0,0,0,0,0,ip,0,1326,0,120156 >>> > > >>> > > Where i'm wrong? >>> >>> >>> -- >>> With best regards, Evgeniy Kozhuhovskiy > > > > -- > With best regards, Evgeniy Kozhuhovskiy -- With best regards, Evgeniy Kozhuhovskiy _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
