Re: [pmacct-discussion] collecting large number of netflows

2016-08-04 Thread Dariush Marsh-Mossadeghi
For that sort of volume I’d investigate using and ELK stack on the backend and de-coupling pmacct from storage and analysis with a message queue like Rabbit. I’ve used this approach (not to that scale though) and it works well. ELK == ElasticSearch Logstash Kibana > > > On 08/04/2016 11:48

Re: [pmacct-discussion] collecting large number of netflows

2016-08-04 Thread Stephen Clark
Hmm... I don't think pmacct directly support cassandra, but is does support MongoDB. Also I would like to be able to filter/sample the data at the point of origin, but I don't think that is possible with pmacctd and the nfprobe module. It only seems to be able to do it at the collector -

[pmacct-discussion] collecting large number of netflows

2016-08-04 Thread Stephen Clark
Hi List, I am looking to collect a large number of netflow records, on the order of a 100 million a day, and store them in a postgres DB. Has anyone done this or something similar using pmacct? Thanks, Steve ___ pmacct-discussion mailing list