Re: [pmacct-discussion] collecting large number of netflows

2016-08-19 Thread Jentsch, Mario
ussion-boun...@pmacct.net] > On Behalf Of Stephen Clark > Sent: Thursday, August 18, 2016 2:24 PM > To: pmacct-discussion@pmacct.net > Subject: Re: [pmacct-discussion] collecting large number of netflows > > On 08/17/2016 08:38 AM, Jentsch, Mario wrote: > > Hey Steve, > >

Re: [pmacct-discussion] collecting large number of netflows

2016-08-18 Thread Stephen Clark
- From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net] On Behalf Of Stephen Clark Sent: Thursday, August 04, 2016 5:01 PM To: pmacct-discussion@pmacct.net Subject: [pmacct-discussion] collecting large number of netflows Hi List, I am looking to collect a large number of netflow records, on the

Re: [pmacct-discussion] collecting large number of netflows

2016-08-17 Thread Jentsch, Mario
Clark > Sent: Thursday, August 04, 2016 5:01 PM > To: pmacct-discussion@pmacct.net > Subject: [pmacct-discussion] collecting large number of netflows > > Hi List, > > I am looking to collect a large number of netflow records, on the order of a > 100 > million a da

Re: [pmacct-discussion] collecting large number of netflows

2016-08-04 Thread Dariush Marsh-Mossadeghi
For that sort of volume I’d investigate using and ELK stack on the backend and de-coupling pmacct from storage and analysis with a message queue like Rabbit. I’ve used this approach (not to that scale though) and it works well. ELK == ElasticSearch Logstash Kibana > > > On 08/04/2016 11:48 AM

Re: [pmacct-discussion] collecting large number of netflows

2016-08-04 Thread Stephen Clark
Hmm... I don't think pmacct directly support cassandra, but is does support MongoDB. Also I would like to be able to filter/sample the data at the point of origin, but I don't think that is possible with pmacctd and the nfprobe module. It only seems to be able to do it at the collector - nfacc

[pmacct-discussion] collecting large number of netflows

2016-08-04 Thread Stephen Clark
Hi List, I am looking to collect a large number of netflow records, on the order of a 100 million a day, and store them in a postgres DB. Has anyone done this or something similar using pmacct? Thanks, Steve ___ pmacct-discussion mailing list http