[pmacct-discussion] pmacct + ELK made easy?

2018-03-03 Thread Mike Hammett
Anyone know of a good A - Z pmacct - ELK stack guide? Debian preferred, but not required. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com ___ pmacct-discussion

Re: [pmacct-discussion] pmacct + ELK made easy?

2018-03-03 Thread Jon Nistor
That would be really awesome if there were a guide :> From: Mike Hammett Reply: pmacct-discussion@pmacct.net Date: March 3, 2018 at 9:03:00 AM To: pmacct-discussion@pmacct.net Subject: 

Re: [pmacct-discussion] Wait for BGP peering and massiv ipfix data

2018-03-03 Thread Paolo Lucente
Hi Andrey, Inline: On Fri, Mar 02, 2018 at 02:06:30PM +0200, Andrey Koblyuk wrote: > 1) sql_startup_delay does not work for me . I would like to postpone the > first data processing/cache purging before BGP peering is up. otherwise the > table contains data without information from BGP

Re: [pmacct-discussion] pmacct + ELK made easy?

2018-03-03 Thread Anthony Caiafa
Depending on your level of netflow you may have to look for an alternative backend. I am currently working on a post that describes how I am using pmacct to process about 100Billion records a day and storing it for visualization with superset. On Sat, Mar 3, 2018 at 11:15 AM Paolo Lucente

Re: [pmacct-discussion] Ability to filter by as_src, net_src in tee plugin

2018-03-03 Thread Paolo Lucente
Hi Stanislaw, Such a feature exists for sFlow/sfacctd in tee mode but not (yet) for NetFlow/IPFIX/nfacctd. There are definitely plans to introduce it already for some time (complexities of the porting include template management, options, etc.), it should happen later in the year with 1.7.2.

Re: [pmacct-discussion] pmacct + ELK made easy?

2018-03-03 Thread Paolo Lucente
Excellent stuff, Anthony. Precisely what we need, experience from ops. Look forward to it and to giving it the proper visibility. Paolo On Sat, Mar 03, 2018 at 04:24:12PM +, Anthony Caiafa wrote: > Depending on your level of netflow you may have to look for an alternative > backend. I am

Re: [pmacct-discussion] Juniper IPFIX (as_src)

2018-03-03 Thread Paolo Lucente
Hi Andrey, Nice solution using bgp_stdcomm_pattern_to_asn to fit the bill, thanks for your feedback. Paolo On Thu, Mar 01, 2018 at 02:21:49PM +0200, Andrey Koblyuk wrote: > Hi, Paolo! > > Thanks for your reply! > > Unfortunately, the configuration you proposed is only partially suitable. >

[pmacct-discussion] MongoDB plugin

2018-03-03 Thread Paolo Lucente
Dearests, I'm trying to sense how much interest there is (still) around a MongoDB plugin since the current plans are to phase it out with the 1.7 train this year. If you have a GitHub account, can you please +1 the following issue to show interest? https://github.com/pmacct/pmacct/issues/187

[pmacct-discussion] Export Netflow with BGP?

2018-03-03 Thread Mike Hammett
My routers (Mikrotik) don't export the ASN information with the netflow data. It looks like the tee plugin can send netflow on to another device. However, it does NOT look like I can use pmacct's ability to join netflow with BGP and then export that as netflow. Correct? - Mike Hammett

Re: [pmacct-discussion] Wait for BGP peering and massiv ipfix data

2018-03-03 Thread Andrey Koblyuk
Hi Paolo! >> 1) sql_startup_delay does not work for me . I would like to postpone the >> first data processing/cache purging before BGP peering is up. otherwise the >> table contains data without information from BGP daemon. > Correlation happens when flow data is received - not upon purge. So

Re: [pmacct-discussion] pmacct + ELK made easy?

2018-03-03 Thread Anthony Caiafa
It seems you can probably build one based off these two https://blog.pierky.com/integration-of-pmacct-with-elasticsearch-and-kibana/ https://blogs.cisco.com/security/step-by-step-setup-of-elk-for-netflow-analytics I am sure with a little more is googling you’ll be able to find something or put

Re: [pmacct-discussion] Export Netflow with BGP?

2018-03-03 Thread Paolo Lucente
Hi Mike, If you are collecting with nfacctd, you can enrich NetFlow data once collecting (not replicating); then you can export to 3rd party elements via files (print plugin) or Kafka/RabbitMQ. Enrichment of BGP data at the replicator is not supported and i doubt it will be in future since it is

Re: [pmacct-discussion] pmacct + ELK made easy?

2018-03-03 Thread Mike Hammett
Perhaps I should back up and request a beginners guide to pmacct. Most of what I've read today has largely assumed you already know what you're doing. I haven't found a good from the ground-up setup guide. I generally prefer installing whatever package is in the distro's repository to make

Re: [pmacct-discussion] pmacct + ELK made easy?

2018-03-03 Thread Job Snijders
Are you saying you would be interested in professional services? ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists