Re: [pmacct-discussion] nfacctd aggregate_filters not working correctly when defined in the same config file

2017-01-22 Thread Aaron Finney
t; Paolo > > On Sat, Jan 21, 2017 at 09:09:25AM -0800, Aaron Finney wrote: > > Hi Paolo, > > > > It's version 1.6.1: > > > > NetFlow Accounting Daemon, nfacctd 1.6.1 (20161001-00+c5). > > > > Thanks, > > > > Aaron > > >

Re: [pmacct-discussion] nfacctd aggregate_filters not working correctly when defined in the same config file

2017-01-21 Thread Aaron Finney
e > 1.6.1 or (much preferrably) master code on GitHub - can you please try > and confirm you experience the same with any of these? > > Paolo > > On Fri, Jan 20, 2017 at 07:03:15PM -0800, Aaron Finney wrote: > > Hello all, > > > > I promise I searched the archives e

[pmacct-discussion] nfacctd aggregate_filters not working correctly when defined in the same config file

2017-01-20 Thread Aaron Finney
Hello all, I promise I searched the archives exhaustively first... We are trying to separate external ingress/egress traffic using aggregate_filter (config below), but it's not working as expected. When we only have one of the sections active and (xv_ext_in OR xv_ext_out) and comment out the

Re: [pmacct-discussion] Issues with Kafka/Avro sending schema to Kafka topic

2017-02-17 Thread Aaron Finney
net> wrote: > > Hi Aaron, > > The feature is post 1.6.1. Can you please switch to master code on GitHub? > > Thanks, > Paolo > > On Thu, Feb 16, 2017 at 10:44:23AM -0800, Aaron Finney wrote: > > Hi Paolo/all, > > > > I've been unable to get nfacctd

[pmacct-discussion] Issues with Kafka/Avro sending schema to Kafka topic

2017-02-16 Thread Aaron Finney
Hi Paolo/all, I've been unable to get nfacctd to send the Avro schema to a Kafka topic - I receive the following message when starting nfacctd: WARN: [/etc/nfacctd.conf:14] Unknown key: kafka_avro_schema_topic. Ignored. WARN: [/etc/nfacctd.conf:15] Unknown key: kafka_avro_schema_refresh_time.

Re: [pmacct-discussion] Configuring pmacct/nfacctd as a Proxy/Ingest node & Visualize Data

2017-06-12 Thread Aaron Finney
I second Catalin's answer; once you have the data in Kafka, you can have multiple consumers working with it for different purposes. Our solution sends raw flow data to Kafka topics with a 24 hour retention time, then we have spark streaming jobs that do near-real-time processing for anomaly

Re: [pmacct-discussion] Load balancing nfacctd

2017-09-05 Thread Aaron Finney
ason, I don’t have to go touch the router configs. > > > On Sep 5, 2017, at 11:35 AM, Aaron Finney <aaron.fin...@openx.com> wrote: > > I'm not sure I follow - do you mean setting up BGP peering of the > collectors to your source devices using the collector VIP as the neigh

Re: [pmacct-discussion] Load balancing nfacctd

2017-09-05 Thread Aaron Finney
long with > netflow? Interested in the solution below but would like to have BGP > aligned with netflow as well. > > On Sep 4, 2017, at 9:48 AM, Aaron Finney <aaron.fin...@openx.com> wrote: > > Great to hear, nice work! > > Aaron > > On Sep 4, 2017 1:55 AM

Re: [pmacct-discussion] Load balancing nfacctd

2017-08-21 Thread Aaron Finney
Hi Yann We use Consul for this, it works very well. https://www.consul.io Aaron On Aug 21, 2017 6:44 AM, "Yann Belin" wrote: Hello, I have been looking into solutions to achieve reliable load balancing of my incoming flows across multiple nfacctd servers / daemons.

Re: [pmacct-discussion] Load balancing nfacctd

2017-09-04 Thread Aaron Finney
one for rabbitmq (e.g. queue length / ram usage). I'm still thinking about more advanced ways to check nfacctd health, if anyone has a suggestion. Cheers, Yann On Mon, Aug 21, 2017 at 4:02 PM, Aaron Finney <aaron.fin...@openx.com> wrote: > Hi Yann > > We use Consul for this