Re: [pmacct-discussion] nfacctd aggregate_filters not working correctly when defined in the same config file

2017-01-23 Thread Paolo Lucente 

Hi Aaron,

Fantastic, thanks for your feedback!

Paolo

On Sun, Jan 22, 2017 at 12:24:36PM -0800, Aaron Finney wrote:
> Hi Paolo,
> 
> I have been doing more work on this trying to isolate where in the pipeline
> the cross-talk is happening, and it looks like it's on my RabbitMQ consumer
> and not nfacctd - apologies! :)
> 
> For our POC I'm trying to consume the data with logstash -> elasticsearch,
> which was supposed to be a quick way to ingest and do some quick modeling
> with the data.
> 
> Thanks again, we are excited about bringing scalability to flow collection
> in our networks, most likely using Riak as a back-end data store.
> 
> Aaron
> 
> 
> On Sun, Jan 22, 2017 at 10:16 AM, Paolo Lucente  wrote:
> 
> >
> > Hi Aaron,
> >
> > Thanks for the feedback. I'm unfortunately unable to reproduce the issue
> > in lab: any chance you can grant me temporary access to the system where
> > the issue is arising? Or if you are in some containerized environment you
> > can pass me that?
> >
> > Cheers,
> > Paolo
> >
> > On Sat, Jan 21, 2017 at 09:09:25AM -0800, Aaron Finney wrote:
> > > Hi Paolo,
> > >
> > > It's version 1.6.1:
> > >
> > > NetFlow Accounting Daemon, nfacctd 1.6.1 (20161001-00+c5).
> > >
> > > Thanks,
> > >
> > > Aaron
> > >
> > >
> > >
> > > On Sat, Jan 21, 2017 at 3:57 AM, Paolo Lucente  wrote:
> > >
> > > >
> > > > Hi Aaron,
> > > >
> > > > Interesting. Can you say what version is this? And if anything before
> > > > 1.6.1 or (much preferrably) master code on GitHub - can you please try
> > > > and confirm you experience the same with any of these?
> > > >
> > > > Paolo
> > > >
> > > > On Fri, Jan 20, 2017 at 07:03:15PM -0800, Aaron Finney wrote:
> > > > > Hello all,
> > > > >
> > > > > I promise I searched the archives exhaustively first...
> > > > >
> > > > > We are trying to separate external ingress/egress traffic using
> > > > > aggregate_filter (config below), but it's not working as expected.
> > When
> > > > we
> > > > > only have one of the sections active and  (xv_ext_in OR xv_ext_out)
> > and
> > > > > comment out the other, we get exactly the data we expect - only
> > external
> > > > > data and either to/from our networks. When we activate both in the
> > > > config,
> > > > > we end up with a mix of both, but not exactly the same data. Any help
> > > > would
> > > > > be greatly appreciated - thanks!
> > > > >
> > > > >
> > > > > Config:
> > > > >
> > > > > daemonize: false
> > > > > nfacctd_port: 2100
> > > > > nfacctd_net: netflow
> > > > > plugins: amqp[xv_ext_in], amqp[xv_ext_out]
> > > > > !
> > > > > amqp_exchange[xv_ext_in]: netflow-in
> > > > > amqp_exchange_type[xv_ext_in]: direct
> > > > > amqp_host[xv_ext_in]: localhost
> > > > > amqp_refresh_time[xv_ext_in]: 5
> > > > > amqp_user[xv_ext_in]: username
> > > > > amqp_passwd[xv_ext_in]: password
> > > > > aggregate[xv_ext_in]: peer_src_ip, src_as, dst_as, src_host,
> > dst_host,
> > > > > src_port, dst_port, in_iface, out_iface, proto, sampling_rate
> > > > > aggregate_filter[xv_ext_in]: not (src net (173.241.240.0/20 or
> > > > 69.6.80.0/20
> > > > > or 199.26.53.0/24 or 209.182.128.0/19)) and not net 10.0.0.0/8
> > > > > amqp_routing_key[xv_ext_in]: xv_in
> > > > > !
> > > > > amqp_exchange[xv_ext_out]: netflow-out
> > > > > amqp_exchange_type[xv_ext_out]: direct
> > > > > amqp_host[xv_ext_out]: localhost
> > > > > amqp_refresh_time[xv_ext_out]: 5
> > > > > amqp_user[xv_ext_out]: username
> > > > > amqp_passwd[xv_ext_out]: password
> > > > > aggregate[xv_ext_out]: peer_src_ip, src_as, dst_as, src_host,
> > dst_host,
> > > > > src_port, dst_port, in_iface, out_iface, proto, sampling_rate
> > > > > aggregate_filter[xv_ext_out]: not (dst net (173.241.240.0/20 or
> > > > 69.6.80.0/20
> > > > > or 199.26.53.0/24 or 209.182.128.0/19)) and not net 10.0.0.0/8
> > > > > amqp_routing_key[xv_ext_out]: xv_out
> > > >
> > > > > ___
> > > > > pmacct-discussion mailing list
> > > > > http://www.pmacct.net/#mailinglists
> > > >
> > > >
> > > > ___
> > > > pmacct-discussion mailing list
> > > > http://www.pmacct.net/#mailinglists
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > *Aaron Finney*Network Engineer | OpenX
> > > 888 East Walnut Street, 2nd Floor | Pasadena, CA 91101
> > > o: +1 (626) 466-1141 x6035 | aaron.fin...@openx.com
> > > *Advertising Age Best Places to Work
> > >  > of-advertising-ages-top-fifty-best-places-to-work-for-2015/>*
> > > *Deloitte's Technology Fast 500™
> > >  > fastest-growing-software-company-north-america-5th-
> > fastest-overall-deloittes-2013-technology-fast-500/>*
> > > www.openx.com   |  Twitter
> > > |  Facebook   |
> > > LinkedIn   |  YouTube
>

Re: [pmacct-discussion] nfacctd aggregate_filters not working correctly when defined in the same config file

2017-01-22 Thread Paolo Lucente 

Hi Aaron,

Thanks for the feedback. I'm unfortunately unable to reproduce the issue
in lab: any chance you can grant me temporary access to the system where
the issue is arising? Or if you are in some containerized environment you
can pass me that?

Cheers,
Paolo
 
On Sat, Jan 21, 2017 at 09:09:25AM -0800, Aaron Finney wrote:
> Hi Paolo,
> 
> It's version 1.6.1:
> 
> NetFlow Accounting Daemon, nfacctd 1.6.1 (20161001-00+c5).
> 
> Thanks,
> 
> Aaron
> 
> 
> 
> On Sat, Jan 21, 2017 at 3:57 AM, Paolo Lucente  wrote:
> 
> >
> > Hi Aaron,
> >
> > Interesting. Can you say what version is this? And if anything before
> > 1.6.1 or (much preferrably) master code on GitHub - can you please try
> > and confirm you experience the same with any of these?
> >
> > Paolo
> >
> > On Fri, Jan 20, 2017 at 07:03:15PM -0800, Aaron Finney wrote:
> > > Hello all,
> > >
> > > I promise I searched the archives exhaustively first...
> > >
> > > We are trying to separate external ingress/egress traffic using
> > > aggregate_filter (config below), but it's not working as expected. When
> > we
> > > only have one of the sections active and  (xv_ext_in OR xv_ext_out) and
> > > comment out the other, we get exactly the data we expect - only external
> > > data and either to/from our networks. When we activate both in the
> > config,
> > > we end up with a mix of both, but not exactly the same data. Any help
> > would
> > > be greatly appreciated - thanks!
> > >
> > >
> > > Config:
> > >
> > > daemonize: false
> > > nfacctd_port: 2100
> > > nfacctd_net: netflow
> > > plugins: amqp[xv_ext_in], amqp[xv_ext_out]
> > > !
> > > amqp_exchange[xv_ext_in]: netflow-in
> > > amqp_exchange_type[xv_ext_in]: direct
> > > amqp_host[xv_ext_in]: localhost
> > > amqp_refresh_time[xv_ext_in]: 5
> > > amqp_user[xv_ext_in]: username
> > > amqp_passwd[xv_ext_in]: password
> > > aggregate[xv_ext_in]: peer_src_ip, src_as, dst_as, src_host, dst_host,
> > > src_port, dst_port, in_iface, out_iface, proto, sampling_rate
> > > aggregate_filter[xv_ext_in]: not (src net (173.241.240.0/20 or
> > 69.6.80.0/20
> > > or 199.26.53.0/24 or 209.182.128.0/19)) and not net 10.0.0.0/8
> > > amqp_routing_key[xv_ext_in]: xv_in
> > > !
> > > amqp_exchange[xv_ext_out]: netflow-out
> > > amqp_exchange_type[xv_ext_out]: direct
> > > amqp_host[xv_ext_out]: localhost
> > > amqp_refresh_time[xv_ext_out]: 5
> > > amqp_user[xv_ext_out]: username
> > > amqp_passwd[xv_ext_out]: password
> > > aggregate[xv_ext_out]: peer_src_ip, src_as, dst_as, src_host, dst_host,
> > > src_port, dst_port, in_iface, out_iface, proto, sampling_rate
> > > aggregate_filter[xv_ext_out]: not (dst net (173.241.240.0/20 or
> > 69.6.80.0/20
> > > or 199.26.53.0/24 or 209.182.128.0/19)) and not net 10.0.0.0/8
> > > amqp_routing_key[xv_ext_out]: xv_out
> >
> > > ___
> > > pmacct-discussion mailing list
> > > http://www.pmacct.net/#mailinglists
> >
> >
> > ___
> > pmacct-discussion mailing list
> > http://www.pmacct.net/#mailinglists
> >
> 
> 
> 
> -- 
> 
> *Aaron Finney*Network Engineer | OpenX
> 888 East Walnut Street, 2nd Floor | Pasadena, CA 91101
> o: +1 (626) 466-1141 x6035 | aaron.fin...@openx.com
> *Advertising Age Best Places to Work
> *
> *Deloitte's Technology Fast 500™
> *
> www.openx.com   |  Twitter
> |  Facebook   |
> LinkedIn   |  YouTube
> 

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] nfacctd aggregate_filters not working correctly when defined in the same config file

2017-01-21 Thread Aaron Finney 
Hi Paolo,

It's version 1.6.1:

NetFlow Accounting Daemon, nfacctd 1.6.1 (20161001-00+c5).

Thanks,

Aaron



On Sat, Jan 21, 2017 at 3:57 AM, Paolo Lucente  wrote:

>
> Hi Aaron,
>
> Interesting. Can you say what version is this? And if anything before
> 1.6.1 or (much preferrably) master code on GitHub - can you please try
> and confirm you experience the same with any of these?
>
> Paolo
>
> On Fri, Jan 20, 2017 at 07:03:15PM -0800, Aaron Finney wrote:
> > Hello all,
> >
> > I promise I searched the archives exhaustively first...
> >
> > We are trying to separate external ingress/egress traffic using
> > aggregate_filter (config below), but it's not working as expected. When
> we
> > only have one of the sections active and  (xv_ext_in OR xv_ext_out) and
> > comment out the other, we get exactly the data we expect - only external
> > data and either to/from our networks. When we activate both in the
> config,
> > we end up with a mix of both, but not exactly the same data. Any help
> would
> > be greatly appreciated - thanks!
> >
> >
> > Config:
> >
> > daemonize: false
> > nfacctd_port: 2100
> > nfacctd_net: netflow
> > plugins: amqp[xv_ext_in], amqp[xv_ext_out]
> > !
> > amqp_exchange[xv_ext_in]: netflow-in
> > amqp_exchange_type[xv_ext_in]: direct
> > amqp_host[xv_ext_in]: localhost
> > amqp_refresh_time[xv_ext_in]: 5
> > amqp_user[xv_ext_in]: username
> > amqp_passwd[xv_ext_in]: password
> > aggregate[xv_ext_in]: peer_src_ip, src_as, dst_as, src_host, dst_host,
> > src_port, dst_port, in_iface, out_iface, proto, sampling_rate
> > aggregate_filter[xv_ext_in]: not (src net (173.241.240.0/20 or
> 69.6.80.0/20
> > or 199.26.53.0/24 or 209.182.128.0/19)) and not net 10.0.0.0/8
> > amqp_routing_key[xv_ext_in]: xv_in
> > !
> > amqp_exchange[xv_ext_out]: netflow-out
> > amqp_exchange_type[xv_ext_out]: direct
> > amqp_host[xv_ext_out]: localhost
> > amqp_refresh_time[xv_ext_out]: 5
> > amqp_user[xv_ext_out]: username
> > amqp_passwd[xv_ext_out]: password
> > aggregate[xv_ext_out]: peer_src_ip, src_as, dst_as, src_host, dst_host,
> > src_port, dst_port, in_iface, out_iface, proto, sampling_rate
> > aggregate_filter[xv_ext_out]: not (dst net (173.241.240.0/20 or
> 69.6.80.0/20
> > or 199.26.53.0/24 or 209.182.128.0/19)) and not net 10.0.0.0/8
> > amqp_routing_key[xv_ext_out]: xv_out
>
> > ___
> > pmacct-discussion mailing list
> > http://www.pmacct.net/#mailinglists
>
>
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
>



-- 

*Aaron Finney*Network Engineer | OpenX
888 East Walnut Street, 2nd Floor | Pasadena, CA 91101
o: +1 (626) 466-1141 x6035 | aaron.fin...@openx.com
*Advertising Age Best Places to Work
*
*Deloitte's Technology Fast 500™
*
www.openx.com   |  Twitter
|  Facebook   |
LinkedIn   |  YouTube

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] nfacctd aggregate_filters not working correctly when defined in the same config file

2017-01-21 Thread Paolo Lucente 

Hi Aaron,

Interesting. Can you say what version is this? And if anything before
1.6.1 or (much preferrably) master code on GitHub - can you please try
and confirm you experience the same with any of these?

Paolo

On Fri, Jan 20, 2017 at 07:03:15PM -0800, Aaron Finney wrote:
> Hello all,
> 
> I promise I searched the archives exhaustively first...
> 
> We are trying to separate external ingress/egress traffic using
> aggregate_filter (config below), but it's not working as expected. When we
> only have one of the sections active and  (xv_ext_in OR xv_ext_out) and
> comment out the other, we get exactly the data we expect - only external
> data and either to/from our networks. When we activate both in the config,
> we end up with a mix of both, but not exactly the same data. Any help would
> be greatly appreciated - thanks!
> 
> 
> Config:
> 
> daemonize: false
> nfacctd_port: 2100
> nfacctd_net: netflow
> plugins: amqp[xv_ext_in], amqp[xv_ext_out]
> !
> amqp_exchange[xv_ext_in]: netflow-in
> amqp_exchange_type[xv_ext_in]: direct
> amqp_host[xv_ext_in]: localhost
> amqp_refresh_time[xv_ext_in]: 5
> amqp_user[xv_ext_in]: username
> amqp_passwd[xv_ext_in]: password
> aggregate[xv_ext_in]: peer_src_ip, src_as, dst_as, src_host, dst_host,
> src_port, dst_port, in_iface, out_iface, proto, sampling_rate
> aggregate_filter[xv_ext_in]: not (src net (173.241.240.0/20 or 69.6.80.0/20
> or 199.26.53.0/24 or 209.182.128.0/19)) and not net 10.0.0.0/8
> amqp_routing_key[xv_ext_in]: xv_in
> !
> amqp_exchange[xv_ext_out]: netflow-out
> amqp_exchange_type[xv_ext_out]: direct
> amqp_host[xv_ext_out]: localhost
> amqp_refresh_time[xv_ext_out]: 5
> amqp_user[xv_ext_out]: username
> amqp_passwd[xv_ext_out]: password
> aggregate[xv_ext_out]: peer_src_ip, src_as, dst_as, src_host, dst_host,
> src_port, dst_port, in_iface, out_iface, proto, sampling_rate
> aggregate_filter[xv_ext_out]: not (dst net (173.241.240.0/20 or 69.6.80.0/20
> or 199.26.53.0/24 or 209.182.128.0/19)) and not net 10.0.0.0/8
> amqp_routing_key[xv_ext_out]: xv_out

> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists