Let me check with my co-worker who wrote a nifty netflow-to-syslog utility in C. Maybe we can share it as open source.
On Fri, Jan 10, 2014 at 1:20 PM, David Lang <[email protected]> wrote: > what sort of throughput can you get from logstash getting netflow logs and > delivering them to rsyslog? > > David Lang > > On Fri, 10 Jan 2014, Mike Hoskins (michoski) wrote: > > Date: Fri, 10 Jan 2014 20:58:03 +0000 >> From: "Mike Hoskins (michoski)" <[email protected]> >> Reply-To: rsyslog-users <[email protected]> >> >> To: rsyslog-users <[email protected]> >> Subject: Re: [rsyslog] Off-Topic: rsyslog-like equivalent for NetFlow? >> >> Logstash setup itself is straightforward (their docs are great), and I can >> attach the full config referenced below + patterns file specific to Cisco, >> minus my IPs and rabbitmq passwords of course...if that's helpful. ;-) >> Nothing too exotic really. >> >> Right now I've got netflow in each colo going through logstash -> rabbitmq >> <- central rabbitmq -> elasticsearch -> kibana to make infosec happy. The >> bulk of the work is on es/kibana side to make pretty dashboards people >> like, though they can tweak quite a bit themselves. >> >> I actually use rsyslog for an entirely different use case (high volume >> application logs), but was thinking the above could be >> modified...inserting rsyslog in the middle so you could output/archive to >> flat file as well as es. That way people who prefer traditional methods >> like grep aren't left in the cold. Elasticsearch is amazing, but this >> would give users a choice of interface. >> >> -----Original Message----- >> From: Nick Syslog <[email protected]> >> Reply-To: rsyslog-users <[email protected]> >> Date: Friday, January 10, 2014 2:34 PM >> To: rsyslog-users <[email protected]> >> Subject: Re: [rsyslog] Off-Topic: rsyslog-like equivalent for NetFlow? >> >> I'm also interested in this solution as I'm about to implement something >>> similar in our enterprise as well... >>> >>> Either that or work on paying to develop something native to rsyslog to >>> accept the traffic and redistribute it. >>> >>> >>> On Fri, Jan 10, 2014 at 11:51 AM, Mike Hoskins (michoski) < >>> [email protected]> wrote: >>> >>> Still working out all the details, but have had luck using logstash >>>> behind >>>> lb to accept netflow inpup, then filter/output as desired...even back >>>> into >>>> rsyslog. ;-) >>>> >>>> input { >>>> >>>> # Syslog inputs >>>> udp { >>>> host => "a.b.c.d" >>>> port => 514 >>>> type => "syslog" >>>> } >>>> tcp { >>>> host => "a.b.c.d" >>>> port => 514 >>>> type => "syslog" >>>> } >>>> >>>> # Netflow input >>>> udp { >>>> host => "a.b.c.d" >>>> codec => netflow {} >>>> port => 2055 >>>> type => "netflow" >>>> } >>>> >>>> # Dummy TCP ports for load balancer probes >>>> tcp { >>>> host => "a.b.c.d" >>>> port => 514 >>>> type => "dummy" >>>> } >>>> tcp { >>>> host => "a.b.c.d" >>>> port => 2055 >>>> type => "dummy" >>>> } >>>> } >>>> >>>> >>>> Last tcp bits being a hack to keep random garbage showing up from lb >>>> probes (my filters drop type dummy). >>>> >>>> -----Original Message----- >>>> From: Robert McIntyre <[email protected]> >>>> Reply-To: rsyslog-users <[email protected]> >>>> Date: Friday, January 10, 2014 1:36 PM >>>> To: "[email protected]" <[email protected]> >>>> Subject: [rsyslog] Off-Topic: rsyslog-like equivalent for NetFlow? >>>> >>>> Hello, folks! Apologies for this question; I know that it's off-topic, >>>>> but hope that it's not too far off. :) >>>>> >>>>> I have an infrastructure using rsyslog to receive, write to text file, >>>>> and forward syslog traffic. I now need to figure out a way to do the >>>>> same things with NetFlow data. I'm querying the internet, but haven't >>>>> found anything as turnkey as rsyslog is for syslog. >>>>> >>>>> Any suggestions? >>>>> >>>>> Thanks! >>>>> Robert >>>>> >>>>> >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> >>>> myriad >>>> >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
