Yes there are ways to build something yourselves …. some open source options that may fit your needs….
One thing to note with traditional net flow is to set a reasonable sampling rate - this impacts the flows per minute that will hit your collector platform. I’ve seen lots of folks use sampling like 1:10 and ultimately kill their platform with the load - more commonly is 1:100 or 1:1000 sampling rates… > On Feb 14, 2017, at 9:45 AM, Travis Johnson <[email protected]> wrote: > > Hi, > > This would have been about 5-6 years ago, but we found a free PHP based > Netflow analysis program that run under Linux. We ran that on a high-end PC > based system we build (i7 processor with 16GB of RAM at the time) and it was > able to handle over 1Gbps of traffic. The user interface was a little rough, > but it provided what we needed at the time... mainly tracking down infected > and high-usage customers and traffic patterns. > > Travis > > > On 2/14/2017 4:08 AM, Paul Stewart wrote: >> I don’t know which one has longer data retention … Arbor is at least a year. >> However, most products in this space will start summarizing the data after >> a certain point in time so understanding how long the data is stored for may >> be of importantance but also understanding the level of that detailed data >> may be important as well. >> >> For us, history is nice to have to check back over time for recurring >> patterns and stuff but not something we use a lot of … past 30-60 days most >> often … going back a year ago typically don’t care much about. >> >> I didn’t spend a lot of time looking at their solution and yes they might >> have an offering worth looking into (not sure) … I like Arbor best for >> features, scaling, and integration with DDOS mitigation. >> >> Attached picture is one of our Arbor systems … top box is Peakflow SP which >> does the flow analysis/reporting for 20 core routers, bottom box is a threat >> mitigation box that does surgical traffic scrubbing of dirty traffic and can >> handle 100G of attack traffic. >> >> >> >>> On Feb 7, 2017, at 12:13 PM, Mike Hammett <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Best in what way? It sounds like Kentik has a longer retention policy than >>> Arbor, which would explain the higher space requirements. >>> >>> >>> So are you saying it may be worth a small shop asking about pricing? >>> >>> >>> >>> ----- >>> Mike Hammett >>> Intelligent Computing Solutions <http://www.ics-il.com/> >>> <https://www.facebook.com/ICSIL> >>> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> >>> <https://www.linkedin.com/company/intelligent-computing-solutions> >>> <https://twitter.com/ICSIL> >>> Midwest Internet Exchange <http://www.midwest-ix.com/> >>> <https://www.facebook.com/mdwestix> >>> <https://www.linkedin.com/company/midwest-internet-exchange> >>> <https://twitter.com/mdwestix> >>> The Brothers WISP <http://www.thebrotherswisp.com/> >>> <https://www.facebook.com/thebrotherswisp> >>> >>> >>> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> >>> From: "Paul Stewart" <[email protected] <mailto:[email protected]>> >>> To: [email protected] <mailto:[email protected]> >>> Sent: Tuesday, February 7, 2017 9:51:38 AM >>> Subject: Re: [AFMUG] Netflow >>> >>> Depends on flow volumes and stuff.. talked to them at NANOG and conference >>> calls … >>> >>> For a low volume shop they seem to have a slick solution - only seen a >>> brief demo. However, depending on volume they do not scale “well” - we >>> were told that we would need several racks of servers to deal with volume :( >>> >>> Arbor Peakflow is the best product out there hands down … but it’s well >>> into 6 figures so your budget may not support it …. >>> >>> On Feb 6, 2017, at 9:05 PM, Mike Hammett <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> I haven't received a quote myself, but I hear it's a few hundred a month. >>> >>> >>> >>> ----- >>> Mike Hammett >>> Intelligent Computing Solutions <http://www.ics-il.com/> >>> <https://www.facebook.com/ICSIL> >>> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> >>> <https://www.linkedin.com/company/intelligent-computing-solutions> >>> <https://twitter.com/ICSIL> >>> Midwest Internet Exchange <http://www.midwest-ix.com/> >>> <https://www.facebook.com/mdwestix> >>> <https://www.linkedin.com/company/midwest-internet-exchange> >>> <https://twitter.com/mdwestix> >>> The Brothers WISP <http://www.thebrotherswisp.com/> >>> <https://www.facebook.com/thebrotherswisp> >>> >>> >>> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> >>> From: "Cassidy B. Larson" <[email protected] <mailto:[email protected]>> >>> To: [email protected] <mailto:[email protected]> >>> Sent: Monday, February 6, 2017 8:04:14 PM >>> Subject: Re: [AFMUG] Netflow >>> >>> How much? >>> >>> >>> >>> On Feb 6, 2017, at 7:00 PM, Mike Hammett <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Kentik is the cat's ass, though it's not a few bucks a month. >>> >>> >>> >>> ----- >>> Mike Hammett >>> Intelligent Computing Solutions <http://www.ics-il.com/> >>> <https://www.facebook.com/ICSIL> >>> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> >>> <https://www.linkedin.com/company/intelligent-computing-solutions> >>> <https://twitter.com/ICSIL> >>> Midwest Internet Exchange <http://www.midwest-ix.com/> >>> <https://www.facebook.com/mdwestix> >>> <https://www.linkedin.com/company/midwest-internet-exchange> >>> <https://twitter.com/mdwestix> >>> The Brothers WISP <http://www.thebrotherswisp.com/> >>> <https://www.facebook.com/thebrotherswisp> >>> >>> >>> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> >>> From: "Sterling Jacobson" <[email protected] >>> <mailto:[email protected]>> >>> To: "[email protected] <mailto:[email protected]>" <[email protected] >>> <mailto:[email protected]>> >>> Sent: Monday, February 6, 2017 7:38:27 PM >>> Subject: [AFMUG] Netflow >>> >>> What are your opinions on Netflow servers/software? >>> >>> I've been doing some research into using Netflow again. >>> Long time ago I used NTOP, but it sucked. >>> Not sure if that's changed or not. >>> >>> Ideally would be a much newer improved interface type system that was >>> hosted for a few bucks a month. >>> Then I could just sign up and point my Netflow streams to it. >>> >>> I need one that is geared towards ISPs, not Datacenter/Servers. >>> >>> I don't care about netflowing and optimizing web sites, I want to profile >>> my customer traffic. >>> Ideally it would include features necessary for CALIA and law enforcement >>> requirements. >>> >>> If it was also great at syslog management that would be a plus. >>> >>> The Dude currently sucks for syslog IMO. >> >
