Which plugin would do that? I'd be interested in saving AS info so at the end of the month I could see what the top 100 AS we exchange traffic with, possibly bgp-next-hop as well.
On 2013-01-14, at 10:31 AM, Adrian Popa <adrian.popa...@gmail.com> wrote: > It depends. You can still process your data faster if you are not interested > in EVERYTHING at once. You could save what you're interested in every 5 > minutes (via a plugin) to an external database and build more elaborate > queries in the database. This way you won't have to plow through 100s of > gigabytes of data when you want to extract only something small. > > For instance, you could query total traffic every 5 minutes with "in if 5 and > dst as 20" (which would be the data of interest for you), and save this total > someplace else. You could then use the data faster in other queries. > > The possibilities are limitless, depends only what you want to do. > > > On Mon, Jan 14, 2013 at 5:20 PM, Jason Lixfeld > <jason-nfsen-disc...@lixfeld.ca> wrote: > Yikes. So processing a months worth of data could take hours and hours and > hours. Sounds like it might not be the best tool for traffic engineering > type business where large data sets would need to get crunched? > > On 2013-01-14, at 10:16 AM, Adrian Popa <adrian.popa...@gmail.com> wrote: > > > Reports are run against the collected data (nfdump files). The larger the > > time interval, the more files nfdump has to process. For my system, I > > collect about 4 days worth of netflow data, and they take up ~170GB of disk > > space. If I run a top x report on the whole interval, it will take up to an > > hour to process. > > > > Sadly, during this time nfdump is not returning any data (since it is still > > chewing on the problem), so your browser/firewall/proxy might conclude that > > the host is not responding and reset the session. > > > > > > On Mon, Jan 14, 2013 at 5:04 PM, Jason Lixfeld > > <jason-nfsen-disc...@lixfeld.ca> wrote: > > > > On 2013-01-10, at 3:08 PM, Peter Haag <ph...@users.sourceforge.net> wrote: > > > > > On 10/1/13 4:40 PM, Jason Lixfeld wrote: > > >> While trying to chew through anything more than a few days worth of > > >> data, I'm faced with a 504 Gateway Time-out error. Is that something > > >> that can be fixed within Nfsen or is that a web server thing? > > > > > > It's most likely a php timeout thing. If you do such overlength queries > > > over the web interface, it might be a good idea > > > to increase this time in php.ini. > > > > Indeed. FastCGI somethingorother. > > > > So it takes 4 minutes to process about a weeks worth of data meaning it > > could take about 15-20 minutes to process a months worth of data. This > > seems like a long period of time to me. Is this normal or what factors > > would affect nfdump's timeliness in processing files? > > ------------------------------------------------------------------------------ > > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, > > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current > > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft > > MVPs and experts. SALE $99.99 this month only -- learn more at: > > http://p.sf.net/sfu/learnmore_122412 > > _______________________________________________ > > Nfsen-discuss mailing list > > Nfsen-discuss@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/nfsen-discuss > > > > ------------------------------------------------------------------------------ Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122412 _______________________________________________ Nfsen-discuss mailing list Nfsen-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
