If you give, an overview what you want to achieve, it could be useful for
others too. So writing a plugin could be done.
Be as precise as you can.
- Peter
On 14/1/13 9:45 PM, Jason Lixfeld wrote:
> Welp, I'm just a dumb router jockey, so perl, databases and shell scripts are
> a little beyond me, I'm afraid :(
>
> On 2013-01-14, at 3:18 PM, Adrian Popa <adrian.popa...@gmail.com> wrote:
>
>> Well, I don't know if there is a specific plugin that does this already, but
>> with a little perl knowledge you can write your own plugin. Otherwise, you
>> can achieve the same result with cron and shell scripts to collect and parse
>> the data...
>>
>> But it's like a DIY kit - some assembly required :)
>>
>>
>> On Mon, Jan 14, 2013 at 10:05 PM, Jason Lixfeld
>> <jason-nfsen-disc...@lixfeld.ca> wrote:
>> Which plugin would do that? I'd be interested in saving AS info so at the
>> end of the month I could see what the top 100 AS we exchange traffic with,
>> possibly bgp-next-hop as well.
>>
>> On 2013-01-14, at 10:31 AM, Adrian Popa <adrian.popa...@gmail.com> wrote:
>>
>>> It depends. You can still process your data faster if you are not
>>> interested in EVERYTHING at once. You could save what you're interested in
>>> every 5 minutes (via a plugin) to an external database and build more
>>> elaborate queries in the database. This way you won't have to plow through
>>> 100s of gigabytes of data when you want to extract only something small.
>>>
>>> For instance, you could query total traffic every 5 minutes with "in if 5
>>> and dst as 20" (which would be the data of interest for you), and save this
>>> total someplace else. You could then use the data faster in other queries.
>>>
>>> The possibilities are limitless, depends only what you want to do.
>>>
>>>
>>> On Mon, Jan 14, 2013 at 5:20 PM, Jason Lixfeld
>>> <jason-nfsen-disc...@lixfeld.ca> wrote:
>>> Yikes. So processing a months worth of data could take hours and hours and
>>> hours. Sounds like it might not be the best tool for traffic engineering
>>> type business where large data sets would need to get crunched?
>>>
>>> On 2013-01-14, at 10:16 AM, Adrian Popa <adrian.popa...@gmail.com> wrote:
>>>
>>>> Reports are run against the collected data (nfdump files). The larger the
>>>> time interval, the more files nfdump has to process. For my system, I
>>>> collect about 4 days worth of netflow data, and they take up ~170GB of
>>>> disk space. If I run a top x report on the whole interval, it will take up
>>>> to an hour to process.
>>>>
>>>> Sadly, during this time nfdump is not returning any data (since it is
>>>> still chewing on the problem), so your browser/firewall/proxy might
>>>> conclude that the host is not responding and reset the session.
>>>>
>>>>
>>>> On Mon, Jan 14, 2013 at 5:04 PM, Jason Lixfeld
>>>> <jason-nfsen-disc...@lixfeld.ca> wrote:
>>>>
>>>> On 2013-01-10, at 3:08 PM, Peter Haag <ph...@users.sourceforge.net> wrote:
>>>>
>>>>> On 10/1/13 4:40 PM, Jason Lixfeld wrote:
>>>>>> While trying to chew through anything more than a few days worth of
>>>>>> data, I'm faced with a 504 Gateway Time-out error. Is that something
>>>>>> that can be fixed within Nfsen or is that a web server thing?
>>>>>
>>>>> It's most likely a php timeout thing. If you do such overlength queries
>>>>> over the web interface, it might be a good idea
>>>>> to increase this time in php.ini.
>>>>
>>>> Indeed. FastCGI somethingorother.
>>>>
>>>> So it takes 4 minutes to process about a weeks worth of data meaning it
>>>> could take about 15-20 minutes to process a months worth of data. This
>>>> seems like a long period of time to me. Is this normal or what factors
>>>> would affect nfdump's timeliness in processing files?
>>>> ------------------------------------------------------------------------------
>>>> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
>>>> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
>>>> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
>>>> MVPs and experts. SALE $99.99 this month only -- learn more at:
>>>> http://p.sf.net/sfu/learnmore_122412
>>>> _______________________________________________
>>>> Nfsen-discuss mailing list
>>>> Nfsen-discuss@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>>>>
>>>
>>>
>>
>>
>
--
Be nice to your netflow data. Use NfSen and nfdump :)
------------------------------------------------------------------------------
Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
and much more. Keep your Java skills current with LearnJavaNow -
200+ hours of step-by-step video tutorials by Java experts.
SALE $49.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122612
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
