Hi Ed,
An interesting proposal. It may be an option to nfcapd to be enabled by user
request. Those using nfdump as standalone
tools could use this option. However, together with NfSen it will not work
unless NfSen gets patched too. If this makes
sense for NfSen is another question as this very single hour per year, which is
the subject of the topic, can not be
selected in the RRD graph anyway. So for those not wanting to loose this hour
still may consider to switch their systems
to GMT.
I may check this as an option for standalone nfdump and may contact you off
list.
Anyway thanks for the thoughts
- Peter
On 9/28/10 17:33, Ed Ravin wrote:
> On Tue, Sep 28, 2010 at 09:45:35AM +0200, Peter Haag wrote:
>> The most pragmatic approach is setting your server to GMT.
>> If you use nfdump as standalone tool, you may change the filename
>> after each cycle by using a shell command as -x '/bin/mv ....' to
>> whatever format you like. Together with NfSen set the system to GMT.
>
> GMT will not work for us - our server needs to be set to localtime
> for other services that it runs. It would also be a big inconvenience
> to have to translate the time every time we go looking through the
> flow data to look up a particular network event.
>
> As for post-processing the filename, I'm not happy with that either -
> I have some scripts that feed a flow file to a program on standard
> input, and the program never learns the filename.
>
> There's also the philosophical issue - I can't tell my boss with
> a straight face that this new software will work great except
> for 60 minutes a year when it loses data unless we do a clumsy
> workaround with the timezone or an external shell script to fix
> the filenames.
>
> Looking over the code, I see that the filenames are generated in
> nfcapd.c and sfcapd.c, and that's pretty straightforward to change.
> The new format of a filename could be <nfcapd.YYYYMMDDHHmmXZZZZ>
> where XZZZZ is a numeric offset from GMT with a leading + or -,
> like "+0100" or "-0400". That's more or less what flow-tools does.
> Since this could disrupt an existing installation, it should probably
> be linked to a new command-line option to turn it on, I see "-G" is
> available (mnemonic - "don't rely on GMT").
>
> In util.c, ParseTime and TimeString would need to be told about
> timezones. That seems simple as well. nfdump would then emit
> timezone data when it writes out the "Time window" information, I
> hope that change is small enough that it doesn't need to be made
> optional with a command line argument.
>
> Then there's expire.c - that's more complex. I'm not through
> reviewing the code yet but I'm hoping that parsing of first_timestring
> and last_timestring can be expanded to include the optional timezone,
> the functions ISO2UNIX() and UNIX2ISO() can be told to look for the
> optional timezone, and then everything will work as before.
>
> Are there any other programs beside nfexpire and nfcapd that parse the
> format of the stored flow filenames? Anything else I might have missed?
>
> If we can agree on what to do, I'll be happy to code the changes.
>
> Thanks,
>
> -- Ed
>
>
>> On 9/25/10 7:13, Ed Ravin wrote:
>>> I'm converting from flow-tools to nfdump and I noticed that one issue
>>> that the nfdump tools, in particular nfcapd, do not seem to have any
>>> time zone support.
>>>
>>> As discussed on this list back in June 2009:
>>>
>>>
>>> http://www.mail-archive.com/[email protected]/msg00240.html
>>>
>>> if you are using time-stamped filenames, nfcapd will overwrite its files
>>> when the local time "falls back" due to a time zone change. The flow-tools
>>> flow-capture program doesn't have this problem because it appends the
>>> timezone to the filename.
>>>
>>> I also noticed that nfdump, when it prints a time, does not show the
>>> timezone. This is a problem for me since I need to port my flow-tools
>>> output parsing scripts and they were looking for the timezone.
>>>
>>> How difficult would it be to fix these problems? As pointed out last
>>> June, there's no good way to use timestamped filenames and avoid file
>>> overwriting unless you keep your computer's time zone in GMT or turn
>>> of daylight time, which causes other problems.
>>>
>>> Thanks,
>>>
>>> -- Ed
>>>
>>> ------------------------------------------------------------------------------
>>> Start uncovering the many advantages of virtual appliances
>>> and start using them to simplify application deployment and
>>> accelerate your shift to cloud computing.
>>> http://p.sf.net/sfu/novell-sfdev2dev
>>> _______________________________________________
>>> Nfdump-discuss mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
>>
>> --
>> --
>> Be nice to your netflow data
>>
>
--
--
Be nice to your netflow data
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
Nfdump-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
