Risto,
Is there another method that you favor of watching dated logfiles?
Regards,
Tim Peiffer
On 11/26/10 4:34 PM, Risto Vaarandi wrote:
> Tim,
> that's a good question. In fact, SEC uses the stat(2) system call for
> checking the attributes of the log file, and the check is applied both
> for the open file descriptor and file name. Normally, those two checks
> return identical results, but there are some special cases. First,
> suppose the file that SEC has opened in the past is deleted. In that
> case, the stat() call applied for the open file handle will return
> file attributes, but the stat() call for file name will fail.
> Furthermore, if the deleted file is recreated, two stat() calls will
> return different attributes (most notably different i-node numbers
> which uniquely identify each file within the given file system).
> Those differences enable SEC to detect if the file which is currently
> open has been removed, or if this open file has been recreated. In the
> first case, SEC will keep the file open and wait for new data that are
> still appended to the end of the nameless file (in fact, although the
> file is not visible in the directory structure anymore, this can
> easily happen if some process has the file still open for writing).
> However, if the file is recreated in the directory structure, SEC will
> immediately switch to new file instance, leaving the old (nameless)
> file behind.
> The symbolic link approach you are using works fine, because:
> 1) the stat() call will always fetch the attributes of the file
> pointed by the symbolic link, not the symbolic link itself (if you
> want to specifically fetch the attributes of the link, lstat() has to
> be used, but SEC uses stat() for all checks),
> 2) if the symbolic link exists but the pointed file does not, SEC sees
> it as if the log file has been removed and not recreated,
> 3) if the file pointed by link is created, SEC will see it as a
> recreation of the log file, and it will switch to new file, processing
> it from the start.
> Was I able to clarify the internals a bit? In short, when symbolic
> link points to non-existent file, SEC keeps the log file for the
> previous day open (although no bytes will be appended to this file
> anymore), but when the log file for the new day appears, SEC will
> immediately switch to this file and process it from the beginning.
> kind regards,
> risto
>
> 2010/11/26 Tim Peiffer<peif...@umn.edu>:
>> I have a backup application that writes logs to file names that are dated..
>> e.g. logfile.MMDDYY , and I am looking for the best way of detecting the
>> existence of the file, and then open the file from the beginning under SEC.
>> The backup application can fire off nearly any time of day, so the file is
>> not guaranteed to be there because of a variable backup window. The
>> directory that the logfile is written to is also read-only to my correlator
>> process. What are some methods of elegantly handling these logs?
>>
>> The -input=logfile* glob is ok, but is only useful on start/restart, with
>> the new logs being effectively invisible until the next restart. I have
>> tried dropping symbolic links in a directory that I have write access to
>> linking the fixed target with the monitored file name logfile.MMDDYY. When
>> the file is created by the application, SEC handles the new monitored file
>> name as a log file shuffle. What I am thinking is that at midnight, I
>> remove the existing link, and then re-link to the monitored file name. I
>> wrote the below example, and it works as expected, but I am interested how
>> SEC will handle the hours in between the time of symbolic link creation, and
>> the time of the monitored file creation.
>>
>> type=Calendar
>> time=* * * * *
>> desc=create new logfile sym link
>> action=spawn ( \
>> cd /Users/peiffer/sec-2.5.3/glob ; \
>> rm logfile ; \
>> ln -s logfile.`date +%m%d%y` logfile \
>> )
>>
>> type=Single
>> ptype=RegExp
>> pattern=(.*)
>> desc=new event
>> action=logonly %s $1
>>
>> Tim
>>
>> --
>> Tim Peiffer
>> Network Support Engineer
>> Office of Information Technology
>> University of Minnesota/NorthernLights GigaPOP
>>
>> +1 612 626-7884 (desk)
>>
>> ------------------------------------------------------------------------------
>> Increase Visibility of Your 3D Game App& Earn a Chance To Win $500!
>> Tap into the largest installed PC base& get more eyes on your game by
>> optimizing for Intel(R) Graphics Technology. Get started today with the
>> Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
>> http://p.sf.net/sfu/intelisp-dev2dev
>> _______________________________________________
>> Simple-evcorr-users mailing list
>> Simple-evcorr-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>>
>>
--
Tim Peiffer
Network Support Engineer
Office of Information Technology
University of Minnesota/NorthernLights GigaPOP
+1 612 626-7884 (desk)
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
