Risto, Is there another method that you favor of watching dated logfiles?
Regards, Tim Peiffer On 11/26/10 4:34 PM, Risto Vaarandi wrote: > Tim, > that's a good question. In fact, SEC uses the stat(2) system call for > checking the attributes of the log file, and the check is applied both > for the open file descriptor and file name. Normally, those two checks > return identical results, but there are some special cases. First, > suppose the file that SEC has opened in the past is deleted. In that > case, the stat() call applied for the open file handle will return > file attributes, but the stat() call for file name will fail. > Furthermore, if the deleted file is recreated, two stat() calls will > return different attributes (most notably different i-node numbers > which uniquely identify each file within the given file system). > Those differences enable SEC to detect if the file which is currently > open has been removed, or if this open file has been recreated. In the > first case, SEC will keep the file open and wait for new data that are > still appended to the end of the nameless file (in fact, although the > file is not visible in the directory structure anymore, this can > easily happen if some process has the file still open for writing). > However, if the file is recreated in the directory structure, SEC will > immediately switch to new file instance, leaving the old (nameless) > file behind. > The symbolic link approach you are using works fine, because: > 1) the stat() call will always fetch the attributes of the file > pointed by the symbolic link, not the symbolic link itself (if you > want to specifically fetch the attributes of the link, lstat() has to > be used, but SEC uses stat() for all checks), > 2) if the symbolic link exists but the pointed file does not, SEC sees > it as if the log file has been removed and not recreated, > 3) if the file pointed by link is created, SEC will see it as a > recreation of the log file, and it will switch to new file, processing > it from the start. > Was I able to clarify the internals a bit? In short, when symbolic > link points to non-existent file, SEC keeps the log file for the > previous day open (although no bytes will be appended to this file > anymore), but when the log file for the new day appears, SEC will > immediately switch to this file and process it from the beginning. > kind regards, > risto > > 2010/11/26 Tim Peiffer<peif...@umn.edu>: >> I have a backup application that writes logs to file names that are dated.. >> e.g. logfile.MMDDYY , and I am looking for the best way of detecting the >> existence of the file, and then open the file from the beginning under SEC. >> The backup application can fire off nearly any time of day, so the file is >> not guaranteed to be there because of a variable backup window. The >> directory that the logfile is written to is also read-only to my correlator >> process. What are some methods of elegantly handling these logs? >> >> The -input=logfile* glob is ok, but is only useful on start/restart, with >> the new logs being effectively invisible until the next restart. I have >> tried dropping symbolic links in a directory that I have write access to >> linking the fixed target with the monitored file name logfile.MMDDYY. When >> the file is created by the application, SEC handles the new monitored file >> name as a log file shuffle. What I am thinking is that at midnight, I >> remove the existing link, and then re-link to the monitored file name. I >> wrote the below example, and it works as expected, but I am interested how >> SEC will handle the hours in between the time of symbolic link creation, and >> the time of the monitored file creation. >> >> type=Calendar >> time=* * * * * >> desc=create new logfile sym link >> action=spawn ( \ >> cd /Users/peiffer/sec-2.5.3/glob ; \ >> rm logfile ; \ >> ln -s logfile.`date +%m%d%y` logfile \ >> ) >> >> type=Single >> ptype=RegExp >> pattern=(.*) >> desc=new event >> action=logonly %s $1 >> >> Tim >> >> -- >> Tim Peiffer >> Network Support Engineer >> Office of Information Technology >> University of Minnesota/NorthernLights GigaPOP >> >> +1 612 626-7884 (desk) >> >> ------------------------------------------------------------------------------ >> Increase Visibility of Your 3D Game App& Earn a Chance To Win $500! >> Tap into the largest installed PC base& get more eyes on your game by >> optimizing for Intel(R) Graphics Technology. Get started today with the >> Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. >> http://p.sf.net/sfu/intelisp-dev2dev >> _______________________________________________ >> Simple-evcorr-users mailing list >> Simple-evcorr-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users >> >> -- Tim Peiffer Network Support Engineer Office of Information Technology University of Minnesota/NorthernLights GigaPOP +1 612 626-7884 (desk) ------------------------------------------------------------------------------ Increase Visibility of Your 3D Game App & Earn a Chance To Win $500! Tap into the largest installed PC base & get more eyes on your game by optimizing for Intel(R) Graphics Technology. Get started today with the Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. http://p.sf.net/sfu/intelisp-dev2dev _______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users