Tim, that's a good question. In fact, SEC uses the stat(2) system call for checking the attributes of the log file, and the check is applied both for the open file descriptor and file name. Normally, those two checks return identical results, but there are some special cases. First, suppose the file that SEC has opened in the past is deleted. In that case, the stat() call applied for the open file handle will return file attributes, but the stat() call for file name will fail. Furthermore, if the deleted file is recreated, two stat() calls will return different attributes (most notably different i-node numbers which uniquely identify each file within the given file system). Those differences enable SEC to detect if the file which is currently open has been removed, or if this open file has been recreated. In the first case, SEC will keep the file open and wait for new data that are still appended to the end of the nameless file (in fact, although the file is not visible in the directory structure anymore, this can easily happen if some process has the file still open for writing). However, if the file is recreated in the directory structure, SEC will immediately switch to new file instance, leaving the old (nameless) file behind. The symbolic link approach you are using works fine, because: 1) the stat() call will always fetch the attributes of the file pointed by the symbolic link, not the symbolic link itself (if you want to specifically fetch the attributes of the link, lstat() has to be used, but SEC uses stat() for all checks), 2) if the symbolic link exists but the pointed file does not, SEC sees it as if the log file has been removed and not recreated, 3) if the file pointed by link is created, SEC will see it as a recreation of the log file, and it will switch to new file, processing it from the start. Was I able to clarify the internals a bit? In short, when symbolic link points to non-existent file, SEC keeps the log file for the previous day open (although no bytes will be appended to this file anymore), but when the log file for the new day appears, SEC will immediately switch to this file and process it from the beginning. kind regards, risto
2010/11/26 Tim Peiffer <peif...@umn.edu>: > I have a backup application that writes logs to file names that are dated.. > e.g. logfile.MMDDYY , and I am looking for the best way of detecting the > existence of the file, and then open the file from the beginning under SEC. > The backup application can fire off nearly any time of day, so the file is > not guaranteed to be there because of a variable backup window. The > directory that the logfile is written to is also read-only to my correlator > process. What are some methods of elegantly handling these logs? > > The -input=logfile* glob is ok, but is only useful on start/restart, with > the new logs being effectively invisible until the next restart. I have > tried dropping symbolic links in a directory that I have write access to > linking the fixed target with the monitored file name logfile.MMDDYY. When > the file is created by the application, SEC handles the new monitored file > name as a log file shuffle. What I am thinking is that at midnight, I > remove the existing link, and then re-link to the monitored file name. I > wrote the below example, and it works as expected, but I am interested how > SEC will handle the hours in between the time of symbolic link creation, and > the time of the monitored file creation. > > type=Calendar > time=* * * * * > desc=create new logfile sym link > action=spawn ( \ > cd /Users/peiffer/sec-2.5.3/glob ; \ > rm logfile ; \ > ln -s logfile.`date +%m%d%y` logfile \ > ) > > type=Single > ptype=RegExp > pattern=(.*) > desc=new event > action=logonly %s $1 > > Tim > > -- > Tim Peiffer > Network Support Engineer > Office of Information Technology > University of Minnesota/NorthernLights GigaPOP > > +1 612 626-7884 (desk) > > ------------------------------------------------------------------------------ > Increase Visibility of Your 3D Game App & Earn a Chance To Win $500! > Tap into the largest installed PC base & get more eyes on your game by > optimizing for Intel(R) Graphics Technology. Get started today with the > Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. > http://p.sf.net/sfu/intelisp-dev2dev > _______________________________________________ > Simple-evcorr-users mailing list > Simple-evcorr-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users > > ------------------------------------------------------------------------------ Increase Visibility of Your 3D Game App & Earn a Chance To Win $500! Tap into the largest installed PC base & get more eyes on your game by optimizing for Intel(R) Graphics Technology. Get started today with the Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. http://p.sf.net/sfu/intelisp-dev2dev _______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
