In this context I am also curious, what would be the effect of using --check-timeout / --poll-timeout, if the log file will be closed or remain open during timeout... I am trying to find a way, how to use SEC in "close after read" mode - used to use this mode in previous log event correlation solution, because keeping log files "always open" causes described problem with their deletion (by external archivation script) on NFS...
>From SEC manual: "Each input file is tracked both by its name and i-node, and input file rotations are handled seamlessly. If the input file is recreated or truncated, SEC will reopen it and process its content from the beginning. If the input file is removed (i.e., there is just an i-node left without a name), SEC will keep the i-node open and wait for the input file recreation." Maybe it would be sufficient having an option to (immediately?) close (re)moved file, instead of keeping original i-node open until its recreation in its original location. pi 7. 2. 2020 o 12:38 Richard Ostrochovský <richard.ostrochov...@gmail.com> napísal(a): > hi Risto, > > thank you for your helpful explanation about inner functionality of SEC. > Closed files in my case were not existing, that was the reason. > > Richard > > ut 4. 2. 2020 o 23:09 Risto Vaarandi <risto.vaara...@gmail.com> > napísal(a): > >> hi Richard, >> >> I have never used SEC for monitoring files on NFS file systems, but I can >> provide few short comments on how input files are handled. After SEC has >> successfully opened an input file, it will be kept open permanently. When >> input file is removed or renamed, input file is still kept open (at that >> point the file handle is pointing to disk blocks that no longer have a >> name). When a new file appears on disk with the same name as original input >> file, SEC will close the file handle pointing to nameless disk blocks and >> will open the new input file, starting to process it from the beginning. >> However, this operation is atomic and the input file will never show as >> "Closed" in the dump file. Status "Closed" in dump file usually indicates >> that SEC was not able to open the input file when it was started or >> restarted with a signal (a common situation when file does not exist or >> there is no permission to open the file), but it can also indicate that SEC >> has closed the file due to IO error when reading from file (that should >> normally not happen and usually means a serious disk issue). >> >> In order to find out why input file is in closed state, I would recommend >> to check the log of SEC (you can set it up with --log command line option). >> For example, if the input file did not exist when SEC was started or >> restarted with a signal, there should be the following lines in the log: >> >> Opening input file test.log >> Input file test.log does not exist! >> >> Also, if file is closed due to IO error, there is a relevant message in >> the log, for example: Input file test.log IO error (Input/output error), >> closing the file. >> >> Hope these comments are helpful, >> risto >> >> >> Hi Risto and friends, >>> >>> I am unsure about one conceptual question about how SEC keeps open >>> monitored files. >>> >>> Using SEC as systemd service, when files stored in NFS (opened via >>> addinput) being watched by SEC are moved elsewhere, and then their removal >>> is tried, NFS persistently keeps .nfsNUMBER files in existence, unable to >>> remove whole folder of them. This functionality of NFS is described e.g. >>> here: >>> https://www.ibm.com/su >>> pport/pages/what-are-nfs-files-accumulate-and-why-cant-they-be-deleted-even-after-stopping-cognos-8 >>> <https://www.ibm.com/support/pages/what-are-nfs-files-accumulate-and-why-cant-they-be-deleted-even-after-stopping-cognos-8> >>> >>> >>> It seems, that SEC running as service (not re-opening and closing each >>> monitored file in each "run") is keeping watched files permanently open, >>> and it is not desirable in such setup. >>> >>> But: when I look into the dump, some files have "status: Open" and some >>> "status: Closed", so maybe this my assumption about log files permanently >>> opened by SEC is not correct - I am confused. >>> >>> How it is in reality? Has somebody experience with described behaviour >>> (SEC+NFS, but it could arise also in other setups) and how to overcome it? >>> >>> Thank you. >>> >>> Richard >>> _______________________________________________ >>> Simple-evcorr-users mailing list >>> Simple-evcorr-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users >>> >>
_______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
