In message <cagfjscobvfj6lfwbnjjmqsdiuildzsceds4ygmgt_omlh1m...@mail.gmail.com> , Risto Vaarandi writes:
>2014-05-26 19:06 GMT+03:00 Natalia Iglesias ><nigles...@lookwisesolutions.com>>: >> We think we have found out the reason of the memory leak, there is >> an external PERL library being called that contains a call to the >> IdentifyFile function to obtain the inode of windows >> files. Whenever this function is called, a memory leak occurs. If >> we disable this call, the memory usage is stable. I assume you mean: Win32::IdentifyFile from: http://search.cpan.org/~clive/Win32-IdentifyFile-1.01/lib/Win32/IdentifyFile.pm >> So we are thinking of a way to bypass this problem, either by using >> another function or assessing if it's still needed. For that we >> would like to ask you a couple of questions about SEC internal >> functioning when reading (Windows) files from the file system: >> >> 1. Does SEC detect that an open file is idle, without any writing >> activity? And if it does, does SEC close the file after a while, or does >> the file remain open until it's explicitly closed by a process? >> > >Do you mean input files which are specified with --input command line >option(s)? If so, they are never closed by SEC (unless a critical IO error >is observed). While keeping the input file open, each input file is >frequently checked for new data, and if the check was not successful, SEC >also checks whether the file has been truncated or recreated (the latter >check is relevant to UNIX only, since UNIX filesystems allow for recreating >an open file, so that the open file would lose its name in the directory >tree). If the file truncation or recreation is observed, SEC will reopen >the file. Can you tell if Win32::IdentifyFile is being called as a result of a stat() on the input file? The web page I referenced aabove says: It is important to call CloseIdentifyFile() in order to avoid a resource leak. Only one call is required, regardless of the number of times IdentifyFile() is called. If you can tell when IdentifyFile is being called, maybe you can manually run the CloseIdentifyFile with a perl sub every now and again (or worst case after every line). If Strawberry Perl is using that library internally as part of it's stat() call or something, I would report this as a bug as Strawberry perl shouldn't be leaking memory. Another possibility is to use something like cygwin perl to run SEC, although it will probably be a slower than a native windows Perl. -- -- rouilj John Rouillard =========================================================================== My employers don't acknowledge my existence much less my opinions. ------------------------------------------------------------------------------ The best possible search technologies are now affordable for all companies. Download your FREE open source Enterprise Search Engine today! Our experts will assist you in its installation for $59/mo, no commitment. Test it for FREE on our Cloud platform anytime! http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk _______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
