On 29/6/2010 1:41 PM, Kostka Bořivoj wrote: > I agree it works fine (and your way of nulling is definitelly better than > mine). > I already indexed about 1GB of data, but I'm not sure about mem leaks, > as my application memory increases constantly during indexing (and it didn't > with previous > version). But this could be of course bug in my app, or just becase the size > of internal > "caches" increases or something else... > This is why I said using cl_demo and cl_test as a testbed to look for crashes and memory leaks is better.
This also depends on how you measure the memory usage for your application. Windows' Task manager isn't reliable for that. > Before you mar thi solved, I'will try to explain, what I meant by duplicate > pointers etc. In one of my previous mails, > just for case you find this important. If this is problem or not strongly > depends on how postingsFreeListDW should be used. > If only items from 0 to postingsFreeCountDW are considered valid and the rest > is just "junk" everything is OK. > But if postingsFreeListDW is supposed to be used beyond the > postingsFreeCountDW (eg. for freeing postings objects during destruction), we > can run into the problems. > > ... > > The question is if > > (1) the tail always contains junk during destruction (and everything is OK, > if yes) or > > (2) it can also contain valid pointers which should be deleted (so memory > leaks are produced by tail nulling) > > Unfortunatelly until now I was not able figure out if (1) or (2) is true. > To correctly answer that, I need to profile the code which is something I cannot do at the moment. I am not the original author of the code - neither in its Java nor C++ shape. If you want to do this, use infoStream for easier tracking. However, I think the way it operates is designed not to have such a scenario. Although Java won't suffer from double-deletions, that class was designed not to hold unused objects when they aren't needed - which is essentially what you are describing here. The answer to your questions will be in the actual place where copying is performed. See DocumentsWriter::getPostings. I think the way copy operations are performed there prevents this from happening. Fact is on destruction all valid objects are in the head of the array, and the tail contains nothing important. That is at least from the use cases in cl_demo and cl_test. The best way to either confirm or rule this is by thoroughly testing this - while both looking for crashes and mem-leaks in various scenarios (multi-threaded?). If you wish, you could post such a question on Java Lucene's dev list. They're sure to answer this correctly, if they'll remember what they did there back in '07 or so :) HTH Itamar ------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first _______________________________________________ CLucene-developers mailing list CLucene-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/clucene-developers
