I agree it works fine (and your way of nulling is definitelly better than mine).
I already indexed about 1GB of data, but I'm not sure about mem leaks,
as my application memory increases constantly during indexing (and it didn't
with previous
version). But this could be of course bug in my app, or just becase the size of
internal
"caches" increases or something else...
Before you mar thi solved, I'will try to explain, what I meant by duplicate
pointers etc. In one of my previous mails,
just for case you find this important. If this is problem or not strongly
depends on how postingsFreeListDW should be used.
If only items from 0 to postingsFreeCountDW are considered valid and the rest
is just "junk" everything is OK.
But if postingsFreeListDW is supposed to be used beyond the postingsFreeCountDW
(eg. for freeing postings objects during destruction), we can run into the
problems.
Consider following scenario:
1. We have postingsFreeListDW with 1024 items alocated, 256 of them used, 768
available
2. Another 256 items are requested by getPostings call, so this is changed now
512 available/512 used
3. balanceRAM is called and 256 items are deleted (at positions 256-511) and
postingsFreeCountDW
is set 256
4. now all used postings previously taken by getPostings are returned using
recyclePostings. Pointers are
written back to positions 256-767, but same pointers (some of them)
positions are also in the tail
of postingsFreeListDW. So if you delete postingsFreeListDW now, you must
get double deletion
because pointer to same object is e.g. on position 511 (valid one) and 1023
(invalid one).
5. if you now call balanceRAM again to free another 256, some of pointers in
invalid tail can point
to deleted memory (this is probably reason of crash fixed by your patch)
The question is if
(1) the tail always contains junk during destruction (and everything is OK, if
yes) or
(2) it can also contain valid pointers which should be deleted (so memory leaks
are produced by tail nulling)
Unfortunatelly until now I was not able figure out if (1) or (2) is true.
Borek
> -----Original Message-----
> From: Itamar Syn-Hershko [mailto:ita...@code972.com]
> Sent: Tuesday, June 29, 2010 12:32 PM
> To: clucene-developers@lists.sourceforge.net
> Subject: Re: [CLucene-dev] vector subscript outofrangeexceptionduringindexing
>
> I tested it a bit, and overall it seems to work just fine. I checked in
> my changes into master and I'm signing off this issue.
>
> If you could test this further (using your app is just fine, but of
> course IndexWriter and DocumentsWriter tests are even better), check
> cl_demo for leaks and try to see whats wrong with testWickedLongTerm -
> that would be great.
>
> Itamar.
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> CLucene-developers mailing list
> CLucene-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
CLucene-developers mailing list
CLucene-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/clucene-developers
