I know. Without a solid test to reproduce the problem this is quite hard to pinpoint and fix. As you can see from this thread, I'm doing my best to resolve this quickly, but due to time constraints apparently I won't be able to do this all on my own.
If you can jump in and help, by providing a test case or more insights, that would be great. See previous messages in this thread for what we have gathered on it so far. Itamar. On 27/6/2010 1:58 PM, justroftest wrote: > hi, > That is the same problem reported here [1] > after updating my master branch, cl_demo can index reuters-21578 without > any crash > cl_test fail on testWickedLongTerm [2] > we are working on a project that use CLucene, we need to index about 2GB > at least, and this problem is really annoying us. > > Regards > Ahmed > [1]http://comments.gmane.org/gmane.comp.jakarta.lucene.clucene.devel/3393 > [2]http://pastebin.com/mR2ANcfC > > Le dimanche 27 juin 2010 à 03:27 +0300, Itamar Syn-Hershko a écrit : > >> Just one other small thing I noticed: apparently this only occurs when >> writer->setRAMBufferSizeMB(0.5); is called. This leads me to think >> something in the process of calculating which postings to flush and >> delete (based on their block size) is wrong. >> >> Uncomment writer->setInfoStream(&std::cout); in the demo application to >> see how this is happening. >> >> Itamar. >> >> On 27/6/2010 12:06 AM, Itamar Syn-Hershko wrote: >> >>> Borek, a quick update: >>> >>> Apparently I was wrong. The 2 issues mentioned in JIRA 1072 were already >>> fixed in 2.3.2, and the core patches attached to it weren't showing up >>> in the release since other check-ins updated them to work differently. >>> >>> So, what you were experiencing is either a CLucene specific bug or some >>> JLucene behavior that isn't tolerated properly in a non-GC environment. >>> Another option is that some code portion hasn't been ported in full, or >>> correctly. >>> >>> I think it is better at this point to investigate where CLucene is >>> failing JL tests and why, and then try to fix this accordingly (instead >>> of tracking loads of memory allocations until you pinpoint the issue you >>> are having and create a test case out of it). >>> >>> testWickedLongTerm (which I ported) is failing in one of the asserts - I >>> think it is worth a look. Another test I think can help is >>> testDocumentsWriterAbort (which I haven't ported yet). Other than that, >>> trying to compare CLucene code files with their parallel Java's, and >>> port more tests which test relevant code portions is also a good idea. >>> >>> Please update your local copy of the master branch, I have checked-in >>> some more updates and tests. I also updated the test suite so it is >>> somewhat easier to port Java tests into it. Let me know how it goes, >>> I'll try to do some more work on this later this week. >>> >>> Itamar. >>> >>> On 25/6/2010 4:06 PM, Kostka Bořivoj wrote: >>> >>> >>>> OK, I'm going to port this. >>>> >>>> >>>> >>> ------------------------------------------------------------------------------ >>> This SF.net email is sponsored by Sprint >>> What will you do first with EVO, the first 4G phone? >>> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first >>> _______________________________________________ >>> CLucene-developers mailing list >>> CLucene-developers@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/clucene-developers >>> >>> >>> >>> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Sprint >> What will you do first with EVO, the first 4G phone? >> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first >> _______________________________________________ >> CLucene-developers mailing list >> CLucene-developers@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/clucene-developers >> > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Sprint > What will you do first with EVO, the first 4G phone? > Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first > _______________________________________________ > CLucene-developers mailing list > CLucene-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/clucene-developers > ------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first _______________________________________________ CLucene-developers mailing list CLucene-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/clucene-developers
