OK I got TestNRTThreads (alone, no crashing) to fail, once I added a CheckIndex to it (and ran under while(1)).
So this is not particular to crashing... it's just because PulsingCodec was using crazy RAM on cloning its TermState. I fixed this in LUCENE-2857. Mike On Mon, Jan 10, 2011 at 6:26 AM, Michael McCandless <[email protected]> wrote: > OK, so this looks be caused by 1) the fact that we are indexing Greek > stop words with the TestNRTThreads test, and 2) Pulsing codec is > horribly inefficient in how it handles pulsed terms that have many > many positions. > > But it's odd we've only hit this failure in the JRE crash test... I've > added a CheckIndex to TestNRTThreads itself and I'll see if that too > can provoke an OOME. It should. > > Mike > > On Mon, Jan 10, 2011 at 5:14 AM, Michael McCandless > <[email protected]> wrote: >> OK I snatched this index and indeed I can reproduce the OOME during >> CheckIndex! The hunt begins :) >> >> Mike >> >> On Sun, Jan 9, 2011 at 10:35 PM, Robert Muir <[email protected]> wrote: >>> On Sun, Jan 9, 2011 at 9:40 PM, Apache Hudson Server >>> <[email protected]> wrote: >>>> Build: https://hudson.apache.org/hudson/job/Lucene-trunk/1421/ >>>> >>>> 1 tests failed. >>>> REGRESSION: >>>> org.apache.lucene.index.TestIndexWriterOnJRECrash.testNRTThreads >>>> >>>> Error Message: >>>> CheckIndex failed >>> >>> maybe this is specific to pulsing? I noticed its failed 3 times with >>> this identical pulsing stacktrace: >>> Lucene-trunk/1421, tests-only/3590, tests-only/3570 >>> >>> However, this time it failed in a nightly build (perhaps the indexes >>> are still available on the hudson machine if we salvage before the >>> next nightly build?) >>> it should be under lucene/build/test/N/jrecrashXXXXXXXXXXtmp/ >>> >>> all 3 times the stacktrace is: >>> test: terms, freq, prox...ERROR [Java heap space] >>> java.lang.OutOfMemoryError: Java heap space >>> at >>> org.apache.lucene.index.codecs.pulsing.PulsingPostingsWriterImpl$Position.clone(PulsingPostingsWriterImpl.java:104) >>> at >>> org.apache.lucene.index.codecs.pulsing.PulsingPostingsWriterImpl$Document.clone(PulsingPostingsWriterImpl.java:74) >>> at >>> org.apache.lucene.index.codecs.pulsing.PulsingPostingsReaderImpl$PulsingTermState.clone(PulsingPostingsReaderImpl.java:72) >>> at >>> org.apache.lucene.index.codecs.pulsing.PulsingPostingsReaderImpl$PulsingDocsEnum.reset(PulsingPostingsReaderImpl.java:234) >>> at >>> org.apache.lucene.index.codecs.pulsing.PulsingPostingsReaderImpl.docs(PulsingPostingsReaderImpl.java:189) >>> at >>> org.apache.lucene.index.codecs.PrefixCodedTermsReader$FieldReader$SegmentTermsEnum.docs(PrefixCodedTermsReader.java:515) >>> at >>> org.apache.lucene.index.CheckIndex.testTermIndex(CheckIndex.java:756) >>> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:489) >>> at org.apache.lucene.util._TestUtil.checkIndex(_TestUtil.java:83) >>> at org.apache.lucene.util._TestUtil.checkIndex(_TestUtil.java:73) >>> at >>> org.apache.lucene.index.TestIndexWriterOnJRECrash.checkIndexes(TestIndexWriterOnJRECrash.java:131) >>> at >>> org.apache.lucene.index.TestIndexWriterOnJRECrash.checkIndexes(TestIndexWriterOnJRECrash.java:137) >>> at >>> org.apache.lucene.index.TestIndexWriterOnJRECrash.testNRTThreads(TestIndexWriterOnJRECrash.java:61) >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
