Steve Rowe created LUCENE-7142:
----------------------------------
Summary: Test2BPoints and Test2BBKDPoints.test2D() failure: wrong
number of points in split
Key: LUCENE-7142
URL: https://issues.apache.org/jira/browse/LUCENE-7142
Project: Lucene - Core
Issue Type: Bug
Reporter: Steve Rowe
My Jenkins found failures in {{Test2BBKDPoints}} and {{Test2BPoints}} in the
same run on branch_6x; I've reported here before seeing them reproduce because
the tests took 12 and 20 hours to run, respectively.
{noformat}
Checking out Revision 39aaa108ac8a85809080e4f7cf2b5ac0cc0d0fe9
(refs/remotes/origin/branch_6x)
[...]
[junit4] HEARTBEAT J0 PID(8991@localhost): 2016-03-26T01:02:55, stalled for
45731s at: Test2BBKDPoints.test2D
[...]
[junit4] Suite: org.apache.lucene.util.bkd.Test2BBKDPoints
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=Test2BBKDPoints
-Dtests.method=test2D -Dtests.seed=E58B6D2DB5BEBF58 -Dtests.nightly=true
-Dtests.slow=true -Dtests.locale=el-GR -Dtests.timezone=Europe/Warsaw
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 45777s J0 | Test2BBKDPoints.test2D <<<
[junit4] > Throwable #1: java.lang.IllegalStateException: wrong number of
points in split: expected=1073743112 but actual=1
[junit4] > at
__randomizedtesting.SeedInfo.seed([E58B6D2DB5BEBF58:C82EDF7D5339FEBB]:0)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1224)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:862)
[junit4] > at
org.apache.lucene.util.bkd.Test2BBKDPoints.test2D(Test2BBKDPoints.java:96)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Suppressed:
org.apache.lucene.index.CorruptIndexException: checksum passed (62a1e57e).
possibly transient resource issue, or a Lucene or JVM bug
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/slow/jenkins/HDD-workspaces/Lucene-core-nightly-monster-6.x/lucene/build/core/test/J0/temp/lucene.util.bkd.Test2BBKDPoints_E58B6D2DB5BEBF58-001/2BBKDPoints2D-001/_0_bkd1_sort_71i.tmp")))
[junit4] > at
org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:409)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.verifyChecksum(BKDWriter.java:988)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1230)
[junit4] > ... 38 more
[junit4] 2> NOTE: leaving temporary files on disk at:
/slow/jenkins/HDD-workspaces/Lucene-core-nightly-monster-6.x/lucene/build/core/test/J0/temp/lucene.util.bkd.Test2BBKDPoints_E58B6D2DB5BEBF58-001
[junit4] 2> NOTE: test params are: codec=CheapBastard,
sim=ClassicSimilarity, locale=el-GR, timezone=Europe/Warsaw
[junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation
1.8.0_45 (64-bit)/cpus=16,threads=1,free=1512802208,total=4148166656
[junit4] 2> NOTE: All tests run in this JVM: [TestCustomSearcherSort,
TestTimSorterWorstCase, TestPayloadsOnVectors, TestShardSearching,
TestNumericDocValuesUpdates, TestCharsRef, TestCompiledAutomaton,
TestFieldValueQuery, TestMutableValues, TestPerFieldPostingsFormat2,
TestAtomicUpdate, TestDocumentsWriterDeleteQueue, TestParallelReaderEmptyIndex,
Test2BNumericDocValues, TestQueryBuilder, TestNumericTokenStream,
TestAllFilesHaveChecksumFooter, TestExitableDirectoryReader, TestDeterminism,
TestTrackingDirectoryWrapper, TestConjunctionDISI, TestFilterLeafReader,
TestWeakIdentityMap, TestBlendedTermQuery, TestIndexWriterOnVMError,
TestSimpleFSDirectory, TestFixedBitDocIdSet, TestNewestSegment,
TestIndexWriterForceMerge, TestTotalHitCountCollector, Test4GBStoredFields,
TestBytesRef, TestSortedSetSortField, TestSentinelIntSet, TestToken,
TestLogMergePolicy, TestIntsRef, TestIndexWriterDelete, TestOperations,
TestNeedsScores, TestSpanFirstQuery, TestPackedInts,
TestMultiTermConstantScore, TestReqExclBulkScorer, TestBinaryTerms,
TestFastCompressionMode, TestDirectoryReaderReopen, TestRollingUpdates,
TestPagedBytes, TestTryDelete, TestSegmentMerger, TestMinShouldMatch2,
TestPositiveScoresOnlyCollector, TestByteArrayDataInput, Test2BPagedBytes,
TestReusableStringReader, TestBytesRefAttImpl, TestPackedTokenAttributeImpl,
TestGrowableByteArrayDataOutput, TestBlockPostingsFormat,
TestBlockPostingsFormat2, TestForUtil, TestLucene50SegmentInfoFormat,
TestLucene50StoredFieldsFormat, TestLucene50StoredFieldsFormatHighCompression,
TestLucene50TermVectorsFormat, TestLucene53NormsFormat,
TestLucene54DocValuesFormat, Test2BSortedDocValuesFixedSorted,
TestAllFilesCheckIndexHeader, TestAllFilesDetectTruncation,
TestDocInverterPerFieldErrorInfo, TestDocValues, TestDuelingCodecsAtNight,
TestFilterDirectoryReader, TestIndexWriterDeleteByQuery,
TestIndexWriterExceptions2, TestMergePolicyWrapper, TestMergeRateLimiter,
TestMixedDocValuesUpdates, TestMultiTermsEnum, TestNRTReaderCleanup,
TestPointValues, TestReadOnlyIndex, TestReaderWrapperDVTypeCheck,
TestSegmentInfos, TestSwappedIndexFiles, TestTermVectors,
TestApproximationSearchEquivalence, TestBooleanCoord, TestBooleanRewrites,
TestEarlyTermination, TestIndexSearcher, TestMultiPhraseEnum,
TestQueryRescorer, TestSortedSetSelector, TestClassicSimilarity,
TestFilterSpans, TestSpanCollection, TestSpanContainQuery, TestSpanOrQuery,
TestSpanTermQuery, TestSpansEnum, TestBufferedChecksum, TestFilterDirectory,
TestMmapDirectory, TestNIOFSDirectory, TestNativeFSLockFactory,
TestRateLimiter, TestSleepingLockWrapper, TestCharsRefBuilder,
TestFrequencyTrackingRingBuffer, TestInPlaceMergeSorter, TestIntArrayDocIdSet,
TestIntroSorter, TestLongBitSet, TestMergedIterator, TestNotDocIdSet,
TestSparseFixedBitDocIdSet, TestTimSorter, TestRegExp, Test2BBKDPoints]
[...]
[junit4] HEARTBEAT J2 PID(8984@localhost): 2016-03-26T06:12:22, stalled for
72912s at: Test2BPoints.test2D
[junit4] Suite: org.apache.lucene.index.Test2BPoints
[junit4] 1> TEST: now CheckIndex
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=Test2BPoints
-Dtests.method=test2D -Dtests.seed=E58B6D2DB5BEBF58 -Dtests.nightly=true
-Dtests.slow=true -Dtests.locale=ms-MY -Dtests.timezone=Europe/Rome
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 72944s J2 | Test2BPoints.test2D <<<
[junit4] > Throwable #1: java.lang.IllegalStateException: this writer hit
an unrecoverable error; cannot complete forceMerge
[junit4] > at
__randomizedtesting.SeedInfo.seed([E58B6D2DB5BEBF58:C82EDF7D5339FEBB]:0)
[junit4] > at
org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1786)
[junit4] > at
org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1736)
[junit4] > at
org.apache.lucene.index.Test2BPoints.test2D(Test2BPoints.java:133)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: java.lang.IllegalStateException: wrong number of
points in split: expected=1073741825 but actual=0
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1224)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:862)
[junit4] > at
org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:115)
[junit4] > at
org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:58)
[junit4] > at
org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:204)
[junit4] > at
org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:168)
[junit4] > at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:117)
[junit4] > at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4099)
[junit4] > at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3679)
[junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
[junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
[junit4] > Suppressed:
org.apache.lucene.index.CorruptIndexException: checksum passed (90c634e2).
possibly transient resource issue, or a Lucene or JVM bug
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/slow/jenkins/HDD-workspaces/Lucene-core-nightly-monster-6.x/lucene/build/core/test/J2/temp/lucene.index.Test2BPoints_E58B6D2DB5BEBF58-001/2BPoints2D-001/_50_bkd0_sort_1iy7.tmp")))
[junit4] > at
org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:409)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.verifyChecksum(BKDWriter.java:988)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1230)
[junit4] > ... 10 more
[junit4] 2> NOTE: leaving temporary files on disk at:
/slow/jenkins/HDD-workspaces/Lucene-core-nightly-monster-6.x/lucene/build/core/test/J2/temp/lucene.index.Test2BPoints_E58B6D2DB5BEBF58-001
[junit4] 2> Mac 26, 2016 6:12:53 AM
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[Lucene Merge
Thread #17,5,TGRP-Test2BPoints]
[junit4] 2> org.apache.lucene.index.MergePolicy$MergeException:
java.lang.IllegalStateException: wrong number of points in split:
expected=1073741825 but actual=0
[junit4] 2> at
__randomizedtesting.SeedInfo.seed([E58B6D2DB5BEBF58]:0)
[junit4] 2> at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
[junit4] 2> at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
[junit4] 2> Caused by: java.lang.IllegalStateException: wrong number of
points in split: expected=1073741825 but actual=0
[junit4] 2> at
org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1224)
[junit4] 2> at
org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:862)
[junit4] 2> at
org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:115)
[junit4] 2> at
org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:58)
[junit4] 2> at
org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:204)
[junit4] 2> at
org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:168)
[junit4] 2> at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:117)
[junit4] 2> at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4099)
[junit4] 2> at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3679)
[junit4] 2> at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
[junit4] 2> at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
[junit4] 2> Suppressed:
org.apache.lucene.index.CorruptIndexException: checksum passed (90c634e2).
possibly transient resource issue, or a Lucene or JVM bug
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/slow/jenkins/HDD-workspaces/Lucene-core-nightly-monster-6.x/lucene/build/core/test/J2/temp/lucene.index.Test2BPoints_E58B6D2DB5BEBF58-001/2BPoints2D-001/_50_bkd0_sort_1iy7.tmp")))
[junit4] 2> at
org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:409)
[junit4] 2> at
org.apache.lucene.util.bkd.BKDWriter.verifyChecksum(BKDWriter.java:988)
[junit4] 2> at
org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1230)
[junit4] 2> ... 10 more
[junit4] 2>
[junit4] 2> Mar 26, 2016 6:12:53 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked
thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard,
sim=ClassicSimilarity, locale=ms-MY, timezone=Europe/Rome
[junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation
1.8.0_45 (64-bit)/cpus=16,threads=1,free=110605288,total=675807232
[junit4] 2> NOTE: All tests run in this JVM: [TestSortedSetDocValues,
TestDocValuesScoring, TestHugeRamFile, TestStringHelper,
TestCodecHoldsOpenFiles, TestNGramPhraseQuery,
TestUsageTrackingFilterCachingPolicy, TestSpanNotQuery, TestFieldType,
TestLRUQueryCache, TestFixedBitSet, TestNumericUtils, TestMultiPhraseQuery,
TestFuzzyQuery, TestCachingTokenFilter, TestMultiLevelSkipList,
TestNumericRangeQuery32, TestIndexWriterNRTIsCurrent, TestForceMergeForever,
TestCachingCollector, TestNumericRangeQuery64, TestDocumentWriter,
TestSegmentTermEnum, TestPositionIncrement, LimitedFiniteStringsIteratorTest,
TestSpanBoostQuery, TestIndexWriterThreadsToSegments,
TestIndexWriterFromReader, TestRecyclingIntBlockAllocator,
TestSingleInstanceLockFactory, TestPrefixCodedTerms, TestMaxPosition,
TestDocValuesRewriteMethod, TestRollingBuffer, TestAssertions,
FuzzyTermOnShortTermsTest, TestTerms, TestMergeSchedulerExternal,
TestSPIClassIterator, TestIOUtils, TestQueryCachingPolicy, TestCrash,
TestIndexWriterReader, TestBooleanMinShouldMatch, TestDuelingCodecs,
TestSortedNumericSortField, TestCodecUtil, TestLucene50FieldInfoFormat,
TestUTF32ToUTF8, TestTermdocPerf, TestIndexWriterOnDiskFull, TestDirectPacked,
TestIndexWriter, TestBytesStore, TestIndexWriterWithThreads,
TestIndexWriterExceptions, TestSearcherManager, TestDeletionPolicy, TestBasics,
TestFlushByRamOrCountsPolicy, TestSloppyPhraseQuery, TestNRTThreads,
TestBoolean2, TestStressAdvance, TestNorms, TestSearchWithThreads,
TestConcurrentMergeScheduler, TestDocumentsWriterStallControl, TestNeverDelete,
TestTopDocsMerge, TestLongPostings, TestMinimize, TestNRTReaderWithThreads,
TestTieredMergePolicy, TestPerFieldPostingsFormat, TestLevenshteinAutomata,
TestBufferedIndexInput, TestTermsEnum, TestSimpleExplanations, TestLockFactory,
TestPhraseQuery, TestCollectionUtil, TestBagOfPositions, TestOmitNorms,
TestComplexExplanations, TestRegexpRandom2, TestNRTCachingDirectory,
TestSegmentReader, TestFastDecompressionMode, TestParallelCompositeReader,
TestTransactions, TestMultiThreadTermVectors, TestSegmentTermDocs, TestSort,
TestMultiDocValues, TestFieldsReader, TestDocValuesIndexing, TestTermsEnum2,
TestLegacyNumericUtils, TestPostingsOffsets, TestMixedCodecs, TestFlex,
TestStressIndexing2, TestRAMDirectory, TestTermVectorsWriter, TestCodecs,
TestCharTermAttributeImpl, TestParallelLeafReader, TestBytesRefHash,
TestThreadedForceMerge, TestStressNRT, TestSloppyPhraseQuery2,
TestDisjunctionMaxQuery, TestOmitTf, TestSameScoresWithThreads,
TestTermRangeQuery, TestCrashCausesCorruptIndex, TestLiveFieldValues,
TestForTooMuchCloning, TestPayloads, TestDoc, TestWildcard, TestSimilarity2,
TestFieldMaskingSpanQuery, TestExternalCodecs, TestNoDeletionPolicy,
TestSimpleExplanationsOfNonMatches, TestTermVectorsReader,
TestPerSegmentDeletes, TestMultiValuedNumericRangeQuery, TestIntBlockPool,
TestTopDocsCollector, TestUniqueTermCount, TestBooleanQuery, TestUnicodeUtil,
TestAllFilesHaveCodecHeader, TestAutomatonQuery, TestDocument,
TestTopFieldCollector, TestFileSwitchDirectory, TestSumDocFreq,
TestIndexableField, TestWildcardRandom, TestPrefixRandom, TestDirectory,
TestRegexpRandom, TestCheckIndex, TestSpanMultiTermQueryWrapper,
TestBooleanQueryVisitSubscorers, TestSizeBoundedForceMerge,
TestIndexFileDeleter, TestConstantScoreQuery, TestMatchAllDocsQuery,
TestParallelTermEnum, TestElevationComparator, TestSubScorerFreqs,
TestConjunctions, TestSimilarity, TestVirtualMethod, TestSimilarityProvider,
TestSetOnce, TestSearch, TestFilterIterator, Test2BPostings,
TestSameTokenSamePosition, TestDocCount, TestBinaryDocument,
TestAutomatonQueryUnicode, TestAttributeSource, TestBooleanScorer,
TestRamUsageEstimator, TestNoMergePolicy, TestNoMergeScheduler,
TestNamedSPILoader, TestIndexWriterLockRelease, TestPhrasePrefixQuery,
TestRollback, TestTwoPhaseCommitTool, TestCloseableThreadLocal,
MultiCollectorTest, TestDocBoost, Test2BPositions, TestIndexCommit, TestTerm,
Test2BPostingsBytes, Test2BPoints]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=Test2BPoints
-Dtests.seed=E58B6D2DB5BEBF58 -Dtests.nightly=true -Dtests.slow=true
-Dtests.locale=ms-MY -Dtests.timezone=Europe/Rome -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J2 | Test2BPoints (suite) <<<
[junit4] > Throwable #1:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=4892, name=Lucene Merge Thread #17,
state=RUNNABLE, group=TGRP-Test2BPoints]
[junit4] > Caused by: org.apache.lucene.index.MergePolicy$MergeException:
java.lang.IllegalStateException: wrong number of points in split:
expected=1073741825 but actual=0
[junit4] > at
__randomizedtesting.SeedInfo.seed([E58B6D2DB5BEBF58]:0)
[junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
[junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
[junit4] > Caused by: java.lang.IllegalStateException: wrong number of
points in split: expected=1073741825 but actual=0
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1224)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:862)
[junit4] > at
org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:115)
[junit4] > at
org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:58)
[junit4] > at
org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:204)
[junit4] > at
org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:168)
[junit4] > at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:117)
[junit4] > at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4099)
[junit4] > at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3679)
[junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
[junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
[junit4] > Suppressed:
org.apache.lucene.index.CorruptIndexException: checksum passed (90c634e2).
possibly transient resource issue, or a Lucene or JVM bug
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/slow/jenkins/HDD-workspaces/Lucene-core-nightly-monster-6.x/lucene/build/core/test/J2/temp/lucene.index.Test2BPoints_E58B6D2DB5BEBF58-001/2BPoints2D-001/_50_bkd0_sort_1iy7.tmp")))
[junit4] > at
org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:409)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.verifyChecksum(BKDWriter.java:988)
[junit4] > at
org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1230)
[junit4] > ... 10 more
[junit4] Completed [415/415 (3!)] on J2 in 82560.59s, 2 tests, 2 errors <<<
FAILURES!
{noformat}
FWIW, here's how the suite was run (via Jenkins):
{noformat}
ant -f lucene/core/build.xml clean test -Dtests.jvms=4 -Dtests.nightly=true
-Dtests.monster=true -Dtests.heapsize=30g
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
