[
https://issues.apache.org/jira/browse/LUCENE-7887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless resolved LUCENE-7887.
----------------------------------------
Resolution: Fixed
Fix Version/s: master (7.0)
I found the cause: IW was in the process of aborting, while threads were
concurrently still trying to apply deleted packets. I simplified the code here
to not bother clearing the delete counters from abort.
> TestIndexWriterWithThreads.testIOExceptionDuringWriteSegmentWithThreadsOnlyOnce()
> failure: numTerms2=0 vs -8
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-7887
> URL: https://issues.apache.org/jira/browse/LUCENE-7887
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Steve Rowe
> Fix For: master (7.0)
>
>
> Non-reproducing failure from
> [https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1338/]:
> {noformat}
> Checking out Revision 9f56698d33d1db9fab6a0d6f63b360b334f71583
> (refs/remotes/origin/master)
> [...]
> [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads
> [junit4] 1> Thread-12393: ERROR: unexpected Throwable:
> [junit4] 1> java.lang.AssertionError: numTerms2=0 vs -8
> [junit4] 1> at
> org.apache.lucene.index.BufferedUpdatesStream.checkDeleteStats(BufferedUpdatesStream.java:376)
> [junit4] 1> at
> org.apache.lucene.index.BufferedUpdatesStream.push(BufferedUpdatesStream.java:85)
> [junit4] 1> at
> org.apache.lucene.index.IndexWriter.publishFrozenUpdates(IndexWriter.java:2655)
> [junit4] 1> at
> org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.finishFlush(DocumentsWriterFlushQueue.java:205)
> [junit4] 1> at
> org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:248)
> [junit4] 1> at
> org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:116)
> [junit4] 1> at
> org.apache.lucene.index.DocumentsWriterFlushQueue.forcePurge(DocumentsWriterFlushQueue.java:138)
> [junit4] 1> at
> org.apache.lucene.index.DocumentsWriter.purgeBuffer(DocumentsWriter.java:200)
> [junit4] 1> at
> org.apache.lucene.index.IndexWriter.purge(IndexWriter.java:5004)
> [junit4] 1> at
> org.apache.lucene.index.DocumentsWriter$ForcedPurgeEvent.process(DocumentsWriter.java:751)
> [junit4] 1> at
> org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5061)
> [junit4] 1> at
> org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5049)
> [junit4] 1> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1719)
> [junit4] 1> at
> org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
> [junit4] 2> NOTE: download the large Jenkins line-docs file by running
> 'ant get-jenkins-line-docs' in the lucene directory.
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterWithThreads
> -Dtests.method=testIOExceptionDuringWriteSegmentWithThreadsOnlyOnce
> -Dtests.seed=66484822FD0371B5 -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt
> -Dtests.locale=es-CR -Dtests.timezone=America/Manaus -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> [junit4] FAILURE 0.06s J2 |
> TestIndexWriterWithThreads.testIOExceptionDuringWriteSegmentWithThreadsOnlyOnce
> <<<
> [junit4] > Throwable #1: java.lang.AssertionError: hit unexpected
> Throwable
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([66484822FD0371B5:2A23C9E43BA35250]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterWithThreads._testMultipleThreadsFailure(TestIndexWriterWithThreads.java:300)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringWriteSegmentWithThreadsOnlyOnce(TestIndexWriterWithThreads.java:497)
> [junit4] > at java.lang.Thread.run(Thread.java:748)
> [junit4] 2> NOTE: leaving temporary files on disk at:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/core/test/J2/temp/lucene.index.TestIndexWriterWithThreads_66484822FD0371B5-001
> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
> {date=PostingsFormat(name=LuceneVarGapFixedInterval),
> field=PostingsFormat(name=LuceneVarGapFixedInterval),
> docid=PostingsFormat(name=Asserting),
> titleTokenized=PostingsFormat(name=LuceneFixedGap),
> body=PostingsFormat(name=LuceneVarGapFixedInterval),
> title=Lucene50(blocksize=128)}, docValues:{dv=DocValuesFormat(name=Direct),
> docid_intDV=DocValuesFormat(name=Memory), field=DocValuesFormat(name=Direct),
> titleDV=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1926,
> maxMBSortInHeap=6.378832546318826, sim=RandomSimilarity(queryNorm=false):
> {field=DFR GB2, titleTokenized=DFR I(n)3(800.0), body=IB LL-D1},
> locale=es-CR, timezone=America/Manaus
> [junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation
> 1.8.0_131 (64-bit)/cpus=4,threads=1,free=120430360,total=441974784
> [junit4] 2> NOTE: All tests run in this JVM: [TestLucene70NormsFormat,
> TestOfflineSorter, TestPrefixQuery, TestStringMSBRadixSorter,
> TestFieldsReader, TestSynonymQuery, TestSpanCollection, TestPackedInts,
> TestLongPostings, TestIndexWriterMerging, TestDocIdSetBuilder,
> TestAxiomaticSimilarity, TestFloatRangeFieldQueries, TestIndexWriterCommit,
> TestRoaringDocIdSet, TestStressAdvance, TestFieldCacheRewriteMethod,
> TestRamUsageEstimator, TestForUtil, TestNativeFSLockFactory,
> TestCompiledAutomaton, TestSegmentTermDocs, TestIOUtils, TestStressDeletes,
> TestHighCompressionMode, TestSearchWithThreads, TestByteBlockPool,
> TestIndexWriterLockRelease, TestSubScorerFreqs, TestBKD,
> TestSimpleExplanationsOfNonMatches, TestPackedTokenAttributeImpl,
> TestStopFilter, TestIndexWriterExceptions2, TestFastCompressionMode,
> TestManyFields, TestExceedMaxTermLength, TestTrackingDirectoryWrapper,
> TestAttributeSource, TestMixedCodecs, TestMergedIterator,
> TestAllFilesHaveCodecHeader, TestNeedsScores, TestEarlyTermination,
> TestNoMergePolicy, TestSetOnce, TestCachingTokenFilter,
> TestTwoPhaseCommitTool, TestIndexReaderClose, TestDemoParallelLeafReader,
> TestHugeRamFile, TestControlledRealTimeReopenThread,
> TestNRTReaderWithThreads, TestPayloads, TestSearchForDuplicates, TestIntsRef,
> TestShardSearching, TestNRTReaderCleanup, TestSegmentReader,
> TestSpanNotQuery, TestPerFieldPostingsFormat2, TestSpans, TestMultiFields,
> Test2BPositions, TestTerms, TestReaderWrapperDVTypeCheck, TestScorerPerf,
> TestBlockPostingsFormat, TestField, TestIndexWriterMergePolicy,
> TestIndexWriterReader, FiniteStringsIteratorTest, TestOperations,
> TestCharsRef, TestStressIndexing2, TestBagOfPostings,
> TestIndexWriterWithThreads]
> [junit4] Completed [189/453 (1!)] on J2 in 5.60s, 12 tests, 1 failure <<<
> FAILURES!
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
