[ https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478194#comment-16478194 ]
Michael McCandless commented on LUCENE-8317: -------------------------------------------- {{fully}} --> {{full}} I'm a little confused: what goes wrong if we allow {{flushControl.getAndResetApplyAllDeletes}} when it's a full flush? Are we really exposing a "future delete" in the flush? > TestStressNRT fails with missing document > ----------------------------------------- > > Key: LUCENE-8317 > URL: https://issues.apache.org/jira/browse/LUCENE-8317 > Project: Lucene - Core > Issue Type: Bug > Reporter: Simon Willnauer > Priority: Major > Attachments: LUCENE-8317.patch, LUCENE-8317.patch > > > {noformat} > 11:39:01 [junit4] Suite: org.apache.lucene.index.TestStressNRT > 11:39:01 [junit4] 1> READER1: FAILED: unexpected exception > 11:39:01 [junit4] 1> java.lang.AssertionError: No documents or > tombstones found for id 49, expected at least 66 > reader=StandardDirectoryReader(segments_g:325:nrt > _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1 > _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1 > _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1) > 11:39:01 [junit4] 1> at org.junit.Assert.fail(Assert.java:93) > 11:39:01 [junit4] 1> at > org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353) > 11:39:01 [junit4] 2> May 16, 2018 7:39:01 PM > com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler > uncaughtException > 11:39:01 [junit4] 2> WARNING: Uncaught exception in thread: > Thread[READER1,5,TGRP-TestStressNRT] > 11:39:01 [junit4] 2> java.lang.RuntimeException: > java.lang.AssertionError: No documents or tombstones found for id 49, > expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt > _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1 > _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1 > _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1) > 11:39:01 [junit4] 2> at > __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0) > 11:39:01 [junit4] 2> at > org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382) > 11:39:01 [junit4] 2> Caused by: java.lang.AssertionError: No documents > or tombstones found for id 49, expected at least 66 > reader=StandardDirectoryReader(segments_g:325:nrt > _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1 > _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1 > _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1) > 11:39:01 [junit4] 2> at org.junit.Assert.fail(Assert.java:93) > 11:39:01 [junit4] 2> at > org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353) > 11:39:01 [junit4] 2> > 11:39:01 [junit4] 2> NOTE: reproduce with: ant test > -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387 > -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU > -Dtests.timezone=Asia/Taipei -Dtests.asserts=true > -Dtests.file.encoding=US-ASCII > 11:39:01 [junit4] ERROR 0.61s J1 | TestStressNRT.test <<< > 11:39:01 [junit4] > Throwable #1: java.lang.RuntimeException: > MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1, > _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1} > 11:39:01 [junit4] > at > org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841) > 11:39:01 [junit4] > at > org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403) > 11:39:01 [junit4] > at java.lang.Thread.run(Thread.java:748) > 11:39:01 [junit4] > Caused by: java.lang.RuntimeException: unclosed > IndexInput: _2l.fdt > 11:39:01 [junit4] > at > org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732) > 11:39:01 [junit4] > at > org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776) > 11:39:01 [junit4] > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.<init>(CompressingStoredFieldsReader.java:150) > 11:39:01 [junit4] > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121) > 11:39:01 [junit4] > at > org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173) > 11:39:01 [junit4] > at > org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43) > 11:39:01 [junit4] > at > org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:126) > 11:39:01 [junit4] > at > org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78) > 11:39:01 [junit4] > at > org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:193) > 11:39:01 [junit4] > at > org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:265) > 11:39:01 [junit4] > at > org.apache.lucene.index.FrozenBufferedUpdates.openSegmentStates(FrozenBufferedUpdates.java:369) > 11:39:01 [junit4] > at > org.apache.lucene.index.FrozenBufferedUpdates.apply(FrozenBufferedUpdates.java:266) > 11:39:01 [junit4] > at > org.apache.lucene.index.IndexWriter.lambda$publishFrozenUpdates$3(IndexWriter.java:2579) > 11:39:01 [junit4] > at > org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5051) > 11:39:01 [junit4] > at > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1587) > 11:39:01 [junit4] > at > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1576) > 11:39:01 [junit4] > at > org.apache.lucene.index.RandomIndexWriter.updateDocument(RandomIndexWriter.java:270) > 11:39:01 [junit4] > at > org.apache.lucene.index.TestStressNRT$1.run(TestStressNRT.java:274)Throwable > #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an > uncaught exception in thread: Thread[id=472, name=READER1, state=RUNNABLE, > group=TGRP-TestStressNRT] > 11:39:01 [junit4] > Caused by: java.lang.RuntimeException: > java.lang.AssertionError: No documents or tombstones found for id 49, > expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt > _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1 > _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1 > _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1) > 11:39:01 [junit4] > at > __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0) > 11:39:01 [junit4] > at > org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382) > 11:39:01 [junit4] > Caused by: java.lang.AssertionError: No documents > or tombstones found for id 49, expected at least 66 > reader=StandardDirectoryReader(segments_g:325:nrt > _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1 > _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1 > _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1) > 11:39:01 [junit4] > at > org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353) > 11:39:01 [junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): > {id=PostingsFormat(name=Memory)}, docValues:{}, maxPointsInLeafNode=188, > maxMBSortInHeap=7.356975177594725, sim=RandomSimilarity(queryNorm=false): > {id=DFR I(ne)L3(800.0)}, locale=en-AU, timezone=Asia/Taipei > 11:39:01 [junit4] 2> NOTE: Linux 4.4.0-1055-aws amd64/Oracle Corporation > 1.8.0_171 (64-bit)/cpus=4,threads=1,free=105189712,total=269484032 > 11:39:01 [junit4] 2> NOTE: All tests run in this JVM: [TestDirectory, > TestWeakIdentityMap, TestDuelingCodecs, TestIndexWriterDelete, > TestLucene70SegmentInfoFormat, TestLucene50TermVectorsFormat, > TestSoftDeletesDirectoryReaderWrapper, TestTopFieldCollectorEarlyTermination, > TestSpanOrQuery, TestVirtualMethod, TestComplexExplanations, > TestMatchesIterator, Test2BPostings, Test2BDocs, TestAddIndexes, > TestFieldType, TestTransactionRollback, TestNot, > TestEarlyTerminatingSortingCollector, TestDirectMonotonic, > TestSpanContainQuery, TestNoMergeScheduler, TestPositionIncrement, > TestIndexWriterUnicode, TestDocInverterPerFieldErrorInfo, > TestGraphTokenizers, TestAtomicUpdate, TestSortRandom, > TestRecyclingByteBlockAllocator, TestBlockPostingsFormat3, > TestSpanExplanations, TestAutomatonQueryUnicode, TestFilterSpans, > TestPriorityQueue, TestNGramPhraseQuery, TestLongRange, > TestSameScoresWithThreads, TestSingleInstanceLockFactory, > TestPrefixInBooleanQuery, TestMutablePointsReaderUtils, TestDoc, > TestIndexSearcher, TestOmitPositions, TestSizeBoundedForceMerge, > TestCodecUtil, Test2BPagedBytes, TestPhraseQuery, TestSpanFirstQuery, > TestTransactions, TestRegexpRandom2, TestDemo, TestMultiTermConstantScore, > TestTermVectors, TestLucene50LiveDocsFormat, TestFilterIterator, > TestMaxPosition, TestIndexTooManyDocs, TestRollback, TestPendingSoftDeletes, > TestSpanExplanationsOfNonMatches, TestMathUtil, TestStressDeletes, > TestStressNRT] > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org