[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478965#comment-16478965
]
ASF subversion and git services commented on LUCENE-8317:
-
Commit 922fd26859cd1e288c8e9ed0d1f22bf75306de90 in lucene-solr's branch
refs/heads/branch_7x from [~simonw]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=922fd26 ]
LUCENE-8317: Prevent concurrent deletes from being applied during full flush
Future deletes could potentially be exposed to flushes/commits/refreshes if the
amount of RAM used by deletes is greater than half of the IW RAM buffer.
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8317.patch, LUCENE-8317.patch, LUCENE-8317.patch,
> LUCENE-8317.patch, LUCENE-8317.patch
>
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:
[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478963#comment-16478963
]
ASF subversion and git services commented on LUCENE-8317:
-
Commit 88f8718f1bfe0e5aeddc6f960cc74513a89c0610 in lucene-solr's branch
refs/heads/master from [~simonw]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=88f8718 ]
LUCENE-8317: Prevent concurrent deletes from being applied during full flush
Future deletes could potentially be exposed to flushes/commits/refreshes if the
amount of RAM used by deletes is greater than half of the IW RAM buffer.
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8317.patch, LUCENE-8317.patch, LUCENE-8317.patch,
> LUCENE-8317.patch, LUCENE-8317.patch
>
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> 11:39:01[junit4]> at
> org
[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478955#comment-16478955
]
Michael McCandless commented on LUCENE-8317:
+1 phew.
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8317.patch, LUCENE-8317.patch, LUCENE-8317.patch,
> LUCENE-8317.patch, LUCENE-8317.patch
>
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:126)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:78)
> 11:39:01[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates
[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478833#comment-16478833
]
Simon Willnauer commented on LUCENE-8317:
-
[~mikemccand] I found one more issue while writing an evil test to cause a ton
of updates of the same doc. Good news is that without my fix it fails very
quickly. But there is a possible NPE in FlushByRamOrCountsPolicy.java that I
fixed, can you take one more look?
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8317.patch, LUCENE-8317.patch, LUCENE-8317.patch,
> LUCENE-8317.patch, LUCENE-8317.patch
>
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
> 11:39:01[junit4]> at
> org.apache.lucene.index.Se
[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478790#comment-16478790
]
Simon Willnauer commented on LUCENE-8317:
-
added changes entry. I think it's ready I will push soon once the test run
finishes
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8317.patch, LUCENE-8317.patch, LUCENE-8317.patch,
> LUCENE-8317.patch
>
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:126)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:78)
> 11:39:01[junit4]> at
> org.apache.lucene.
[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478759#comment-16478759
]
Michael McCandless commented on LUCENE-8317:
OK, phew, awesome find! +1
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8317.patch, LUCENE-8317.patch, LUCENE-8317.patch
>
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:126)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:78)
> 11:39:01[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:193)
> 11:39:01
[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478528#comment-16478528
]
Simon Willnauer commented on LUCENE-8317:
-
> I'm a little confused: what goes wrong if we allow
> flushControl.getAndResetApplyAllDeletes when it's a full flush? Are we
> really exposing a "future delete" in the flush?
yes we are exposing a future delete. We also exposing the future delete of the
update which is worse I guess. If you remove the IW changes I made the test
fails with 0 docs every time.
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8317.patch, LUCENE-8317.patch
>
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
> 11:39:01[junit4]
[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478194#comment-16478194
]
Michael McCandless commented on LUCENE-8317:
{{fully}} --> {{full}}
I'm a little confused: what goes wrong if we allow
{{flushControl.getAndResetApplyAllDeletes}} when it's a full flush? Are we
really exposing a "future delete" in the flush?
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8317.patch, LUCENE-8317.patch
>
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:126)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentRea
[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477617#comment-16477617
]
Simon Willnauer commented on LUCENE-8317:
-
here is a patch. [~mikemccand] you might be interested in this
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8317.patch
>
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:126)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:78)
> 11:39:01[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:193)
> 11:39:01[junit
[jira] [Commented] (LUCENE-8317) TestStressNRT fails with missing document
[
https://issues.apache.org/jira/browse/LUCENE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477611#comment-16477611
]
Simon Willnauer commented on LUCENE-8317:
-
this failed on our CI today and I think it's a real issue but pretty long
standing. I have a test and a fix for it that shows the problem. I will attach
in a bit.
> TestStressNRT fails with missing document
> -
>
> Key: LUCENE-8317
> URL: https://issues.apache.org/jira/browse/LUCENE-8317
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
>
> {noformat}
> 11:39:01[junit4] Suite: org.apache.lucene.index.TestStressNRT
> 11:39:01[junit4] 1> READER1: FAILED: unexpected exception
> 11:39:01[junit4] 1> java.lang.AssertionError: No documents or
> tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 1> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 1> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2> May 16, 2018 7:39:01 PM
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 11:39:01[junit4] 2> WARNING: Uncaught exception in thread:
> Thread[READER1,5,TGRP-TestStressNRT]
> 11:39:01[junit4] 2> java.lang.RuntimeException:
> java.lang.AssertionError: No documents or tombstones found for id 49,
> expected at least 66 reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at
> __randomizedtesting.SeedInfo.seed([B7A02DE785EE2387]:0)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:382)
> 11:39:01[junit4] 2> Caused by: java.lang.AssertionError: No documents
> or tombstones found for id 49, expected at least 66
> reader=StandardDirectoryReader(segments_g:325:nrt
> _2i(7.4.0):c114/106:delGen=1 _2h(7.4.0):c76/75:delGen=1
> _2j(7.4.0):c32/28:delGen=1 _2k(7.4.0):c1 _2l(7.4.0):C38/23:delGen=1
> _2n(7.4.0):C6/4:delGen=1 _2m(7.4.0):C23/16:delGen=1)
> 11:39:01[junit4] 2> at org.junit.Assert.fail(Assert.java:93)
> 11:39:01[junit4] 2> at
> org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:353)
> 11:39:01[junit4] 2>
> 11:39:01[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=B7A02DE785EE2387
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-AU
> -Dtests.timezone=Asia/Taipei -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 11:39:01[junit4] ERROR 0.61s J1 | TestStressNRT.test <<<
> 11:39:01[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 7 open files: {_2h.cfs=1,
> _2n.fdt=1, _2k.cfs=1, _2l.fdt=1, _2j.cfs=1, _2i.cfs=1, _2m.fdt=1}
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> 11:39:01[junit4]> at
> org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:403)
> 11:39:01[junit4]> at java.lang.Thread.run(Thread.java:748)
> 11:39:01[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _2l.fdt
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
> 11:39:01[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:150)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> 11:39:01[junit4]> at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:126)
> 11:39:01[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:78)
> 11:39:01[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates
