[jira] [Commented] (LUCENE-9405) IndexWriter incorrectly calls closeMergeReaders twice when the merged segment is 100% deleted
[
https://issues.apache.org/jira/browse/LUCENE-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136616#comment-17136616
]
Michael McCandless commented on LUCENE-9405:
Thanks [~simonw]!
> IndexWriter incorrectly calls closeMergeReaders twice when the merged segment
> is 100% deleted
> -
>
> Key: LUCENE-9405
> URL: https://issues.apache.org/jira/browse/LUCENE-9405
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: master (9.0), 8.6
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This is the first spinoff from a [controversial PR to add a new index-time
> feature to Lucene to merge small segments during
> commit|https://github.com/apache/lucene-solr/pull/1552]. This can
> substantially reduce the number of small index segments to search.
> See specifically [this discussion
> there|https://github.com/apache/lucene-solr/pull/1552#discussion_r440298695].
> {{IndexWriter}} seems to be missing a {{success = true}} inside
> {{mergeMiddle}} in the case where all segments being merged have 100%
> deletions and the segments will simply be dropped.
> In this case, in master today, I think we are incorrectly calling
> {{closeMergedReaders}} twice, first with {{suppressExceptions = false}} and
> second time with {{true}}.
> There is a [dedicated test case here showing the
> issue|https://github.com/apache/lucene-solr/commit/cab5ef5e6f2bdcda59fd669a298ec137af9d],
> but that test case relies on changes in the controversial feature (added
> {{MergePolicy.findFullFlushMerges}}). I think it should be possible to make
> another test case show the bug without that controversial feature, and I am
> unsure why our existing randomized tests have not uncovered this yet ...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9405) IndexWriter incorrectly calls closeMergeReaders twice when the merged segment is 100% deleted
[
https://issues.apache.org/jira/browse/LUCENE-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136131#comment-17136131
]
ASF subversion and git services commented on LUCENE-9405:
-
Commit cb8b9a9cf4a9a7eb798bcfdb4abb2f0f68efb760 in lucene-solr's branch
refs/heads/branch_8x from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=cb8b9a9 ]
LUCENE-9405: Ensure IndexWriter only closes merge readers once. (#1580)
IndexWriter incorrectly calls closeMergeReaders twice when the
merged segment is 100% deleted ie. would produce a fully deleted
segment.
> IndexWriter incorrectly calls closeMergeReaders twice when the merged segment
> is 100% deleted
> -
>
> Key: LUCENE-9405
> URL: https://issues.apache.org/jira/browse/LUCENE-9405
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This is the first spinoff from a [controversial PR to add a new index-time
> feature to Lucene to merge small segments during
> commit|https://github.com/apache/lucene-solr/pull/1552]. This can
> substantially reduce the number of small index segments to search.
> See specifically [this discussion
> there|https://github.com/apache/lucene-solr/pull/1552#discussion_r440298695].
> {{IndexWriter}} seems to be missing a {{success = true}} inside
> {{mergeMiddle}} in the case where all segments being merged have 100%
> deletions and the segments will simply be dropped.
> In this case, in master today, I think we are incorrectly calling
> {{closeMergedReaders}} twice, first with {{suppressExceptions = false}} and
> second time with {{true}}.
> There is a [dedicated test case here showing the
> issue|https://github.com/apache/lucene-solr/commit/cab5ef5e6f2bdcda59fd669a298ec137af9d],
> but that test case relies on changes in the controversial feature (added
> {{MergePolicy.findFullFlushMerges}}). I think it should be possible to make
> another test case show the bug without that controversial feature, and I am
> unsure why our existing randomized tests have not uncovered this yet ...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9405) IndexWriter incorrectly calls closeMergeReaders twice when the merged segment is 100% deleted
[
https://issues.apache.org/jira/browse/LUCENE-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136130#comment-17136130
]
ASF subversion and git services commented on LUCENE-9405:
-
Commit 47cffbcdd8aa4895c32b0b7a64379fd9f6dd02d5 in lucene-solr's branch
refs/heads/master from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=47cffbc ]
LUCENE-9405: Ensure IndexWriter only closes merge readers once. (#1580)
IndexWriter incorrectly calls closeMergeReaders twice when the
merged segment is 100% deleted ie. would produce a fully deleted
segment.
> IndexWriter incorrectly calls closeMergeReaders twice when the merged segment
> is 100% deleted
> -
>
> Key: LUCENE-9405
> URL: https://issues.apache.org/jira/browse/LUCENE-9405
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This is the first spinoff from a [controversial PR to add a new index-time
> feature to Lucene to merge small segments during
> commit|https://github.com/apache/lucene-solr/pull/1552]. This can
> substantially reduce the number of small index segments to search.
> See specifically [this discussion
> there|https://github.com/apache/lucene-solr/pull/1552#discussion_r440298695].
> {{IndexWriter}} seems to be missing a {{success = true}} inside
> {{mergeMiddle}} in the case where all segments being merged have 100%
> deletions and the segments will simply be dropped.
> In this case, in master today, I think we are incorrectly calling
> {{closeMergedReaders}} twice, first with {{suppressExceptions = false}} and
> second time with {{true}}.
> There is a [dedicated test case here showing the
> issue|https://github.com/apache/lucene-solr/commit/cab5ef5e6f2bdcda59fd669a298ec137af9d],
> but that test case relies on changes in the controversial feature (added
> {{MergePolicy.findFullFlushMerges}}). I think it should be possible to make
> another test case show the bug without that controversial feature, and I am
> unsure why our existing randomized tests have not uncovered this yet ...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
