[ https://issues.apache.org/jira/browse/LUCENE-8253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440943#comment-16440943 ]
ASF subversion and git services commented on LUCENE-8253: --------------------------------------------------------- Commit d904112428184ce9c1726313add5d184f4014a72 in lucene-solr's branch refs/heads/master from [~simonw] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d904112 ] LUCENE-8253: Don't create ReadersAndUpdates for foreign segments IndexWriter#numDeletesToMerge was creating a ReadersAndUpdates for all incoming SegmentCommitInfo even if that info wasn't private to the IndexWriter. This is an illegal use of this API but since it's transitively public via MergePolicy#findMerges we have to be conservative with regestiering ReadersAndUpdates. In IndexWriter#numDeletesToMerge we can only use existing ones. This means for soft-deletes we need to react earlier in order to produce accurate numbers. This change partially rolls back the changes in LUCENE-8253. Instead of registering the readers once they are pulled via IndexWriter#numDeletesToMerge we now check if segments are fully deleted on flush which is very unlikely and can be done in a lazy fashion ie. it's only paying the extra cost of opening a reader and checking all soft-deletes if soft deletes are used and present in the flushed segment. This has the side-effect that flushed segments that are 100% hard deleted are also cleaned up right after they are flushed, previously these segments were sticking around for a while until they got picked for a merge or received another delete. This also closes LUCENE-8256 > ForceMergeDeletes does not merge soft-deleted segments > ------------------------------------------------------ > > Key: LUCENE-8253 > URL: https://issues.apache.org/jira/browse/LUCENE-8253 > Project: Lucene - Core > Issue Type: Bug > Affects Versions: 7.4, master (8.0) > Reporter: Nhat Nguyen > Assignee: Simon Willnauer > Priority: Major > Attachments: LUCENE-8253.patch, test-merge.patch > > > IndexWriter#forceMergeDeletes should merge segments having soft-deleted > documents as hard-deleted documents if we configured "softDeletesField" in an > IndexWriterConfig. > Attached is a failed test. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org