[
https://issues.apache.org/jira/browse/LUCENE-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708993#action_12708993
]
Michael McCandless commented on LUCENE-1634:
--------------------------------------------
bq. As the API current stands, I think the behavior is to assume a subclass of
LogMergePolicy.
Hmm -- that's not intended. What goes wrong if you use a custom MergePolicy
that's not a LogMergePolicy?
(There are utility methods on IndexWriter, eg set/getUseCompoundFile, etc.,
that will only work if your MergePolicy is a LogMergePolicy, but that shouldn't
prevent you from using a non-LogMergePolicy as long as you don't use these
utility methods).
> LogMergePolicy should use the number of deleted docs when deciding which
> segments to merge
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-1634
> URL: https://issues.apache.org/jira/browse/LUCENE-1634
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Yasuhiro Matsuda
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1634.patch
>
>
> I found that IndexWriter.optimize(int) method does not pick up large segments
> with a lot of deletes even when most of the docs are deleted. And the
> existence of such segments affected the query performance significantly.
> I created an index with 1 million docs, then went over all docs and updated a
> few thousand at a time. I ran optimize(20) occasionally. What saw were large
> segments with most of docs deleted. Although these segments did not have
> valid docs they remained in the directory for a very long time until more
> segments with comparable or bigger sizes were created.
> This is because LogMergePolicy.findMergeForOptimize uses the size of segments
> but does not take the number of deleted documents into consideration when it
> decides which segments to merge. So, a simple fix is to use the delete count
> to calibrate the segment size. I can create a patch for this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
