jimczi commented on a change in pull request #32:
URL: https://github.com/apache/lucene/pull/32#discussion_r599954418



##########
File path: lucene/core/src/java/org/apache/lucene/index/DirectoryReader.java
##########
@@ -56,7 +57,24 @@
    * @throws IOException if there is a low-level IO error
    */
   public static DirectoryReader open(final Directory directory) throws 
IOException {
-    return StandardDirectoryReader.open(directory, null);
+    return StandardDirectoryReader.open(directory, null, null);
+  }
+
+  /**
+   * Returns a IndexReader for the the index in the given Directory
+   *
+   * @param directory the index directory
+   * @param leafSorter a comparator for sorting leaf readers. Providing 
leafSorter is useful for
+   *     indices on which it is expected to run many queries with particular 
sort criteria (e.g. for
+   *     time-based indices this is usually a descending sort on timestamp). 
In this case {@code
+   *     leafSorter} should sort leaves according to this sort criteria. 
Providing leafSorter allows
+   *     to speed up this particular type of sort queries by early terminating 
while iterating
+   *     through segments and segments' documents.

Review comment:
       It seems to work well in practice with the tiered merge policy and 
natural merging. The natural order of insertion seems to be preserved nicely in 
the levels. Things get complicated when sorting on descending order though. The 
big segments tend to have the data in ascending order. So in this scenario, 
it's much slower to retrieve the latest documents if the index was force merged 
to 1 segment or few big ones. In Elasticsearch we shuffle the order of the 
segments on force merge to alleviate this bias. 
   In this PR we opted for a single sort per index so I like the idea of having 
a merge policy that is optimized for this use case. I think it's a good 
compromise for cases where index sorting is too costly at the document level.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to