mayya-sharipova edited a comment on issue #1351: LUCENE-9280: Collectors to skip noncompetitive documents URL: https://github.com/apache/lucene-solr/pull/1351#issuecomment-614931785 I have run another round of benchmarks, this time comparing the performance of this PR VS master as we don't need any special sort field. [Here](https://github.com/mayya-sharipova/luceneutil/commit/c3166e4fc44e7fcddcd1672112c96364d9f464e5) are the changes made to luceneutil. **wikimedium10m**: 10 millon docs ``` TaskQPS baseline StdDevQPS patch StdDev Pct diff HighTermDayOfYearSort 50.93 (5.6%) 49.31 (10.9%) -3.2% ( -18% - 14%) TermDTSort 83.37 (5.9%) 129.95 (41.2%) 55.9% ( 8% - 109%) WARNING: cat=HighTermDayOfYearSort: hit counts differ: 541957 vs 541957+ WARNING: cat=TermDTSort: hit counts differ: 506054 vs 1861+ ``` **wikimediumall**: about 33 million docs ``` TaskQPS baseline StdDevQPS patch StdDev Pct diff HighTermDayOfYearSort 23.37 (4.4%) 21.76 (8.8%) -6.9% ( -19% - 6%) TermDTSort 31.86 (3.5%) 108.33 (49.6%) 240.0% ( 180% - 303%) WARNING: cat=HighTermDayOfYearSort: hit counts differ: 1275574 vs 1275574+ WARNING: cat=TermDTSort: hit counts differ: 1474717 vs 1070+ ``` Here we have two sorts: - Int sort on a day of year. Slight decrease of performance: **-6.9% – -3.2%,**. There was an attempt to do the optimization, but the optimization was eventually not run as every time [estimatedNumberOfMatches](https://github.com/apache/lucene-solr/pull/1351/files#diff-aff67e212aa0edd675ec31c068cb642bR268) was not selective enough. The reason for that the data here a day of the year in the range [1, 366], and all segments contain various values through a segment, so this data is not really a target for optimization. - Long sort on date field (msecSinceEpoch). Speedups: **55.9% – 240.0%**.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
