[
https://issues.apache.org/jira/browse/LUCENE-10125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421269#comment-17421269
]
Adrien Grand commented on LUCENE-10125:
---------------------------------------
The slowdown for stored fields pointed out this change where we optimized bulk
merges for sorted indices: https://github.com/apache/lucene/pull/134. It looks
like only semi-optimized merges are affected, ie. when the input readers are
considered too dirty and we copy encoded data but still have to recompress the
data. This makes sense to me since the new approach has a bit more overhead: it
consults the DocIdMerger on every doc, etc.
> Investigate indexing throughput regression on NYC Taxis between 2021-04-12
> and 2021-05-24
> -----------------------------------------------------------------------------------------
>
> Key: LUCENE-10125
> URL: https://issues.apache.org/jira/browse/LUCENE-10125
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Adrien Grand
> Assignee: Adrien Grand
> Priority: Minor
> Attachments: LUCENE-10125_hack.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> There's been a significant drop in indexing throughput between 2021-04-12 and
> 2021-05-24 on the NYC Taxis benchmark. Unfortunately several suspects have
> been merged during that period of time so we might need to git bisect to
> figure out which one is responsible for the regression. Interestingly the
> sorted index looks less affected than the non-sorted indexes.
> https://home.apache.org/~mikemccand/lucenebench/sparseResults.html#index_throughput
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
