[
https://issues.apache.org/jira/browse/SOLR-17841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18059178#comment-18059178
]
Renato Haeberli commented on SOLR-17841:
----------------------------------------
[~epugh] [~ab]
I did some changes to ExitableDirectoryReaderSearch, in order to do multiple
runs, with and without MultiThreadedSearcher. I also added different queries to
test. Here the version I was using:
[https://github.com/renatoh/solr/blob/SOLR-1784-Investigate_multithreade_search_performance_bottlenecks/solr/benchmark/src/java/org/apache/solr/bench/search/ExitableDirectoryReaderSearch.java]
I could not find any scenario, even with 50M documents in the index and a
force-merge to 5 within the test setup, where MultiThreadedSearcher is faster.
For range searches, test 'testRangeQuery', the performance of
MultiThreadedSearcher is very poor, roughly 5 times slower.
The profiling [^profile.html] points to following call stack:
{code:java}
java.lang.Thread.State: RUNNABLE
at
org.apache.lucene.codecs.lucene103.Lucene103PostingsReader$BlockPostingsEnum.bufferIntoBitSet(Lucene103PostingsReader.java:1104)
at
org.apache.lucene.codecs.lucene103.Lucene103PostingsReader$BlockPostingsEnum.intoBitSet(Lucene103PostingsReader.java:1002)
at org.apache.lucene.util.FixedBitSet.or(FixedBitSet.java:380)
at org.apache.lucene.util.DocIdSetBuilder.add(DocIdSetBuilder.java:201)
at
org.apache.solr.query.SolrRangeQuery$ConstWeight.getSegState(SolrRangeQuery.java:521)
at
org.apache.solr.query.SolrRangeQuery$ConstWeight.scorerInternal(SolrRangeQuery.java:542)
at
org.apache.solr.query.SolrRangeQuery$ConstWeight.scorerSupplier(SolrRangeQuery.java:562)
at
org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:290)
at org.apache.lucene.search.IndexSearcher.searchLeaf(IndexSearcher.java:839)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:809)
at
org.apache.lucene.search.IndexSearcher.lambda$search$1(IndexSearcher.java:779)
at
org.apache.lucene.search.IndexSearcher$$Lambda/0x0000007801a26328.call(Unknown
Source)
at java.util.concurrent.FutureTask.run([email protected]/FutureTask.java:317)
at org.apache.lucene.search.TaskExecutor$Task.run(TaskExecutor.java:173)
at org.apache.lucene.search.TaskExecutor.invokeAll(TaskExecutor.java:111)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:783)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:750)
at
org.apache.solr.search.MultiThreadedSearcher.searchCollectorManagers(MultiThreadedSearcher.java:98)
{code}
> Investigate multithreaded search performance bottlenecks
> --------------------------------------------------------
>
> Key: SOLR-17841
> URL: https://issues.apache.org/jira/browse/SOLR-17841
> Project: Solr
> Issue Type: Improvement
> Reporter: Ishan Chattopadhyaya
> Priority: Blocker
> Fix For: 10.0
>
> Attachments: profile.html
>
>
> Ever since multithreaded search (SOLR-13350) was introduced, there has been
> not much effort to benchmark the performance benefit or investigate the
> performance bottlenecks. Anecdotally, it is not faster than single threaded
> execution.
> Thanks to [~kevinliang01] [~liangkaiwen] [~liangkw16] and [~dsmiley] for
> bringing this up, yesterday on a dense vector working group meeting.
> I'm marking this as a release 10.0 blocker, because it is embarrassing to be
> releasing with such a gaping hole.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
