[
https://issues.apache.org/jira/browse/SOLR-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733690#comment-13733690
]
Gun Akkor commented on SOLR-2548:
-
I would like to revive this ticket, if possible. We have an index with about 10
fields that we regularly facet on. These fields are either multi-valued or are
of type TextField, so facet code chooses FC as the facet method, and uses the
UnInvertedField instances to count each facet field, which takes several
seconds per field in our case. So, multi-thread execution of getTermCounts()
reduces the overall facet time considerably.
I started with the patch that was posted against 3.1 and modified it a little
bit to take into account previous comments made by Yonik and Adrien. The new
patch applies against 4.2.1, uses the already existing facetExecutor thread
pool, and is configured per request via a facet.threads request param. If the
param is not supplied, the code defaults to directExecutor and runs sequential
as before. So, code should behave as is if user chooses not to submit number of
threads to use.
Also in the process of testing, I noticed that
UnInvertedField.getUnInvertedField() call was synchronized too early, before
the call to new UnInvertedField(field, searcher) if the field is not in the
field value cache. Because its init can take several seconds, synchronizing on
the cache in that duration was effectively serializing the execution of the
multiple threads.
So, I modified it (albeit inelegantly) to synchronize later (in our case cache
hit ratio is low, so this makes a difference).
The patch is still incomplete, as it does not extend this framework to possibly
other calls like ranges and dates, but it is a start.
Multithreaded faceting
--
Key: SOLR-2548
URL: https://issues.apache.org/jira/browse/SOLR-2548
Project: Solr
Issue Type: Improvement
Components: search
Affects Versions: 3.1
Reporter: Janne Majaranta
Priority: Minor
Labels: facet
Attachments: SOLR-2548_for_31x.patch, SOLR-2548.patch
Add multithreading support for faceting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
