[
https://issues.apache.org/jira/browse/SOLR-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Khludnev updated SOLR-7730:
-----------------------------------
Description:
every time we count facets on DocValues fields in Solr on many segments index
we see the unnecessary hotspot:
{code}
....
at
org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:248)
at
org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
at
org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedSetDocValues(SlowCompositeReaderWrapper.java:176)
at
org.apache.solr.request.DocValuesFacets.getCounts(DocValuesFacets.java:72)
at
org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:460) ....
{code}
the reason is SlowCompositeReaderWrapper.getSortedSetDocValues() Line 136 and
SlowCompositeReaderWrapper.getSortedDocValues() Line 174
before return composite doc values, SCWR merges segment field infos, which is
expensive, but after fieldinfo is merged, it checks *only* docvalue type in it.
This dv type check can be done much easier in per segment basis.
This patch gets some performance gain for those who count DV facets in Solr.
was:
every time we count facets on DocValues fields in Solr on many segments index
we see the unnecessary hotspot:
{code}
....
at
org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:248)
at
org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
at
org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedSetDocValues(SlowCompositeReaderWrapper.java:176)
at
org.apache.solr.request.DocValuesFacets.getCounts(DocValuesFacets.java:72)
at
org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:460) ....
{code}
the reason is
[SCWR.getSortedSetDocValues()|https://github.com/apache/lucene-solr/blob/trunk/lucene/core/src/java/org/apache/lucene/index/SlowCompositeReaderWrapper.java#L136]
and
[SCWR.getSortedDocValues()|https://github.com/apache/lucene-solr/blob/trunk/lucene/core/src/java/org/apache/lucene/index/SlowCompositeReaderWrapper.java#L174]
before return composite doc values, SCWR merges segment field infos, which is
expensive, but after fieldinfo is merged, it checks *only* docvalue type in it.
This dv type check can be done much easier in per segment basis.
This patch gets some performance gain for those who count DV facets in Solr.
> speed-up faceting on doc values fields
> --------------------------------------
>
> Key: SOLR-7730
> URL: https://issues.apache.org/jira/browse/SOLR-7730
> Project: Solr
> Issue Type: Improvement
> Components: faceting
> Affects Versions: 5.2.1
> Reporter: Mikhail Khludnev
> Labels: patch
> Fix For: 5.3
>
> Attachments: LUCENE-7730.patch
>
>
> every time we count facets on DocValues fields in Solr on many segments index
> we see the unnecessary hotspot:
> {code}
> ....
> at
> org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:248)
> at
> org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
> at
> org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedSetDocValues(SlowCompositeReaderWrapper.java:176)
> at
> org.apache.solr.request.DocValuesFacets.getCounts(DocValuesFacets.java:72)
> at
> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:460) ....
> {code}
> the reason is SlowCompositeReaderWrapper.getSortedSetDocValues() Line 136 and
> SlowCompositeReaderWrapper.getSortedDocValues() Line 174
> before return composite doc values, SCWR merges segment field infos, which is
> expensive, but after fieldinfo is merged, it checks *only* docvalue type in
> it. This dv type check can be done much easier in per segment basis.
> This patch gets some performance gain for those who count DV facets in Solr.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
