[
https://issues.apache.org/jira/browse/LUCENE-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084365#comment-17084365
]
Mikhail Khludnev commented on LUCENE-7701:
------------------------------------------
Hi, [~romseygeek], would you mind if I followup here? Turns out, if:
# {{group.truncate=true}}
# {{group.sort=docvalues_enabled_field asc}}
The following hotspot pops up:
{code}
"stackTrace":["org.apache.lucene.store.ByteBufferIndexInput.slice(ByteBufferIndexInput.java:268)",
"org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.slice(ByteBufferIndexInput.java:347)",
"org.apache.lucene.store.IndexInput.randomAccessSlice(IndexInput.java:122)",
"org.apache.lucene.codecs.lucene80.Lucene80DocValuesProducer$TermsDict.<init>(Lucene80DocValuesProducer.java:943)",
"org.apache.lucene.codecs.lucene80.Lucene80DocValuesProducer.getSorted(Lucene80DocValuesProducer.java:750)",
"org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.getSorted(PerFieldDocValuesFormat.java:329)",
"org.apache.lucene.index.DocValues.getSorted(DocValues.java:367)",
"org.apache.lucene.search.FieldComparator$TermOrdValComparator.getSortedDocValues(FieldComparator.java:709)",
"org.apache.lucene.search.FieldComparator$TermOrdValComparator.getLeafComparator(FieldComparator.java:714)",
"org.apache.lucene.search.grouping.AllGroupHeadsCollector$SortingGroupHead.<init>(AllGroupHeadsCollector.java:266)",
"org.apache.lucene.search.grouping.AllGroupHeadsCollector$SortingGroupHeadsCollector.newGroupHead(AllGroupHeadsCollector.java:250)",
"org.apache.lucene.search.grouping.AllGroupHeadsCollector.collect(AllGroupHeadsCollector.java:133)",
"org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:440)",
"org.apache.solr.search.grouping.CommandHandler.execute(CommandHandler.java:158)",
"org.apache.solr.handler.component.QueryComponent.doProcessGroupedDistributedSearchSecondPhase(QueryComponent.java:1399)",
"org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:386)",
{code}
I read it as follows, when group collector encounter new group it creates field
compactor for this value that opens DocValues that turns out to be a way more
expensive to open rather than old -good- {{FieldCache}}. I think DocValues
should somehow to be reused between groups. WDYT?
> Refactor grouping collectors
> ----------------------------
>
> Key: LUCENE-7701
> URL: https://issues.apache.org/jira/browse/LUCENE-7701
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Alan Woodward
> Priority: Major
> Fix For: 7.0
>
> Attachments: LUCENE-7701.patch, LUCENE-7701.patch
>
>
> Grouping currently works via abstract collectors, which need to be overridden
> for each way of defining a group - currently we have two, 'term' (based on
> SortedDocValues) and 'function' (based on ValueSources). These collectors
> all have a lot of repeated code, and means that if you want to implement your
> own group definitions, you need to override four or five different classes.
> This would be easier to deal with if instead the 'group selection' code was
> abstracted out into a single interface, and the various collectors were
> changed to concrete implementations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
