[
https://issues.apache.org/jira/browse/SOLR-12572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16566117#comment-16566117
]
Varun Thacker edited comment on SOLR-12572 at 8/1/18 10:48 PM:
---------------------------------------------------------------
Hi Amrit,
I've taken your last patch and cleaned it up a little. I think the patch is
looking in good shape.
Will run some tests for correctness on this patch later today. Let's capture
some perf numbers
I'm thinking of indexing 25M docs with just "id" and then executing this query
: {{q=match_all&sort=id desc&fl=id}} with and without the patch. This will
test when a field is reused in sort and fl how much speed improvement do we
get. If the numbers look good I'd imagine more sort and fl fields will bring
larger improvements.
was (Author: varunthacker):
Hi Amrit,
I've taken your last patch and cleaned it up a little. I think the patch is
looking in good shape.
Will run some tests for correctness on this patch later today. Let's capture
some perf numbers
I'm thinking of indexing 25M docs with just "id" and then executing this query
: {{q=match_all&sort=id desc&fl=id}} with and without the patch. This will
test us when a field is reused in sort and fl how much speed improvement do we
get. If the numbers look good I'd imagine more sort and fl fields will bring
more improvements.
> Reuse fieldvalues computed while sorting at writing in ExportWriter
> -------------------------------------------------------------------
>
> Key: SOLR-12572
> URL: https://issues.apache.org/jira/browse/SOLR-12572
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: streaming expressions
> Reporter: Amrit Sarkar
> Assignee: Varun Thacker
> Priority: Minor
> Attachments: SOLR-12572.patch, SOLR-12572.patch, SOLR-12572.patch,
> SOLR-12572.patch, SOLR-12572.patch, SOLR-12572.patch
>
>
> While exporting result through "/export" handler,
> {code:java}
> http://localhost:8983/solr/core_name/export?q=my-query&sort=severity+desc,timestamp+desc&fl=severity,timestamp,msg
> {code}
> Doc-values are sought for all the {{sort}} fields defined (in this example
> 'severity, 'timestamp'). When we stream out docs we again make doc-value
> seeks against the {{fl}} fields ('severity','timestamp','msg') .
> In most common use-cases we have {{fl = sort}} fields, or atleast the sort
> fields are subset of {{fl}} fields, so if we can *pre-collect* the values
> while sorting it, we can reduce the doc-value seeks potentially bringing
> *speed improvement*.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
