[jira] [Updated] (SOLR-11891) BinaryResponseWriter fetches unnecessary fields
[ https://issues.apache.org/jira/browse/SOLR-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated SOLR-11891: Attachment: SOLR-11891.patch > BinaryResponseWriter fetches unnecessary fields > --- > > Key: SOLR-11891 > URL: https://issues.apache.org/jira/browse/SOLR-11891 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Response Writers >Affects Versions: 5.4, 6.4.2, 6.6.2 >Reporter: wei wang >Priority: Major > Attachments: DocsStreamer.java.diff, SOLR-11891.patch, > SOLR-11891.patch.BAD > > > We observe that solr query time increases significantly with the number of > rows requested, even all we retrieve for each document is just fl=id,score. > Debugged a bit and see that most of the increased time was spent in > BinaryResponseWriter, converting lucene document into SolrDocument. Inside > convertLuceneDocToSolrDoc(): > [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L182] > > I am a bit puzzled why we need to iterate through all the fields in the > document. Why can’t we just iterate through the requested field list? > [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L156] > > e.g. when pass in the field list as > sdoc = convertLuceneDocToSolrDoc(doc, rctx.getSearcher().getSchema(), fnames) > and just iterate through fnames, there is a significant performance boost in > our case. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11891) BinaryResponseWriter fetches unnecessary fields
[ https://issues.apache.org/jira/browse/SOLR-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated SOLR-11891: Attachment: SOLR-11891.patch.BAD > BinaryResponseWriter fetches unnecessary fields > --- > > Key: SOLR-11891 > URL: https://issues.apache.org/jira/browse/SOLR-11891 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Response Writers >Affects Versions: 5.4, 6.4.2, 6.6.2 >Reporter: wei wang >Priority: Major > Attachments: DocsStreamer.java.diff, SOLR-11891.patch.BAD > > > We observe that solr query time increases significantly with the number of > rows requested, even all we retrieve for each document is just fl=id,score. > Debugged a bit and see that most of the increased time was spent in > BinaryResponseWriter, converting lucene document into SolrDocument. Inside > convertLuceneDocToSolrDoc(): > [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L182] > > I am a bit puzzled why we need to iterate through all the fields in the > document. Why can’t we just iterate through the requested field list? > [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L156] > > e.g. when pass in the field list as > sdoc = convertLuceneDocToSolrDoc(doc, rctx.getSearcher().getSchema(), fnames) > and just iterate through fnames, there is a significant performance boost in > our case. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11891) BinaryResponseWriter fetches unnecessary fields
[ https://issues.apache.org/jira/browse/SOLR-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wei wang updated SOLR-11891: Attachment: DocsStreamer.java.diff > BinaryResponseWriter fetches unnecessary fields > --- > > Key: SOLR-11891 > URL: https://issues.apache.org/jira/browse/SOLR-11891 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Response Writers >Affects Versions: 5.4, 6.4.2, 6.6.2 >Reporter: wei wang >Priority: Major > Attachments: DocsStreamer.java.diff > > > We observe that solr query time increases significantly with the number of > rows requested, even all we retrieve for each document is just fl=id,score. > Debugged a bit and see that most of the increased time was spent in > BinaryResponseWriter, converting lucene document into SolrDocument. Inside > convertLuceneDocToSolrDoc(): > [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L182] > > I am a bit puzzled why we need to iterate through all the fields in the > document. Why can’t we just iterate through the requested field list? > [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L156] > > e.g. when pass in the field list as > sdoc = convertLuceneDocToSolrDoc(doc, rctx.getSearcher().getSchema(), fnames) > and just iterate through fnames, there is a significant performance boost in > our case. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
