[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604724#comment-16604724 ] ASF subversion and git services commented on SOLR-12625: Commit 712aa631beeb324c1931e9a0bd7de0b6da10df3c in lucene-solr's branch refs/heads/branch_7x from [~ctargett] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=712aa63 ] SOLR-12625: fix typos..."an" -> "and" > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Fix For: master (8.0), 7.5 > > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch, > SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604719#comment-16604719 ] ASF subversion and git services commented on SOLR-12625: Commit 4f0558800786c087391e04828d5e38d7ca7693dc in lucene-solr's branch refs/heads/master from [~ctargett] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4f05588 ] SOLR-12625: fix typos..."an" -> "and" > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Fix For: master (8.0), 7.5 > > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch, > SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586763#comment-16586763 ] ASF subversion and git services commented on SOLR-12625: Commit 66d500b5a59e1aefe9170e8c5cb70a9e0b0f1033 in lucene-solr's branch refs/heads/jira/http2 from [~cp.erick...@gmail.com] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=66d500b ] SOLR-12625: Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Fix For: master (8.0), 7.5 > > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch, > SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586282#comment-16586282 ] ASF subversion and git services commented on SOLR-12625: Commit 82c64af84b903df40e457ed6e338b3abf43a7534 in lucene-solr's branch refs/heads/branch_7x from [~cp.erick...@gmail.com] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=82c64af ] SOLR-12625: Combine SolrDocumentFetcher and RetrieveFieldsOptimizer (cherry picked from commit 66d500b5a59e1aefe9170e8c5cb70a9e0b0f1033) > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Fix For: master (8.0), 7.5 > > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch, > SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586278#comment-16586278 ] ASF subversion and git services commented on SOLR-12625: Commit 66d500b5a59e1aefe9170e8c5cb70a9e0b0f1033 in lucene-solr's branch refs/heads/master from [~cp.erick...@gmail.com] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=66d500b ] SOLR-12625: Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch, > SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16584599#comment-16584599
]
Lucene/Solr QA commented on SOLR-12625:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate ref guide {color} |
{color:green} 1m 0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 40m
30s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-12625 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12936083/SOLR-12625.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns validaterefguide |
| uname | Linux lucene1-us-west 4.4.0-130-generic #156~14.04.1-Ubuntu SMP Thu
Jun 14 13:51:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 9f615fb |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 |
| Default Java | 1.8.0_172 |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/164/testReport/ |
| modules | C: solr solr/core solr/solr-ref-guide U: solr |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/164/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch,
> SOLR-12625.patch, SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16584362#comment-16584362 ] Erick Erickson commented on SOLR-12625: --- Final patch unless there are objections. This patch differs from the last one only in that the test now fires of several threads in order to insure that the document fetcher is not inappropriately re-using the fetch optimizer. Will commit over the weekend. > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch, > SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583188#comment-16583188
]
Lucene/Solr QA commented on SOLR-12625:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 4m 6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 4m 6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 4m 6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 24s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.cdcr.CdcrBidirectionalTest |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-12625 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935902/SOLR-12625.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 585ba16 |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | 1.8.0_172 |
| unit |
https://builds.apache.org/job/PreCommit-SOLR-Build/163/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/163/testReport/ |
| modules | C: solr solr/core U: solr |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/163/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch,
> SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582871#comment-16582871 ] Erick Erickson commented on SOLR-12625: --- Latest version incorporating David's comments and general clean-up as well as _finally_ figuring out that Trie fields are SORTED_SET and Point fields are SORTED_NUMERIC and making that adjustment in the test code. Pending objections, I'll probably commit this this weekend. The test has considerable randomization in it, one of the relatively rare problems was duplicate values so I also added explicit duplicate values in the fields to insure coverage there. 1,000 beast iterations last night and all is well. I'll hit it again tonight just in case my "minor cosmetic rearrangements" this morning foo'd something. > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch, > SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581324#comment-16581324 ] Erick Erickson commented on SOLR-12625: --- [~dsmiley] Thanks! A lot of things moved around as I was getting it all to work and there's cruft laying around, thanks for pointing it out. I'll fix those. And look at it all again after my eyes rest a bit ;) The test kind of grew into an ungainly beast. There's a lot of work just to get the schema in place, then not a lot of code to test. Basically it's: * use a lot of code setting up a field type for all the kinds of fields I want to test. * use a little code to populate a few documents with the data (one weakness is it's the same data for each doc, I'll look at that). * spend not much code at all insuring that the fields ** were retrieved from the expected place ** are the values expected And not I have a better idea what's useful if I ever get time to go forward with SOLR-10229. > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581158#comment-16581158 ] David Smiley commented on SOLR-12625: - Little review * Overall I'm loving it * RetrieveFieldsOptimizer is now an inner class but you either forgot to declare it static, or you forgot to remove the now pointless argument to the SolrDocumentFetcher within it since it will already have access to the doc fetcher. * I like the Supplier use * Wow there's a lot to that test. A minor point: those classes you created local to the test (e.g. FieldHolder) could be simpler by forgoing JavaBean conventions, as they are internal so just use field access. > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580708#comment-16580708 ] Erick Erickson commented on SOLR-12625: --- Latest iteration, with unit tests (which is what I've been spending most of my time on in the past few days). I'm experimenting with the tests on how easy/difficult it would be to just use the mutable schemas rather than continue the proliferation of schemas, so some of the code in the test may move elsewhere, but in a future JIRA. Not final yet, there are several TODOs and the like. Do note that the unit tests showed some bugs in the optimizer that I fixed. Any feedback welcome. > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12625.patch, SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577764#comment-16577764 ] Erick Erickson commented on SOLR-12625: --- Nikolay: Thanks for pointing this out. I'm in the middle of rearranging RetrieveFieldsOptimizer and writing tests that exercise the optimization process, especially in terms of determining that fields are fetched from the expected places, I'll incorporate your fix into anything I find. > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[ https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575754#comment-16575754 ] David Smiley commented on SOLR-12625: - Sounds good. Looking forward to the next patch. This'll be a nice step forward. > Combine SolrDocumentFetcher and RetrieveFieldsOptimizer > --- > > Key: SOLR-12625 > URL: https://issues.apache.org/jira/browse/SOLR-12625 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12625.patch, SOLR-12625.patch > > > We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The > relationship between the two is unclear at first glance. Using > SolrDocumentFetcher by itself is (or can be) inefficient. > WDYT about combining the two? Is there a good reason you would want to > use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer? > Ideally I'd want to be able to write code like: > solrDocumentFetcher.fillDocValuesMostEfficiently > That created an optimizer and "did the right thing". > Assigning to myself to keep track, but if anyone feels motivated feel free to > take it over. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575481#comment-16575481
]
Erick Erickson commented on SOLR-12625:
---
{quote}Can you give a couple examples of what the caller is
{quote}
I only have a single case at the moment in TermVectorComponent. I'm going to
give up returning any kind of transformed SolrDocument as part of this JIRA. We
can raise a separate Jira if we want to add in a to-native option.
I think just combining these two classes is enough here and the
"simplification" of having the SolrDocument come back with specific types isn't
useful enough outside of this one case to make it worth spending more time on.
I do wonder whether we can re-use/abstract/whatever some of the writer code
somehow, but don't want to go there now.
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch, SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575213#comment-16575213
]
David Smiley commented on SOLR-12625:
-
{quote}1> making the code convert to native types and have a getValAsString()
method that at least conceals the above ugliness when the user doesn't care?
{quote}
+1... maybe warn that the native type might actually be a number or date,
therefore this method might do a conversion.
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch, SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575035#comment-16575035
]
Erick Erickson commented on SOLR-12625:
---
{quote}Per chance TestChildDocTransformer?
{quote}
Yeah, and the latest iteration has fails in three Transformers
{quote}It'd be nice if SolrReturnFields didn't itself have to reference
{quote}
Putting it in SolrReturnFields was super-ugly, either a bidirectional reference
which I dislike too or I would have had to pass 4 different lists , neither of
which I could stand. Having just a set/get for the optimizer isn't great
either, but it's better.
{quote}First, why just Strings when the type might be something else like a
Date?
{quote}
What does the calling code look like then? I have this SolrDocument and I want
to get "field1". Do I have code like
{code:java}
if (val instanceof String) {
String eoeS = (String)val;
} else if (val instanceof Date) {
Date eoeD = (Date)val;
} etc?
{code}
That's its own kind of ugly.
or
{code:java}
if (fieldType instanceof IntValueFieldType) {
if (multiValued) {
writers[i] = new MultiFieldWriter(field, fieldType, schemaField,
true);
} else {
writers[i] = new IntFieldWriter(field);
}
} else if (fieldType instanceof LongValueFieldType) {
if (multiValued) {
writers[i] = new MultiFieldWriter(field, fieldType, schemaField,
true);
} else {
writers[i] = new LongFieldWriter(field);
}
} else if (fieldType instanceof FloatValueFieldType) {
etc
{code}
That said, it's another kind of ugly to have code that parses the string that's
been automagically converted.
WDYT about
1> making the code convert to native types and have a getValAsString() method
that at least conceals the above ugliness when the user doesn't care?
2> forgetting the boolean for SolrDocumentFetcher.solrDoc, just convert to
native types regardless. I don't really see a good purpose to be served by
allowing the option here. The goal is to make it simpler to use so one less
option seems good.
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch, SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574303#comment-16574303
]
Erick Erickson commented on SOLR-12625:
---
OK, here's another cut. It's still a bit inelegant, it just stuffs a
RetrieveFieldsOptimizer into SolrReturnFields. At least that's concealed from
most users though and the initialization is done only once per use of srf.
I'm not all that crazy about having to use s SolrReturnFields rather than
ReturnFields in some places, but don't really see a lightweight alternative.
{{Putting getValuesAsStrings(fieldName)}}
over in SolrDocument doesn't work either as that would require a dependency on
luceneCore for the "StoredField" class.
So this gets rid of the opaque Object and still only initializes the
RetrieveFieldsOptimizer once.
It also implements your suggestion to tell the proposed solrDoc to convert all
the fields to Strings up-front.
I haven't tested this very much yet, putting it up for perusal.
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574271#comment-16574271
]
David Smiley commented on SOLR-12625:
-
bq. docFetcher is in the SolrIndexSearcher and can interleave docs, fetching
different fields. This came to light because one of the tests fetches children
(IIRC) which means there's a different set of fields to be retrieved on
subsequent calls.
Interesting; which test is this? Per chance TestChildDocTransformer? Any way,
I don't imagine this would be a problem since if you want different fields for
children, then you could simply use a different SolrReturnFields, and so
there's no mismatch.
bq. Are you thinking of putting the RetrieveFieldsOptimizer in SolrReturnFields
and Moving the class over there?
I didn't completely think that through yet. Eh... it's debatable. It'd be
nice if SolrReturnFields didn't itself have to reference anything in
SolrDocumentFetcher since that would create a bidirectional reference which I'd
like to avoid as a matter of design taste. So therefore I think most of the
decision logic should go in SolrDocumentFetcher in some form. I've always
found SolrReturnFields to be a bit confusing so I don't love adding stuff to
make it more complex but if these two field lists are clearly documented (to
include when and who sets them) then it's okay I think. Maybe names like
"fetchStoredFields" and "fetchDocValueFields".
bq. How about just put the method over in SolrDocument like {{String[]
getValuesAsStrings(fieldName)}}? Seems cleaner.
First, why just Strings when the type might be something else like a Date? So
there's that. Then... I don't really like values like Lucene IndexableField
winding up as-is on a SolrDocument. It's a SolrDocument, not a Lucene
Document. But I understand the motivation: it was *a* way (though not the only
possible way) to have lazy loading as late as possible. But this has led to
some clumsy gotchas with consumers needing to be aware they have to deal with
this Lucene IndexableField value when they just want a String (or a Date etc.).
I bet the lazy load aspect might be better implemented as another subclass of
SolrDocument or in SolrDocument itself, so that the caller doesn't have to even
be aware. But that seems out of scope of this issue. My proposed boolean flag
would be minor enough, though.
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573982#comment-16573982
]
Erick Erickson commented on SOLR-12625:
---
Thanks for looking at this!
{quote}I think we can do better
{quote}
I sure hope so
{quote}
...Object just feels clumsy to me... The call to {{solrDoc}} would inspect the
SolrReturnFields to see if it has the computed information that is today
present on RetrieveFieldsOptimizer, and if not then compute that...The call to
{{solrDoc}} would inspect the SolrReturnFields to see if it has the computed
information that is today present on RetrieveFieldsOptimizer...
{quote}
Me too. Which is one reason I posted it as a PoC so you could generate better
ideas ;).
docFetcher is in the SolrIndexSearcher and can interleave docs, fetching
different fields. This came to light because one of the tests fetches children
(IIRC) which means there's a different set of fields to be retrieved on
subsequent calls. One of the things I also want to do is write a test that has
multiple threads querying for different field lists to stress this kind of
thing.
Are you thinking of putting the RetrieveFieldsOptimizer in SolrReturnFields and
Moving the class over there?
{quote}Perhaps we could add a boolean to the proposed solrDoc
{quote}
How about just put the method over in SolrDocument like
{{String[] getValuesAsStrings(fieldName)?}} Seems cleaner.
{quote}Your change to SolrDocument appears purely code formatting; true?
{quote}
Rats. I've been having trouble with my new machine and getting IntelliJ to
really _not_ screw around with lines that haven't been changed, looks like I
have more work to do there. Yep, purely formatting, I'll revert.
Thanks again.
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573893#comment-16573893
]
David Smiley commented on SOLR-12625:
-
Thanks for filing this Erick. I like some of what you did but I think we can
do better. The method docFetcher.optimizeForFetchingTheseFields that is
declared to return Object just feels clumsy to me. What if instead we use
ReturnFields (== SolrReturnFields) to hold this metadata on how some fields
should be returned? In this scheme, the caller would quite simply call
{{docFetcher.solrDoc(returnFields)}}. The call to {{solrDoc}} would inspect
the SolrReturnFields to see if it has the computed information that is today
present on RetrieveFieldsOptimizer, and if not then compute that. Subsequent
calls will find it and it won't need to be re-computed.
RE getValsAsString: I hear ya; it's too complex. Again; thanks for filing this
issue -- I love cleanups. Perhaps we could add a boolean to the proposed
{{solrDoc(...)}} method to declare if the values should be resolved using their
field types to appropriate objects (be it String or Number subclass or Date,
etc.). WDYT?
Your change to SolrDocument appears purely code formatting; true?
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12625) Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
[
https://issues.apache.org/jira/browse/SOLR-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572134#comment-16572134
]
Erick Erickson commented on SOLR-12625:
---
Here's a PoC, it has nocommits etc. [~dsmiley] and [~caomanhdat2] I'm
particularly interested in whether you think it is a good approach.
Really I hacked together something quickly to see if it makes architectural
sense. In this version, the pattern to use this is:
{code:java}
Object fetchOptimizer = docFetcher.optimizeForFetchingTheseFields(srf);
while (more docs) {
List vals =
docFetcher.getValsAsStrings(docFetcher.docOptimizedFetch(docId,
fetchOptimizer), uniqFieldName);
// do whatever you want with the returned values. Single-valued fields
will return a 1-element list.
}
{code}
Notes:
Since SolrDocumentFetcher is reused I elected to require that the callers get
back an object (intended to be opaque) that's currently a
RetrieveFieldsOptimizer that then needs to be passed in to fetch the actual
document. It's a little clumsy, but...
The names I've given the new methods are clumsy, suggestions welcome. I do want
something to indicate that we want to get things in an optimized fashion to
draw attention to it though.
Finally, I find the necessity of having _calling_ code try to resolve objects
error-prone and unnecessarily complex when you just want a value, man. So I
added
{code:java}
public List getValsAsStrings(SolrDocument sdoc, String field) {
{code}
to SolrDocumentFetcher. It does leave it up to the caller to make the
distinction between single-valued and multi-valued fields when it's important
though.
I'll look this all over several more times before it's ready to commit of
course, but this is the first thing that I tried that seems to work. All tests
pass (but I haven't run precommit yet).
Speaking of tests, I've commented everything in *TestRetrieveFieldsOptimizer*
since this is a PoC, if I go forward with this I'll have to figure out how to
test this. I'm not quite sure how to determine that we really do only get
values from DV fields or only stored fields or a combination as appropriate. We
can, of course, figure out that we get the results back in the docs, but that
doesn't tell us where we get them _from_, stored or DV.
> Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
> ---
>
> Key: SOLR-12625
> URL: https://issues.apache.org/jira/browse/SOLR-12625
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-12625.patch
>
>
> We have SolrDocumentFetcher and RetrieveFieldsOptimizer. The
> relationship between the two is unclear at first glance. Using
> SolrDocumentFetcher by itself is (or can be) inefficient.
> WDYT about combining the two? Is there a good reason you would want to
> use SolrDocumentFetcher _instead_ of RetrieveFieldsOptimizer?
> Ideally I'd want to be able to write code like:
> solrDocumentFetcher.fillDocValuesMostEfficiently
> That created an optimizer and "did the right thing".
> Assigning to myself to keep track, but if anyone feels motivated feel free to
> take it over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
