[jira] [Commented] (SOLR-6813) distrib.singlePass does not work for expand-request - start/rows included
[
https://issues.apache.org/jira/browse/SOLR-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14247963#comment-14247963
]
Per Steffensen commented on SOLR-6813:
--
I would prefer making it work instead of saying that single-pass in not
supported for expand-queries. But I guess that is easy for me to say :-)
If I understand you explanations correctly, expand-queries ought to (almost)
always fail for distributed. Also when rows/start is not set (default start=0
and rows=10). Can you explain why it does not fail in those cases
{code}
query(q, *:*, fq, {!collapse field=+group+}, defType,
edismax, bf, field(test_ti), expand, true, fl,*,score,
ShardParams.DISTRIB_SINGLE_PASS, true);
query(q, *:*, fq, {!collapse field=+group+}, defType,
edismax, bf, field(test_ti), expand, true, expand.sort, test_tl
desc, fl,*,score, ShardParams.DISTRIB_SINGLE_PASS, true);
query(q, *:*, fq, {!collapse field=+group+}, defType,
edismax, bf, field(test_ti), expand, true, expand.sort, test_tl
desc, expand.rows, 1, fl,*,score, ShardParams.DISTRIB_SINGLE_PASS,
true);
//Test no expand results
query(q, test_ti:5, fq, {!collapse field=+group+}, defType,
edismax, bf, field(test_ti), expand, true, expand.sort, test_tl
desc, expand.rows, 1, fl,*,score, ShardParams.DISTRIB_SINGLE_PASS,
true);
//Test zero results
query(q, test_ti:5434343, fq, {!collapse field=+group+},
defType, edismax, bf, field(test_ti), expand, true, expand.sort,
test_tl desc, expand.rows, 1, fl,*,score,
ShardParams.DISTRIB_SINGLE_PASS, true);
//Test page 2
query(q, *:*, start,1, rows, 1, fq, {!collapse
field=+group+}, defType, edismax, bf, field(test_ti), expand,
true, fl,*,score, ShardParams.DISTRIB_SINGLE_PASS, true);
{code}
Only the last one above fails?
distrib.singlePass does not work for expand-request - start/rows included
-
Key: SOLR-6813
URL: https://issues.apache.org/jira/browse/SOLR-6813
Project: Solr
Issue Type: Bug
Components: multicore, search
Reporter: Per Steffensen
Assignee: Joel Bernstein
Labels: distributed_search, search
Attachments: test_that_reveals_the_problem.patch
Using distrib.singlePass does not work for expand-requests. Even after the
fix provided to SOLR-6812, it does not work for requests where you add start
and/or rows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6813) distrib.singlePass does not work for expand-request - start/rows included
[ https://issues.apache.org/jira/browse/SOLR-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14248200#comment-14248200 ] Joel Bernstein commented on SOLR-6813: -- The test will fail when the number of collapsed groups returned by the query is larger then the page size. For example with start=1 and rows=1 the controlRsp will have one expanded group, while the rsp will have multiple expanded groups. With default page size of 10, both the controlRsp and rsp will have four expanded groups so the tests pass. We can solve the test failures easily with a change to handleResults to remove groups that are not in the final docList. The distributed deep paging problem will continue to exist. The only way I see to resolve that problem is with the two pass model. distrib.singlePass is not the most efficient approach for every situation, so we can pick and choose when to apply it. [~shalinmangar], how does distrib.singlePass work with standard grouping? I suspect grouping still does multi-pass because it has a different distributed flow. distrib.singlePass does not work for expand-request - start/rows included - Key: SOLR-6813 URL: https://issues.apache.org/jira/browse/SOLR-6813 Project: Solr Issue Type: Bug Components: multicore, search Reporter: Per Steffensen Assignee: Joel Bernstein Labels: distributed_search, search Attachments: test_that_reveals_the_problem.patch Using distrib.singlePass does not work for expand-requests. Even after the fix provided to SOLR-6812, it does not work for requests where you add start and/or rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6813) distrib.singlePass does not work for expand-request - start/rows included
[ https://issues.apache.org/jira/browse/SOLR-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14248410#comment-14248410 ] Per Steffensen commented on SOLR-6813: -- bq. The distributed deep paging problem will continue to exist But isnt that a performance problem only. Functionality-wise it will work doing deep paging, right? If it is just a performance problem, basically you should not use distrib.singlePass if you are deep paging, I guess we should not disallow using it together with deep paging. It is just a bad idea performance-wise to use distrib.singlePass together with deep paging. distrib.singlePass does not work for expand-request - start/rows included - Key: SOLR-6813 URL: https://issues.apache.org/jira/browse/SOLR-6813 Project: Solr Issue Type: Bug Components: multicore, search Reporter: Per Steffensen Assignee: Joel Bernstein Labels: distributed_search, search Attachments: test_that_reveals_the_problem.patch Using distrib.singlePass does not work for expand-requests. Even after the fix provided to SOLR-6812, it does not work for requests where you add start and/or rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6813) distrib.singlePass does not work for expand-request - start/rows included
[ https://issues.apache.org/jira/browse/SOLR-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14247399#comment-14247399 ] Joel Bernstein commented on SOLR-6813: -- My initial thoughts... In distrib.singlePass mode the ExpandComponent will be returning more documents then are needed to satisfy the query. Here is the basic logic: 1) In non-distributed mode: Return expanded groups for all documents in the docList. 2) In distributed mode: Return expanded groups for all documents referenced in the ID parameter. This ensured that only documents in the current page were expanded. With distrib.singlePass mode the ExpandComponent will behave like #1. So if the page size is 10 and there are ten shards, each shard will return 10 expanded groups. So there will be 100 expanded groups in the output. To resolve this issue the handleResponses method in the ExpandComponent is going to have to remove expanded groups that are not in the final merged docList. distrib.singlePass does not work for expand-request - start/rows included - Key: SOLR-6813 URL: https://issues.apache.org/jira/browse/SOLR-6813 Project: Solr Issue Type: Bug Components: multicore, search Reporter: Per Steffensen Assignee: Joel Bernstein Labels: distributed_search, search Attachments: test_that_reveals_the_problem.patch Using distrib.singlePass does not work for expand-requests. Even after the fix provided to SOLR-6812, it does not work for requests where you add start and/or rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6813) distrib.singlePass does not work for expand-request - start/rows included
[ https://issues.apache.org/jira/browse/SOLR-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14247423#comment-14247423 ] Joel Bernstein commented on SOLR-6813: -- Another thought... We also appear to have a deep paging issue to consider with distrib.singlePass on when the ExpandComponent is in play. The ExpandComponent will fetch groups for all documents in the docList when distrib.singlePass is on. With distributed deep paging the docList continues to grow as the user pages deeper into the result set. This means that more expanded groups will be fetched, making the deep paging problems much worse. In normal two pass distributed mode, the ExpandComponent can use the ID list to eliminate the deep paging issue. So, in a nutshell we may be slowing things down quite a bit when using distrib.singlePass with the ExpandComponent. We should consider turning off distrib.singlePass if the ExpandComponent is in use. distrib.singlePass does not work for expand-request - start/rows included - Key: SOLR-6813 URL: https://issues.apache.org/jira/browse/SOLR-6813 Project: Solr Issue Type: Bug Components: multicore, search Reporter: Per Steffensen Assignee: Joel Bernstein Labels: distributed_search, search Attachments: test_that_reveals_the_problem.patch Using distrib.singlePass does not work for expand-requests. Even after the fix provided to SOLR-6812, it does not work for requests where you add start and/or rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6813) distrib.singlePass does not work for expand-request - start/rows included
[ https://issues.apache.org/jira/browse/SOLR-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246182#comment-14246182 ] Joel Bernstein commented on SOLR-6813: -- Ok, I'll take a look. distrib.singlePass does not work for expand-request - start/rows included - Key: SOLR-6813 URL: https://issues.apache.org/jira/browse/SOLR-6813 Project: Solr Issue Type: Bug Components: multicore, search Reporter: Per Steffensen Labels: distributed_search, search Attachments: test_that_reveals_the_problem.patch Using distrib.singlePass does not work for expand-requests. Even after the fix provided to SOLR-6812, it does not work for requests where you add start and/or rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6813) distrib.singlePass does not work for expand-request - start/rows included
[ https://issues.apache.org/jira/browse/SOLR-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245034#comment-14245034 ] Shalin Shekhar Mangar commented on SOLR-6813: - I think I'm gonna have to defer this to [~joel.bernstein]. I am not very familiar with the workings of expand component either. distrib.singlePass does not work for expand-request - start/rows included - Key: SOLR-6813 URL: https://issues.apache.org/jira/browse/SOLR-6813 Project: Solr Issue Type: Bug Components: multicore, search Reporter: Per Steffensen Labels: distributed_search, search Attachments: test_that_reveals_the_problem.patch Using distrib.singlePass does not work for expand-requests. Even after the fix provided to SOLR-6812, it does not work for requests where you add start and/or rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6813) distrib.singlePass does not work for expand-request - start/rows included
[
https://issues.apache.org/jira/browse/SOLR-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231428#comment-14231428
]
Per Steffensen commented on SOLR-6813:
--
Unlike for SOLR-6812 (and SOLR-6795 and SOLR-6796) I am not going to provide a
fix here, because I really do not know enough about expand-requests to be able
to say which response is actually the correct one - {{controlRsp}} or {{rsp}}
in {{BaseDistributedSearchTestCase.query}} when doing the modified request from
{{test_that_reveals_the_problem.patch}}
distrib.singlePass does not work for expand-request - start/rows included
-
Key: SOLR-6813
URL: https://issues.apache.org/jira/browse/SOLR-6813
Project: Solr
Issue Type: Bug
Components: multicore, search
Reporter: Per Steffensen
Labels: distributed_search, search
Attachments: test_that_reveals_the_problem.patch
Using distrib.singlePass does not work for expand-requests. Even after the
fix provided to SOLR-6812, it does not work for requests where you add start
and/or rows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
