[jira] [Commented] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261464#comment-17261464
]
John Gallagher commented on SOLR-14413:
---
Thanks for reporting [~hossman]. I didn't know about the "last definition
wins" semantics.
I removed the last line of that configuration: [^SOLR-14413.testfix.patch]
I don't believe we need to switch the handler name, because it should not
affect other tests unless they pass the sleep parameter explicitly.
I reran the test using the parameters in your log, and it passed.
{code:java}
./gradlew test --tests CursorPagingTest.testTimeAllowed
-Dtests.seed=4D63CDC87865C2E0 -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=en-MO -Dtests.timezone=Africa/Asmara -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
> Task :solr:core:test
:solr:core:test (SUCCESS): 1 test(s)
The slowest tests (exceeding 500 ms) during this run:
2.12s CursorPagingTest.testTimeAllowed (:solr:core)
BUILD SUCCESSFUL in 4m 13s
263 actionable tasks: 263 executed
{code}
The test failure was this line:
[https://github.com/apache/lucene-solr/blob/a7391fb73ef9169c58bc20291fdcefcbd47fa0a8/solr/core/src/test/org/apache/solr/CursorPagingTest.java#L571]
{{assertTrue("Should have experienced at least one partialResult", partialCount
> 0);}}
I had added DelayingSearchComponent to make partial results _more_ likely
(well, 100% likely, I was hoping), not less.
I guess my own machine was too slow to complete an initial query without
partial results. It didn't need the aid of the artificially injected delay.
Apologies for the flakiness you experienced
John
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Assignee: Mike Drob
>Priority: Minor
> Fix For: 8.8, master (9.0)
>
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413-jg-update2.patch, SOLR-14413-jg-update3.patch, SOLR-14413.patch,
> SOLR-14413.testfix.patch, Screen Shot 2020-10-23 at 10.08.26 PM.png, Screen
> Shot 2020-10-23 at 10.09.11 PM.png, image-2020-08-18-16-56-41-736.png,
> image-2020-08-18-16-56-59-178.png, image-2020-08-21-14-18-36-229.png,
> timeallowed_cursormarks_results.txt
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: SOLR-14413.testfix.patch
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Assignee: Mike Drob
>Priority: Minor
> Fix For: 8.8, master (9.0)
>
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413-jg-update2.patch, SOLR-14413-jg-update3.patch, SOLR-14413.patch,
> SOLR-14413.testfix.patch, Screen Shot 2020-10-23 at 10.08.26 PM.png, Screen
> Shot 2020-10-23 at 10.09.11 PM.png, image-2020-08-18-16-56-41-736.png,
> image-2020-08-18-16-56-59-178.png, image-2020-08-21-14-18-36-229.png,
> timeallowed_cursormarks_results.txt
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224785#comment-17224785
]
John Gallagher commented on SOLR-14413:
---
[~mdrob], [~bvd], any thoughts on the updated test and documentation?
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413-jg-update2.patch, SOLR-14413-jg-update3.patch, SOLR-14413.patch,
> Screen Shot 2020-10-23 at 10.08.26 PM.png, Screen Shot 2020-10-23 at 10.09.11
> PM.png, image-2020-08-18-16-56-41-736.png, image-2020-08-18-16-56-59-178.png,
> image-2020-08-21-14-18-36-229.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220001#comment-17220001
]
John Gallagher commented on SOLR-14413:
---
[~bvd] the issue was with an assumption that the test was making.
It is not always true that every document will be found when using timeAllowed
and cursorMark in combination. There may be holes in the result sets, but at
least the ordering between and within result sets will be correct with respect
to the sort.
This is what I had suspected was the case when I proposed allowing the
combination, but I didn't have an example at the time. I think it is still a
good idea to allow these parameters in combination (its something that you
could encounter when using shards.tolerant and cursorMark in combination, and
that combination is allowed).
When using timeAllowed and cursorMark in combination, and there are multiple
segments in the index, it is possible that a query may terminate before
visiting the matching documents in every segment. The hint for this is in the
warning message's stack trace associated with the failing seed you found in the
previous revision:
[https://gist.github.com/slackhappy/1a48d56e10679404cea3441f87a0fecc#file-gistfile1-txt-L6]
. "The request took too long to iterate over terms." occurs while in a
specific segment, which prevents iterating on to the next segment.
I have updated my pull request:
[https://github.com/apache/lucene-solr/pull/1436]
And I have updated my proposed documentation changes to include a mention that
results may be missing if partialResults is true:
!Screen Shot 2020-10-23 at 10.08.26 PM.png|width=545,height=114!
!Screen Shot 2020-10-23 at 10.09.11 PM.png|width=577,height=161!
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413-jg-update2.patch, SOLR-14413.patch, Screen Shot 2020-10-23 at
> 10.08.26 PM.png, Screen Shot 2020-10-23 at 10.09.11 PM.png,
> image-2020-08-18-16-56-41-736.png, image-2020-08-18-16-56-59-178.png,
> image-2020-08-21-14-18-36-229.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: SOLR-14413-jg-update3.patch
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413-jg-update2.patch, SOLR-14413-jg-update3.patch, SOLR-14413.patch,
> Screen Shot 2020-10-23 at 10.08.26 PM.png, Screen Shot 2020-10-23 at 10.09.11
> PM.png, image-2020-08-18-16-56-41-736.png, image-2020-08-18-16-56-59-178.png,
> image-2020-08-21-14-18-36-229.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: Screen Shot 2020-10-23 at 10.09.11 PM.png
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413-jg-update2.patch, SOLR-14413.patch, Screen Shot 2020-10-23 at
> 10.08.26 PM.png, Screen Shot 2020-10-23 at 10.09.11 PM.png,
> image-2020-08-18-16-56-41-736.png, image-2020-08-18-16-56-59-178.png,
> image-2020-08-21-14-18-36-229.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: Screen Shot 2020-10-23 at 10.08.26 PM.png
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413-jg-update2.patch, SOLR-14413.patch, Screen Shot 2020-10-23 at
> 10.08.26 PM.png, image-2020-08-18-16-56-41-736.png,
> image-2020-08-18-16-56-59-178.png, image-2020-08-21-14-18-36-229.png,
> timeallowed_cursormarks_results.txt
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182060#comment-17182060
]
John Gallagher commented on SOLR-14413:
---
Thanks [~mdrob] for reviewing!
I added new documentation for omitHeader, advising against omitting when using
parameters that can lead to partial results, because the header flag informs
the interpretation of the result values.
!image-2020-08-21-14-18-36-229.png|width=558,height=180!
I updated timeAllowed documentation as well.
I incorporated [~bvd]'s test idea of generating a result set, running it to
completion, and confirming all documents were found in the correct order, while
also asserting that at least one partialResults event occurred.
I have generated a new patch, SOLR-14413-jg-update2.patch, and updated my PR:
[https://github.com/apache/lucene-solr/pull/1436]
John
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413.patch, image-2020-08-18-16-56-41-736.png,
> image-2020-08-18-16-56-59-178.png, image-2020-08-21-14-18-36-229.png,
> timeallowed_cursormarks_results.txt
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: SOLR-14413-jg-update2.patch
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413-jg-update2.patch, SOLR-14413.patch,
> image-2020-08-18-16-56-41-736.png, image-2020-08-18-16-56-59-178.png,
> image-2020-08-21-14-18-36-229.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: image-2020-08-21-14-18-36-229.png
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413.patch, image-2020-08-18-16-56-41-736.png,
> image-2020-08-18-16-56-59-178.png, image-2020-08-21-14-18-36-229.png,
> timeallowed_cursormarks_results.txt
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180102#comment-17180102
]
John Gallagher commented on SOLR-14413:
---
Hi folks, sorry I stepped away from this for a bit, and thanks [~mdrob] , and
[~epugh] for the comments, and [~bvd] for contributing!
To Mike's earlier point: when partialResults is true, you cannot be sure
whether there are more results or not. You could summarize it this way:
|| ||partialResults not present|| partialResults: true||
|more results?|nextCursorMark != cursorMark|unknown|
After figuring out how to properly generate documentation, I took a stab at
adding two notes to the documentation:
1. In timeAllowed documentation,
common-query-parameters.html#timeallowed-parameter
!image-2020-08-18-16-56-59-178.png|width=508,height=116!
2. In the Constraints when using Cursors documentation,
pagination-of-results.html#constraints-when-using-cursors
!image-2020-08-18-16-56-41-736.png|width=410,height=250!
Please let me know what you think of these additions.
I was also able to reproduce [~bvd]'s case where nextCursorMark was null (the
field actually wasn't present in the response at all). I tracked it down to
[this code in
SearchHandler.java|[https://github.com/slackhappy/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L349]].
When the Search times out very early, an exception is thrown before responses
can be handled, which skips the calculation of the next cursor mark. In that
case, the response is null, and the SearchHandler creates an empty response to
return (numFound 0, docs:[]).
I added code to also return the original cursorMark in that case, since we
haven't progressed (it is the value you would want to pass to a subsequent
search).
I added a basic test to CursorPagingTest that checks that the parameters can be
used in conjunction, and that the nextCursorMark returned is a valid one that
can be used in subsequent requests. Perhaps that test can be combined with
[~bvd]'s. I struggled a bit to come up with a test that would produce reliable
results when using timeAllowed. I borrowed a bit from
ExitableDirectoryReaderTest's use of a DelayingSearchComponent, which helped
somewhat.
The updated patch file is SOLR-14413-jg-update1.patch, and I have updated the
[corresponding PR|[https://github.com/apache/lucene-solr/pull/1436]].
John
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413.patch, image-2020-08-18-16-56-41-736.png,
> image-2020-08-18-16-56-59-178.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: SOLR-14413-jg-update1.patch
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413.patch, image-2020-08-18-16-56-41-736.png,
> image-2020-08-18-16-56-59-178.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: image-2020-08-18-16-56-59-178.png
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413.patch,
> image-2020-08-18-16-56-41-736.png, image-2020-08-18-16-56-59-178.png,
> timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: (was: image-2020-08-18-16-56-03-425.png)
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413.patch,
> image-2020-08-18-16-56-41-736.png, image-2020-08-18-16-56-59-178.png,
> timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: (was: image-2020-08-18-16-56-03-355.png)
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413.patch,
> image-2020-08-18-16-56-41-736.png, image-2020-08-18-16-56-59-178.png,
> timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: image-2020-08-18-16-56-03-425.png
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413.patch,
> image-2020-08-18-16-56-03-355.png, image-2020-08-18-16-56-03-425.png,
> image-2020-08-18-16-56-41-736.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: image-2020-08-18-16-56-03-355.png
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413.patch,
> image-2020-08-18-16-56-03-355.png, image-2020-08-18-16-56-03-425.png,
> image-2020-08-18-16-56-41-736.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: image-2020-08-18-16-56-41-736.png
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413.patch,
> image-2020-08-18-16-56-03-355.png, image-2020-08-18-16-56-03-425.png,
> image-2020-08-18-16-56-41-736.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: timeallowed_cursormarks_results.txt
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413.patch, timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085971#comment-17085971
]
John Gallagher commented on SOLR-14413:
---
Hi [~ichattopadhyaya], thanks for having a look! I've attached a patch, and
created a PR here: [https://github.com/apache/lucene-solr/pull/1436]
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Status: Patch Available (was: Open)
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413.patch
>
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Attachment: SOLR-14413.patch
Status: Open (was: Open)
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Reporter: John Gallagher
>Priority: Minor
> Attachments: SOLR-14413.patch
>
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Status: Open (was: Patch Available)
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Reporter: John Gallagher
>Priority: Minor
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Status: Patch Available (was: Open)
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Reporter: John Gallagher
>Priority: Minor
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14413) allow timeAllowed and cursorMark parameters
[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Gallagher updated SOLR-14413:
--
Description:
Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
timeAllowed parameters were not allowed in combination ("Can not search using
both cursorMark and timeAllowed")
, from [QueryComponent.java|#L359]]:
{code:java}
if (null != rb.getCursorMark() && 0 < timeAllowed) {
// fundamentally incompatible
throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not search
using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
CommonParams.TIME_ALLOWED);
} {code}
While theoretically impure to use them in combination, it is often desirable to
support cursormarks-style deep paging and attempt to protect Solr nodes from
runaway queries using timeAllowed, in the hopes that most of the time, the
query completes in the allotted time, and there is no conflict.
However if the query takes too long, it may be preferable to end the query and
protect the Solr node and provide the user with a somewhat inaccurate sorted
list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is frequently
used to prevent runaway load. In fact, cursorMark and shards.tolerant are
allowed in combination, so any argument in favor of purity would be a bit
muddied in my opinion.
This was discussed once in the mailing list that I can find:
[https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
It did not look like there was strong support for preventing the combination.
I have tested cursorMark and timeAllowed combination together, and even when
partial results are returned because the timeAllowed is exceeded, the
cursorMark response value is still valid and reasonable.
was:
Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
timeAllowed parameters were not allowed in combination ("Can not search using
both cursorMark and timeAllowed")
, from
[QueryComponent.java|[https://github.com/apache/lucene-solr/blob/03363f413f2134594b012175deb3f10ec9384400/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L359]]:
{code:java}
if (null != rb.getCursorMark() && 0 < timeAllowed) {
// fundamentally incompatible
throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not search
using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
CommonParams.TIME_ALLOWED);
} {code}
While theoretically impure to use them in combination, it is often desirable to
support cursormarks-style deep paging and attempt to protect Solr nodes from
runaway queries using timeAllowed, in the hopes that most of the time, the
query completes in the allotted time, and there is no conflict.
However if the query takes too long, it may be preferable to end the query and
protect the Solr node and provide the user with a somewhat inaccurate sorted
list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is frequently
used to prevent runaway load. In fact, cursorMark and shards.tolerant are
allowed in combination, so any argument in favor of purity would be a bit
muddied in my opinion.
This was discussed once in the mailing list that I can find:
[https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
It did not look like there was strong support for preventing the combination.
> allow timeAllowed and cursorMark parameters
> ---
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Reporter: John Gallagher
>Priority: Minor
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
>
[jira] [Created] (SOLR-14413) allow timeAllowed and cursorMark parameters
John Gallagher created SOLR-14413:
-
Summary: allow timeAllowed and cursorMark parameters
Key: SOLR-14413
URL: https://issues.apache.org/jira/browse/SOLR-14413
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: search
Reporter: John Gallagher
Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
timeAllowed parameters were not allowed in combination ("Can not search using
both cursorMark and timeAllowed")
, from
[QueryComponent.java|[https://github.com/apache/lucene-solr/blob/03363f413f2134594b012175deb3f10ec9384400/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L359]]:
{code:java}
if (null != rb.getCursorMark() && 0 < timeAllowed) {
// fundamentally incompatible
throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not search
using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
CommonParams.TIME_ALLOWED);
} {code}
While theoretically impure to use them in combination, it is often desirable to
support cursormarks-style deep paging and attempt to protect Solr nodes from
runaway queries using timeAllowed, in the hopes that most of the time, the
query completes in the allotted time, and there is no conflict.
However if the query takes too long, it may be preferable to end the query and
protect the Solr node and provide the user with a somewhat inaccurate sorted
list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is frequently
used to prevent runaway load. In fact, cursorMark and shards.tolerant are
allowed in combination, so any argument in favor of purity would be a bit
muddied in my opinion.
This was discussed once in the mailing list that I can find:
[https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
It did not look like there was strong support for preventing the combination.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
