[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[ https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378485#comment-16378485 ] Ere Maijala commented on SOLR-11982: One more version to make sure a rule is only split into two parts so that e.g. `replicaLocation:http://server1` works. > Add support for preferReplicaTypes parameter > > > Key: SOLR-11982 > URL: https://issues.apache.org/jira/browse/SOLR-11982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (8.0), 7.3 >Reporter: Ere Maijala >Priority: Minor > Labels: patch-available, patch-with-test > Attachments: SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch > > > It would be nice to have the possibility to easily prefer certain replica > types in a similar fashion to preferLocalShards. I'll be coming up with a > patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which > would mean that NRT replicas wouldn't be hit with queries unless they're the > only ones available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[ https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378477#comment-16378477 ] Ere Maijala commented on SOLR-11982: I liked the idea of "replicaLocation" enough that I implemented support for it in the latest patch. > Add support for preferReplicaTypes parameter > > > Key: SOLR-11982 > URL: https://issues.apache.org/jira/browse/SOLR-11982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (8.0), 7.3 >Reporter: Ere Maijala >Priority: Minor > Labels: patch-available, patch-with-test > Attachments: SOLR-11982.patch, SOLR-11982.patch > > > It would be nice to have the possibility to easily prefer certain replica > types in a similar fashion to preferLocalShards. I'll be coming up with a > patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which > would mean that NRT replicas wouldn't be hit with queries unless they're the > only ones available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[ https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377049#comment-16377049 ] Ere Maijala commented on SOLR-11982: The latest patch switches to the syntax [~tomasflobbe] suggested apart from the "local" option being "replicaLocal". This is actually quite nice in that the order of rules can be used to define which ones are the most important. One option to go a bit further would be to define "replicaLocation" instead of "replicaLocal" which, for now, would only support "local" as the value but could be easily extended to support e.g "replicaLocation:http://server2.something;. What do you think? > Add support for preferReplicaTypes parameter > > > Key: SOLR-11982 > URL: https://issues.apache.org/jira/browse/SOLR-11982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (8.0), 7.3 >Reporter: Ere Maijala >Priority: Minor > Labels: patch-available, patch-with-test > Attachments: SOLR-11982.patch > > > It would be nice to have the possibility to easily prefer certain replica > types in a similar fashion to preferLocalShards. I'll be coming up with a > patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which > would mean that NRT replicas wouldn't be hit with queries unless they're the > only ones available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[ https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376892#comment-16376892 ] Ere Maijala commented on SOLR-11982: [~tomasflobbe] At a further thought, maybe including preferLocalShards will work. I'll put up a new patch shortly. > Add support for preferReplicaTypes parameter > > > Key: SOLR-11982 > URL: https://issues.apache.org/jira/browse/SOLR-11982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (8.0), 7.3 >Reporter: Ere Maijala >Priority: Minor > Labels: patch-available, patch-with-test > Attachments: SOLR-11982-v2.patch, SOLR-11982-v3.patch, > SOLR-11982.patch > > > It would be nice to have the possibility to easily prefer certain replica > types in a similar fashion to preferLocalShards. I'll be coming up with a > patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which > would mean that NRT replicas wouldn't be hit with queries unless they're the > only ones available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[ https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374052#comment-16374052 ] Ere Maijala commented on SOLR-11982: [~tomasflobbe] I agree that it makes sense to unify the syntax. I'm not sure, though, that this should include preferLocalShards. I want to be able to say that I prefer local PULL replicas over remote PULL replicas, and with your proposed API that's not possible, right? > Add support for preferReplicaTypes parameter > > > Key: SOLR-11982 > URL: https://issues.apache.org/jira/browse/SOLR-11982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (8.0), 7.3 >Reporter: Ere Maijala >Priority: Minor > Labels: patch-available, patch-with-test > Attachments: SOLR-11982-v2.patch, SOLR-11982-v3.patch, > SOLR-11982.patch > > > It would be nice to have the possibility to easily prefer certain replica > types in a similar fashion to preferLocalShards. I'll be coming up with a > patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which > would mean that NRT replicas wouldn't be hit with queries unless they're the > only ones available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373310#comment-16373310
]
Tomás Fernández Löbbe commented on SOLR-11982:
--
Thanks [~emaijala], this looks really good. Again, I'm trying to unify the API
with the work done in SOLR-10880. SOLR-10880 attempts to filter the replicas
that can be used for resolving a query using {{shards.filter}}. It feels like
what you are doing here is sorting the replicas, to prefer some over others.
How do you feel about using the a parameter like {{shards.sort}}, and making
the value something like {{replicaType:PULL,replicaType:TLOG}}? {{local}} could
actually be just a keyword of this new parameter, and someone could do:
{{...=replicaType:PULL,local}} meaning "use replicas of type PULL
if available, otherwise prefer local cores, otherwise any replica}}. In the
future, the replica tags, as defined in SOLR-10880 could also be used for
sorting and not only for filtering (i.e.
{{...=local,replicaProp:rack1,replicaProp:dc1}}).
If so, we can deprecate {{preferLocalShards}}
> Add support for preferReplicaTypes parameter
>
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
> Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-v2.patch, SOLR-11982-v3.patch,
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica
> types in a similar fashion to preferLocalShards. I'll be coming up with a
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which
> would mean that NRT replicas wouldn't be hit with queries unless they're the
> only ones available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[ https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371431#comment-16371431 ] Ere Maijala commented on SOLR-11982: Patch v3 attached. This makes sure to use a new collection for the replica type test and that there are always replicas for all types. > Add support for preferReplicaTypes parameter > > > Key: SOLR-11982 > URL: https://issues.apache.org/jira/browse/SOLR-11982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (8.0), 7.3 >Reporter: Ere Maijala >Priority: Minor > Labels: patch-available, patch-with-test > Attachments: SOLR-11982-v2.patch, SOLR-11982-v3.patch, > SOLR-11982.patch > > > It would be nice to have the possibility to easily prefer certain replica > types in a similar fashion to preferLocalShards. I'll be coming up with a > patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which > would mean that NRT replicas wouldn't be hit with queries unless they're the > only ones available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[ https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371415#comment-16371415 ] Ere Maijala commented on SOLR-11982: I'm getting intermittent test failures with the v2 patch, investigating... > Add support for preferReplicaTypes parameter > > > Key: SOLR-11982 > URL: https://issues.apache.org/jira/browse/SOLR-11982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (8.0), 7.3 >Reporter: Ere Maijala >Priority: Minor > Labels: patch-available, patch-with-test > Attachments: SOLR-11982-v2.patch, SOLR-11982.patch > > > It would be nice to have the possibility to easily prefer certain replica > types in a similar fashion to preferLocalShards. I'll be coming up with a > patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which > would mean that NRT replicas wouldn't be hit with queries unless they're the > only ones available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[ https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371370#comment-16371370 ] Ere Maijala commented on SOLR-11982: I've attached a new patch that should address all the comments. Sorry for the whitespace changes, they should be reverted now. [~cpoerschke] Thanks for the feedback! The original patch didn't take the order of the preferred replica types to account, but the new patch does, since it can be useful too. I also tried to clarify the documentation so that the use case for both parameters is more clear. Does this help? [~erickerickson] Thanks, that was indeed a typo, now fixed. > Add support for preferReplicaTypes parameter > > > Key: SOLR-11982 > URL: https://issues.apache.org/jira/browse/SOLR-11982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (8.0), 7.3 >Reporter: Ere Maijala >Priority: Minor > Labels: patch-available, patch-with-test > Attachments: SOLR-11982-v2.patch, SOLR-11982.patch > > > It would be nice to have the possibility to easily prefer certain replica > types in a similar fashion to preferLocalShards. I'll be coming up with a > patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which > would mean that NRT replicas wouldn't be hit with queries unless they're the > only ones available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[ https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370495#comment-16370495 ] Erick Erickson commented on SOLR-11982: --- [~emaijala] Is this bit a typo in the docs? Shouldn't it be "preferReplicaTypes=PULL,TLOG" rather than "preferLocalShards"??? Solr allows you to pass an optional string parameter named `preferReplicaTypes` to indicate that a distributed query should prefer replicas of given types when available. In other words, if a query includes e.g. `*preferLocalShards*=PULL,TLOG`,... [~cpoerschke] One trick for whitespace bits if you use IntelliJ is to > apply the patch > command-9 to show local changes > bring the context menu up on the first one and select diff... > up in the upper left there's a drop-down and one of the choices is "ignore > whitespace/newlines" or something like that. I agree though it's better to not reformat lots of whitespace Also, one option in IntelliJ that should be clicked is to auto-format only lines changed. There's an option somewhere in the IntelliJ preferences to insure this > Add support for preferReplicaTypes parameter > > > Key: SOLR-11982 > URL: https://issues.apache.org/jira/browse/SOLR-11982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (8.0), 7.3 >Reporter: Ere Maijala >Priority: Minor > Labels: patch-available, patch-with-test > Attachments: SOLR-11982.patch > > > It would be nice to have the possibility to easily prefer certain replica > types in a similar fashion to preferLocalShards. I'll be coming up with a > patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which > would mean that NRT replicas wouldn't be hit with queries unless they're the > only ones available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter
[
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370480#comment-16370480
]
Christine Poerschke commented on SOLR-11982:
Hello [~emaijala],
thanks for opening this ticket and attaching a patch with tests and solr ref
guide documentation update.
I agree the ability to prefer certain replica types would be a nice new feature.
Question: would {{preferReplicaTypes=PULL,TLOG}} and
{{preferReplicaTypes=TLOG,PULL}} be equivalent or would there be a difference
i.e. the first type most preferred, the second type next preferred, and any
other unmentioned types all equivally unpreferred?
Thanks for clarifying (in the documentation update part of the patch) the
intended behaviour when both the existing {{preferLocalShards}} and the new
{{preferReplicaTypes}} are specified.
{code}
+This parameter overrides `preferLocalShards=true`. Both can be defined, but
replicas of preferred types are always selected over local shards.
{code}
{{preferReplicaTypes overrides preferLocalShards}} vs. {{preferLocalShards vs.
preferReplicaTypes}} - what might the use cases in either scenario be? Or
perhaps the two parameters could be mutually exclusive for less potential user
confusion and a simpler implementation. What do you think?
Specific patch feedback from taking only a quick look: there seem to be quite a
few whitespace change formatting changes which makes it tricky to 'see' the
actual changes. Steps along the following lines could be one way to try and
undo those whitespace reformats:
{code}
git checkout -b master-solr-11982 -t origin/master
git apply SOLR-11982.patch
git diff -w > temp.patch
git checkout HEAD --
git apply --ignore-whitespace temp.patch
{code}
> Add support for preferReplicaTypes parameter
>
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
> Labels: patch-available, patch-with-test
> Attachments: SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica
> types in a similar fashion to preferLocalShards. I'll be coming up with a
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which
> would mean that NRT replicas wouldn't be hit with queries unless they're the
> only ones available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
