[
https://issues.apache.org/jira/browse/SOLR-13257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16882405#comment-16882405
]
Christine Poerschke commented on SOLR-13257:
--------------------------------------------
Thanks [~mgibney] for creating and working on this ticket!
I have some "requirements" or "intended behaviour" type questions or thoughts,
perhaps best posed via examples.
1. The out-of-the-box behaviour (in the absence of {{shards.preference}} or
similar parameters) is, as you say, a random choice.
*
** Example: A collection with three replicas (replicaA, replicaB, replicaC),
all of which are available at the time of searching.
** Expected behaviour: The list of available replicas is randomly shuffled
i.e. there are {{3! = 6}} possibilities overall: {{[ [A, B, C], [A, C, B], [B,
A, C], [B, C, A], [C, A, B], [C, B, A] ]}}
2. The objective of this ticket i.e. the proposed behaviour (in the absence of
{{shards.preference}} or similar parameters) is reduce or remove the
random-ness of the out-of-the-box behaviour.
*
** Example: A collection with three replicas (replicaA, replicaB, replicaC),
all of which are available at the time of searching. The request indicates
(directly via a numeric value parameter or indirectly via a non-numeric value
parameter that can be hashed) the affinity to an element in or a portion of the
list of replicas. Let's assume the affinity expressed translates into replica
{{A}}.
** Expected behaviour:
*** Possibility 1: The preferred replica is {{A}} but beyond the first choice
the random-ness remains for load balancing purposes i.e. there are {{2! = 2}}
possibilities overall: {{[ [A, B, C], [A, C, B] ]}}
*** Possibility 2: The preferred replica is {{A}} and also beyond that first
choice there is no random-ness i.e. the replica list is sorted
deterministically to give just one possibility: {{[A, B, C]}}
3. The existing {{shards.preference}} parameter influences the replica choices
(in the absence of new affinity logic).
(caveat: I presume the below is the existing behaviour of the existing code but
have not recently read the code in detail to fully convince myself that this is
so.)
*
** Example 1: A collection with six replicas spread equally across three
locations. The request indicates a preference for "Europe" location.
** Expected behaviour: The preferred replicas (based on location) are at the
start of the list and the not-preferred replicas are at the end of the list.
The ordering within the start and end portions of the list is random giving
{{2! * 4! = 2*1 * 4*3*2*1 = 2 * 24 = 48}} possibilities overall.
*
** Example 2: A collection with six replicas spread equally across three
locations and the two replicas at each location are of different types e.g.
{{PULL}} and {{TLOG}}. The request indicates a preference primarily for
"Europe" location and secondarily for "PULL" type.
** Expected behaviour: The preferred replicas (based on location) are at the
start of the list and the not-preferred replicas are at the end of the list.
The ordering within the start portion of the list becomes deterministic via the
secondary preference (e.g. {{[ replicaB1(europe,pull), replicaB2(europe,tlog)
]}}) and the ordering within the end portion of the list is narrowed down via
the secondary preference but some random-ness remains giving 4 possibilities
overall:
{code:java}
[ replicaB1(europe,pull), replicaB2(europe,tlog),
replicaA1(america,pull), replicaC1(asia,pull),
replicaA2(america,tlog), replicaC2(asia,tlog) ]
[ replicaB1(europe,pull), replicaB2(europe,tlog),
replicaA1(america,pull), replicaC1(asia,pull),
replicaC2(asia,tlog), replicaA2(america,tlog) ]
[ replicaB1(europe,pull), replicaB2(europe,tlog),
replicaC1(asia,pull), replicaA1(america,pull),
replicaC2(asia,tlog), replicaA2(america,tlog) ]
[ replicaB1(europe,pull), replicaB2(europe,tlog),
replicaC1(asia,pull), replicaA1(america,pull),
replicaA2(america,tlog), replicaC2(asia,tlog) ]
{code}
4. The new shard affinity logic combined with a single {{shards.preference}}
parameter.
*
** Example: A collection with six replicas spread equally across three
locations. The request indicates a preference for "Europe" location.
** Expected behaviour: The preferred replicas (based on location) are at the
start of the list and the not-preferred replicas are at the end of the list.
*** The ordering within the "Europe" start portion of the list becomes
deterministic via the new shard affinity logic.
*** Does the new shard affinity logic influence the ordering within the end
portion of the list too?
5. The new shard affinity logic combined with multiple {{shards.preference}}
parameters.
*
** At its simplest each {{shards.preference}} parameter will have two choices
e.g. "PULL" vs. "TLOG" replica type or "Europe" vs. unknown replica location.
** Assuming two {{shards.preference}} parameters with two choices each the
resulting list will have four portions (each potentially containing multiple
replicas):
{code:java}
primaryPreference=preferredValue, secondaryPreference=preferredValue
primaryPreference=preferredValue, secondaryPreference=notPreferredValue
primaryPreference=notPreferredValue, secondaryPreference=preferredValue
primaryPreference=notPreferredValue, secondaryPreference=notPreferredValue
{code}
*
** With more than two {{shards.preference}} parameters the number of portions
in the list will increase.
** Does the new shard affinity logic influence the ordering within first
(start) portion of the list only or does it influence the ordering within the
other portions too?
----
End of brain dump, slightly longer than expected, oops.
----
Next re-reading the existing ticket comments and skimming over the {{.adoc}}
changes in the latest patch to try and answer the questions above:
* Any {{shards.preference=replica.base:something}} parameter (other than
{{shards.preference=replica.base:random}}) will remove all random-ness from the
choices.
* With respect to the new shard affinity logic's interaction with the existing
{{shards.preference}} parameters, in principle things could get quite
complicated there but you very elegantly solve that by designating that
{{replica.base}} may be used only once and only as a tie-breaker i.e. as the
last of potentially multiple preferences.
* Taking the {{shards.preference=replica.location:Europe,replica.type:PULL}}
example from the above, the addition of
{{shards.preference=replica.base:stable:2:fooBar&fooBar=0}} would result in the
ordering ordering below because
** {{replicaA1(america,pull) and {{replicaC1(asia,pull)}} are equivalent (both
don't match the primary replica.location preference and both do match the
secondary replica.type preference) and {{replica.base:stable}} now determines
their relative ordering
** {{replicaA2(america,tlog)}} and {{replicaC2(asia,tlog)}} are equivalent
(both match neither the primary replica.location preference nor the secondary
replica.type preference) and {{replica.base:stable}} now determines their
relative ordering.
{code:java}
[ replicaB1(europe,pull), replicaB2(europe,tlog),
replicaA1(america,pull), replicaC1(asia,pull),
replicaA2(america,tlog), replicaC2(asia,tlog) ]
{code}
Does my thought process above kind of make sense and are the resulting answers
accurate/fair?
> Enable replica routing affinity for better cache usage
> ------------------------------------------------------
>
> Key: SOLR-13257
> URL: https://issues.apache.org/jira/browse/SOLR-13257
> Project: Solr
> Issue Type: New Feature
> Components: SolrCloud
> Reporter: Michael Gibney
> Priority: Minor
> Attachments: AffinityShardHandlerFactory.java, SOLR-13257.patch,
> SOLR-13257.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For each shard in a distributed request, Solr currently routes each request
> randomly via
> [ShufflingReplicaListTransformer|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/component/ShufflingReplicaListTransformer.java]
> to a particular replica. In setups with replication factor >1, this normally
> results in a situation where subsequent requests (which one would hope/expect
> to leverage cached results from previous related requests) end up getting
> routed to a replica that hasn't seen any related requests.
> The problem can be replicated by issuing a relatively expensive query (maybe
> containing common terms?). The first request initializes the
> {{queryResultCache}} on the consulted replicas. If replication factor >1 and
> there are a sufficient number of shards, subsequent requests will likely be
> routed to at least one replica that _hasn't_ seen the query before. The
> replicas with uninitialized caches become a bottleneck, and from the client's
> perspective, many subsequent requests appear not to benefit from caching at
> all.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
