saurabhd336 commented on code in PR #12276:
URL: https://github.com/apache/pinot/pull/12276#discussion_r1458821960
##########
pinot-broker/src/main/java/org/apache/pinot/broker/routing/instanceselector/MultiStageReplicaGroupSelector.java:
##########
@@ -86,7 +87,16 @@ Pair<Map<String, String>, Map<String, String>>
select(List<String> segments, int
SegmentStates segmentStates, Map<String, String> queryOptions) {
// Create a copy of InstancePartitions to avoid race-condition with
event-listeners above.
InstancePartitions instancePartitions = _instancePartitions;
- int replicaGroupSelected = requestId %
instancePartitions.getNumReplicaGroups();
+ int replicaGroupSelected;
+ if (_useStickyRouting) {
+ // When using sticky routing, we want to iterate over the
instancePartitions in order to ensure deterministic
+ // selection of replica group across queries i.e. same instance replica
group id is picked each time.
+ // Since the instances within a selected replica group are iterated in
order, the assignment within a selected
+ // replica group is guaranteed to be deterministic.
+ replicaGroupSelected = 0;
Review Comment:
So the idea with the `MultiStageReplicaGroupSelector` is to ensure colocated
tables get routed to same set of servers IIUC. If we use table hash here, that
goal will not be achieved (the non sticky routing path uses requestId % and not
tableName hash).
It's possible to
1) Use table hash here -> preferring even distribution across replica group
for different tables over colocation.
2) Start from 0 -> preferring colocation at the cost of potential hotspots.
3) Not support stick routing when using the `MultiStageReplicaGroupSelector`
I went with 2, but open to suggestions here. cc: @walterddr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
