xinghuayu007 commented on a change in pull request #5104:
URL: https://github.com/apache/incubator-doris/pull/5104#discussion_r549651544
##########
File path: fe/fe-core/src/main/java/org/apache/doris/qe/SimpleScheduler.java
##########
@@ -92,6 +92,38 @@ public static TNetworkAddress getHost(long backendId,
backends, locations.size()));
}
+ public static TScanRangeLocation getLocation(TScanRangeLocation
minLocation,
Review comment:
Because function `computeScanRangeAssignmentByScheduler` need
`minLocation` to set volume_id. Therefore it must return param `minLocation`
// Volume is optional, so we need to set the value and the is-set bit
scanRangeParams.setVolumeId(minLocation.volume_id);
##########
File path: fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
##########
@@ -1275,15 +1275,40 @@ private void computeScanRangeAssignmentByColocate(
}
}
- // randomly choose a backend from the TScanRangeLocations for a certain
bucket sequence.
- private void getExecHostPortForFragmentIDAndBucketSeq(TScanRangeLocations
seqLocation, PlanFragmentId fragmentId, Integer bucketSeq) throws Exception {
- int randomLocation = new
Random().nextInt(seqLocation.locations.size());
+ //ensure bucket sequence distribued to every host evenly
+ private void getExecHostPortForFragmentIDAndBucketSeq(TScanRangeLocations
seqLocation, PlanFragmentId fragmentId, Integer bucketSeq,
+
HashMap<TNetworkAddress, Long> assignedBytesPerHost) throws Exception {
Reference<Long> backendIdRef = new Reference<Long>();
- TNetworkAddress execHostPort =
SimpleScheduler.getHost(seqLocation.locations.get(randomLocation).backend_id,
seqLocation.locations, this.idToBackend, backendIdRef);
+ distributeHost(seqLocation, assignedBytesPerHost, backendIdRef);
+ Backend backend = this.idToBackend.get(backendIdRef.getRef());
+ TNetworkAddress execHostPort = new TNetworkAddress(backend.getHost(),
backend.getBePort());
this.addressToBackendID.put(execHostPort, backendIdRef.getRef());
this.fragmentIdToSeqToAddressMap.get(fragmentId).put(bucketSeq,
execHostPort);
}
+ public TScanRangeLocation distributeHost(TScanRangeLocations seqLocation,
Review comment:
ok. I will modify it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
