zuston commented on code in PR #2093:
URL:
https://github.com/apache/incubator-uniffle/pull/2093#discussion_r1746866503
##########
client-spark/common/src/main/java/org/apache/spark/shuffle/handle/MutableShuffleHandleInfo.java:
##########
@@ -52,7 +52,7 @@ public class MutableShuffleHandleInfo extends
ShuffleHandleInfoBase {
*/
private Map<Integer, Map<Integer, List<ShuffleServerInfo>>>
partitionReplicaAssignedServers;
- private Map<String, Set<ShuffleServerInfo>> excludedServerToReplacements;
+ private Map<String, List<ShuffleServerInfo>> excludedServerToReplacements;
Review Comment:
If using list instead of set, is it possible to store the duplicate
replacement servers?
##########
server/src/main/java/org/apache/uniffle/server/HugePartitionUtils.java:
##########
@@ -148,4 +148,9 @@ public static boolean limitHugePartition(
}
return false;
}
+
+ public static boolean hasExceedPartitionSplitLimit(
Review Comment:
Why not extract the general logic to limit the partitioned data? I think the
partition split and huge partition limit could be scoped in the same abstract
limit policies.
##########
proto/src/main/proto/Rss.proto:
##########
@@ -62,6 +62,8 @@ message RequireBufferResponse {
int64 requireBufferId = 1;
StatusCode status = 2;
string retMsg = 3;
+ // need split partitions
+ repeated int32 partitionIds = 4;
Review Comment:
why not marking the partitionIds name as the `needSplitPartitionIds`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
