zuston commented on code in PR #1889:
URL:
https://github.com/apache/incubator-uniffle/pull/1889#discussion_r1679246269
##########
common/src/main/java/org/apache/uniffle/common/ShufflePartitionedData.java:
##########
@@ -21,12 +21,20 @@
public class ShufflePartitionedData {
+ private static final ShufflePartitionedBlock[] EMPTY_BLOCK_LIST =
+ new ShufflePartitionedBlock[] {};
private int partitionId;
- private ShufflePartitionedBlock[] blockList;
+ private final ShufflePartitionedBlock[] blockList;
+ private final long totalBlockSize;
public ShufflePartitionedData(int partitionId, ShufflePartitionedBlock[]
blockList) {
this.partitionId = partitionId;
- this.blockList = blockList;
+ this.blockList = blockList == null ? EMPTY_BLOCK_LIST : blockList;
+ long size = 0L;
Review Comment:
This loop to summary the size could be operated in the `toPartitionedBlock`
method, which could be avoid duplicate loop.
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerConf.java:
##########
@@ -492,6 +492,17 @@ public class ShuffleServerConf extends RssBaseConf {
+ HUGE_PARTITION_SIZE_THRESHOLD.key()
+ "'");
+ public static final ConfigOption<Long> MAX_PARTITION_SIZE_THRESHOLD =
+ ConfigOptions.key("rss.server.max-partition.size.threshold")
Review Comment:
How about `rss.server.partition.maxSizeThreshold` ? cc @jerqi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
