zuston commented on code in PR #471:
URL: https://github.com/apache/incubator-uniffle/pull/471#discussion_r1070868831
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerConf.java:
##########
@@ -342,6 +342,20 @@ public class ShuffleServerConf extends RssBaseConf {
.noDefaultValue()
.withDescription("The env key to get json source of local storage media
provider");
+ public static final ConfigOption<Long> HUGE_PARTITION_SIZE_THRESHOLD =
ConfigOptions
+ .key("rss.server.huge-partition.size.threshold")
+ .longType()
+ .defaultValue(20 * 1024 * 1024 * 1024L)
Review Comment:
> Is 20GB has some meaningful background?
This value depends on the capacity of a single disk. For example, the single
disk capacity is 1TB in our internal env, and I found the max size of huge
partition in one single disk is 5. So the total size of huge partition in local
disk is 100g (10%)，this is an acceptable config value.
This part will be described more after all subtasks are finished.
> It would be great if you could add some suggestion values in the doc
description.
Also could you please update the configuration doc?
Yes. I think I will add a section about huge partition after all subtasks
are finished. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
