advancedxy commented on code in PR #471:
URL: https://github.com/apache/incubator-uniffle/pull/471#discussion_r1070964736
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerConf.java:
##########
@@ -342,6 +342,20 @@ public class ShuffleServerConf extends RssBaseConf {
.noDefaultValue()
.withDescription("The env key to get json source of local storage media
provider");
+ public static final ConfigOption<Long> HUGE_PARTITION_SIZE_THRESHOLD =
ConfigOptions
+ .key("rss.server.huge-partition.size.threshold")
+ .longType()
+ .defaultValue(20 * 1024 * 1024 * 1024L)
Review Comment:
> Yes. I think I will add a section about huge partition after all subtasks
are finished. WDYT?
Sounds good to me. You can create a new issue to tracking the documentation
issue or just polish this docs in the last subtask.
> This value depends on the capacity of a single disk. For example, the
single disk capacity is 1TB in our internal env, and I found the max size of
huge partition in one single disk is 5. So the total size of huge partition in
local disk is 100g (10%)，this is an acceptable config value.
This part will be described more after all subtasks are finished.
This is helpful, please do include this part when you are updating the
documentaion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
