jerqi commented on code in PR #2492:
URL: https://github.com/apache/uniffle/pull/2492#discussion_r2113118211
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerConf.java:
##########
@@ -499,6 +499,46 @@ public class ShuffleServerConf extends RssBaseConf {
"Threshold when flushing shuffle data to persistent storage,
recommend value would be 256K, "
+ "512K, or even 1M");
+ public static final ConfigOption<Boolean> SERVER_SHUFFLE_BUFFER_LAB_ENABLE =
+ ConfigOptions.key("rss.server.shuffleBuffer.lab.enable")
+ .booleanType()
+ .defaultValue(true)
+ .withDescription("Whether enable LAB(Local allocation buffer) for
shuffle buffers.");
+
+ public static final ConfigOption<Integer>
SERVER_SHUFFLE_BUFFER_LAB_CHUNK_SIZE =
+ ConfigOptions.key("rss.server.shuffleBuffer.lab.chunk.size")
+ .intType()
+ .defaultValue(1024 * 1024) // 1MB
+ .withDescription(
+ "Defines the pre-allocated chunk size per partition for LAB. "
+ + "Each partition reserves one chunk of this size. "
+ + "Larger values may cause memory waste when processing many
partitions, "
+ + "while smaller values generate excessive small chunks,
increasing GC frequency and overhead. "
+ + "Configure based on expected data size, available memory,
and GC tolerance to balance efficiency.");
+
+ public static final ConfigOption<Double>
SERVER_SHUFFLE_BUFFER_LAB_MAX_ALLOC_RATIO =
+ ConfigOptions.key("rss.server.shuffleBuffer.lab.maxAlloc.ratio")
+ .doubleType()
+ .checkValue(
+ ConfigUtils.DOUBLE_VALIDATOR_ZERO_TO_ONE,
+ "The lab max alloc ratio must be between 0.0 and 1.0")
+ .defaultValue(0.2)
+ .withDescription(
+ "If the block size is not small, we don't need to put it in the
chunk."
+ + " If the ratio is 0.2, it means the blocks which size is
less or equal than "
+ + SERVER_SHUFFLE_BUFFER_LAB_CHUNK_SIZE.key()
+ + " * 0.2 will be put in the chunk.");
+
+ public static final ConfigOption<Double>
SERVER_SHUFFLE_BUFFER_LAB_CHUNK_POOL_CAPACITY_RATIO =
+
ConfigOptions.key("rss.server.shuffleBuffer.lab.chunkPool.capacity.ratio")
Review Comment:
Could you use camel style for last segment only?
The middle segment don't use camel style.
`shuffleBuffer` -> `buffer`
`lab.chunkPool.capacity.ratio` -> `labChunkPoolCapacityRatio`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
