rickyma commented on PR #1531: URL: https://github.com/apache/incubator-uniffle/pull/1531#issuecomment-1948807025
> 1. Do we need to modify the logic of method `pickFlushedShuffle`? You can refer to the comment [[#1472][part-2] fix(server): Reuse ByteBuf when decoding shuffle blocks instead of reallocating it #1521 (comment)](https://github.com/apache/incubator-uniffle/pull/1521#issuecomment-1942988443) > 2. Could extract some methods to make the logic more clearer? We reuse `highWaterMark` and `lowWaterMark`: ``` this.capacity = conf.getSizeAsBytes(ShuffleServerConf.SERVER_BUFFER_CAPACITY); if (this.capacity < 0) { this.capacity = nettyServerEnabled ? (long) (NettyUtils.getMaxDirectMemory() * conf.getDouble(ShuffleServerConf.SERVER_BUFFER_CAPACITY_RATIO)) : (long) (heapSize * conf.getDouble(ShuffleServerConf.SERVER_BUFFER_CAPACITY_RATIO)); } this.highWaterMark = (long) (capacity / 100.0 * conf.get(ShuffleServerConf.SERVER_MEMORY_SHUFFLE_HIGHWATERMARK_PERCENTAGE)); this.lowWaterMark = (long) (capacity / 100.0 * conf.get(ShuffleServerConf.SERVER_MEMORY_SHUFFLE_LOWWATERMARK_PERCENTAGE)); ``` So we don't need to modify the logic of method `pickFlushedShuffle`. The `pickedFlushSize` in `pickFlushedShuffle` will become `estimatedSize` in Netty mode, because it comes from `shuffleSizeMap` which will be modified in `cacheShuffleData` -> `updateShuffleSize`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
