zuston opened a new pull request, #2408: URL: https://github.com/apache/uniffle/pull/2408
### What changes were proposed in this pull request? Introduce the load_balance mode for partition split ### Why are the changes needed? Firstly, thanks the great work to @maobaolong . This work is based on the #2093, this introduces the load_balance mode for the partition split. As we know, if the partition is big, the partition split will be activated to reassign to another server. For the default impl, the reassign logic is pipeline. it will reassign for first server -> second -> third until reaching the max reassignment server num limit. But for the huge partition with huge write throughput at the same time, I hope this can write the multi servers for load balance to speed up writing. ### Does this PR introduce _any_ user-facing change? Yes. 1. `rss.client.reassign.partitionSplitMode`. Default value PIPELINE (that is consistent with previous codebase) 2. `rss.client.reassign.partitionSplitLoadBalanceServerNumber` . Default value is 10. Only valid for load balance mode. ### How was this patch tested? 1. Internal spark jobs tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
