dybyte commented on PR #9833: URL: https://github.com/apache/seatunnel/pull/9833#issuecomment-3268640452
> Can we provide a best practice to guide customers in configuration, such as recommended calculation formulas @zhangshenghang Based on local performance tests, increasing the partition count provides significant benefits when the number of tasks exceeds approximately 20,000. As a practical guideline, a partition count of around 2,000 tends to offer the best balance between reducing lock contention and minimizing overhead. It is recommended to start with this value and then adjust based on your cluster size and workload characteristics. Please note that these recommendations are based on my test results, and the optimal values may vary depending on your environment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
