rickyma commented on PR #1759: URL: https://github.com/apache/incubator-uniffle/pull/1759#issuecomment-2144692335
emmm. This might be due to the relatively small memory configuration of the executor, which is actually an issue with the task resource configurations. How many reduce partitions do you have in your downstream stage? There must be a lot, right? This can also be optimized from the perspective of task configurations. How much memory is configured for your executors? What is the spill size value? How many reduce partitions are there in the downstream stage? Have you tried `rss.client.memory.spill.ratio` mentioned in https://github.com/apache/incubator-uniffle/issues/1727#issuecomment-2121933432. It will help you reduce small blocks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
