cloud-fan commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872219986
I think the normal case should be setting `COALESCE_PARTITIONS_MIN_PARTITION_NUM`to 1 and then tune the advisory size. This new config is really only for the default case where `COALESCE_PARTITIONS_MIN_PARTITION_NUM` is not set and falls back to the spark default parallelism. I'll mark it as an internal conf. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
