maropu commented on pull request #29473: URL: https://github.com/apache/spark/pull/29473#issuecomment-679105200
Thanks for the work, @imback82. Just a question; we cannot set a simpler rule to select which strategy (reapportioning or coalescing) we use when reading buckets? I think it is a bit annoying to set true to `repartitionBucketsInJoin.enabled` or `coalesceBucketsInJoin.enabled` data-by-data. What's a factor to make a difference between the two strategies, e.g., bucket size? For example, if one has too small buckets (this case is extreme though), repartitioning for higher cardinality might not be able to help for better performance. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
