viirya commented on issue #23627: [SPARK-26672][SQL] SinglePartition may not satisfies HashClusteredDistribu… URL: https://github.com/apache/spark/pull/23627#issuecomment-456759301 > Data inserted into TEST_TABLE will not be bucketed since after LimitExec the output partitioning will be SinglePartition, and in current logic, it satisfies the HashClusteredDistribution, so no shuffle will be added. Have you observed such issue in real case? I think bucketing is added when writing data to files. In single partition bucketing should work.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
