sunchao commented on pull request #35552: URL: https://github.com/apache/spark/pull/35552#issuecomment-1045438357
> I agree DataFrame has no problem on this given existence of repartition(). But how about SQL statement? Do we provide the same for SQL statement? Could end users inject the hint anywhere before operator and it will take effect on the exact place? My understanding is you can already provide hint in SQL (see SPARK-28746). _If_ we do need to introduce this (which is still under discussion here), I'm wondering why we have to bring back `HashClusteredDistribution` instead of just changing the behavior of `HashPartitioning.satisfies` according to the config. This config is also very similar to the existing `spark.sql.requireAllClusterKeysForCoPartition`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
