Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/19467#discussion_r144403807
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
---
@@ -50,7 +50,8 @@ case class EnsureRequirements(conf: SQLConf) extends
Rule[SparkPlan] {
numPartitions: Int): Partitioning = {
requiredDistribution match {
case AllTuples => SinglePartition
- case ClusteredDistribution(clustering) =>
HashPartitioning(clustering, numPartitions)
+ case ClusteredDistribution(clustering, desiredPartitions) =>
--- End diff --
Update scala docs saying that numPartitions param is only if the
distribution does not specify it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
