Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/5762#discussion_r29301550
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala ---
@@ -245,6 +245,19 @@ case class Distinct(partial: Boolean, child:
SparkPlan) extends UnaryNode {
}
}
+/**
+ * :: DeveloperApi ::
+ * Return a new RDD that has exactly `numPartitions` partitions.
+ */
+@DeveloperApi
+case class Coalesce(numPartitions: Int, shuffle: Boolean, child:
SparkPlan) extends UnaryNode {
--- End diff --
@rxin Should I rename this to `Repartition`? There are a lot of conflicts
coming from `catalyst` and `sql`. In fact, the `Coalesce` function in catalyst
fits it's usage, which is to `combine (elements) in a mass or whole.`. Here, we
are basically repartitioning the dataset. `Coalesce` with a higher number of
partitions sounds weird. Also it might be weird to have two different types of
`Coalesce`. What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
