Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/8511#discussion_r38265914
--- Diff:
core/src/main/scala/org/apache/spark/rdd/MapPartitionsWithPreparationRDD.scala
---
@@ -24,6 +24,8 @@ import org.apache.spark.{Partition, Partitioner,
TaskContext}
/**
* An RDD that applies a user provided function to every partition of the
parent RDD, and
* additionally allows the user to prepare each partition before computing
the parent partition.
+ *
+ * TODO(davies): remove this once SPARK-10342 is fixed
--- End diff --
It is also unclear we want to remove this even with better memory
management. You might still want this with those. I would just remove this
comment.
On Aug 29, 2015, at 1:37 PM, Yin Huai <[email protected]> wrote:
In
core/src/main/scala/org/apache/spark/rdd/MapPartitionsWithPreparationRDD.scala
<https://github.com/apache/spark/pull/8511#discussion_r38265746>:
> @@ -24,6 +24,8 @@ import org.apache.spark.{Partition, Partitioner,
TaskContext}
> /**
> * An RDD that applies a user provided function to every partition of
the parent RDD, and
> * additionally allows the user to prepare each partition before
computing the parent partition.
> + *
> + * TODO(davies): remove this once SPARK-10342 is fixed
Based on this comment, it is not clear what need to be removed (remove this
class or remove changes of this PR). Can you add a comment in SPARK-10342
about what need to be removed?
â
Reply to this email directly or view it on GitHub
<https://github.com/apache/spark/pull/8511/files#r38265746>.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
