[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4074 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70611752 @srowen Thanks Sean, I committed this with a minor re-word of the title. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70470288 @pwendell No more changes from my side. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4074#discussion_r23139343
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -436,6 +436,12 @@ trait JavaRDDLike[T, This : JavaRDDLike[T, This]]
extends Serializable {
def first(): T = rdd.first()
/**
+ * @return true if and only if the RDD contains no elements at all. Note
that an RDD
+ * may be empty even when it has at least 1 partition.
+ */
+ def isEmpty(): Boolean = rdd.isEmpty()
--- End diff --
Sounds good.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70435986 LGTM @srowen - are you still working on it or is it good from your end? Will leave a bit of time for others to comment as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4074#discussion_r23136117
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -436,6 +436,12 @@ trait JavaRDDLike[T, This : JavaRDDLike[T, This]]
extends Serializable {
def first(): T = rdd.first()
/**
+ * @return true if and only if the RDD contains no elements at all. Note
that an RDD
+ * may be empty even when it has at least 1 partition.
+ */
+ def isEmpty(): Boolean = rdd.isEmpty()
--- End diff --
Ah, right of course. I forgot about this issue. OK, will update MiMa and
perhaps update the javadoc in `JavaRDDLike` to this effect?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4074#discussion_r23136110
--- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala ---
@@ -545,6 +546,12 @@ class RDDSuite extends FunSuite with
SharedSparkContext {
assert(sortedTopK === nums.sorted(ord).take(5))
}
+ test(isEmpty) {
+assert(sc.emptyRDD.isEmpty())
+assert(sc.parallelize(Seq[Int]()).isEmpty())
+assert(!sc.parallelize(Seq(1)).isEmpty())
--- End diff --
I think the `sc.parallelize(Seq[Int]()` case actually has multiple
partitions but I'll add this too. Also, I'll check the case where the first
partition is empty but others aren't.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70422508 [Test build #25730 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25730/consoleFull) for PR 4074 at commit [`191bb9f`](https://github.com/apache/spark/commit/191bb9fbb0066d314f7acc421e5febfe4aa12c2e). * This patch **does not merge cleanly**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70427761 [Test build #25731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25731/consoleFull) for PR 4074 at commit [`66885b8`](https://github.com/apache/spark/commit/66885b8f42cdce41294d35616e9a0c1775680200). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70430670 [Test build #25731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25731/consoleFull) for PR 4074 at commit [`66885b8`](https://github.com/apache/spark/commit/66885b8f42cdce41294d35616e9a0c1775680200). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70430674 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25731/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70426776 [Test build #25730 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25730/consoleFull) for PR 4074 at commit [`191bb9f`](https://github.com/apache/spark/commit/191bb9fbb0066d314f7acc421e5febfe4aa12c2e). * This patch **passes all tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70426779 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25730/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70362340 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70362399 [Test build #25701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25701/consoleFull) for PR 4074 at commit [`d76f8e3`](https://github.com/apache/spark/commit/d76f8e3cbe10f2ed5239281d6098d619640368d5). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70364288 [Test build #25701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25701/consoleFull) for PR 4074 at commit [`d76f8e3`](https://github.com/apache/spark/commit/d76f8e3cbe10f2ed5239281d6098d619640368d5). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70364290 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25701/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user ash211 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4074#discussion_r23125261
--- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala ---
@@ -545,6 +546,12 @@ class RDDSuite extends FunSuite with
SharedSparkContext {
assert(sortedTopK === nums.sorted(ord).take(5))
}
+ test(isEmpty) {
+assert(sc.emptyRDD.isEmpty())
+assert(sc.parallelize(Seq[Int]()).isEmpty())
+assert(!sc.parallelize(Seq(1)).isEmpty())
--- End diff --
I don't think this tests the case where there are multiple partitions but
no data in any of the partitions. Maybe add something like
`assert(sc.parallelize(Seq(1,2,3), 3).filter(_ 0).isEmpty())`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4074#discussion_r23129922
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -436,6 +436,12 @@ trait JavaRDDLike[T, This : JavaRDDLike[T, This]]
extends Serializable {
def first(): T = rdd.first()
/**
+ * @return true if and only if the RDD contains no elements at all. Note
that an RDD
+ * may be empty even when it has at least 1 partition.
+ */
+ def isEmpty(): Boolean = rdd.isEmpty()
--- End diff --
Okay sounds good @srowen want to just add an exclusion then?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/4074#discussion_r23129615
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -436,6 +436,12 @@ trait JavaRDDLike[T, This : JavaRDDLike[T, This]]
extends Serializable {
def first(): T = rdd.first()
/**
+ * @return true if and only if the RDD contains no elements at all. Note
that an RDD
+ * may be empty even when it has at least 1 partition.
+ */
+ def isEmpty(): Boolean = rdd.isEmpty()
--- End diff --
So this is actually a legitimate API break _if_ we think users are
themselves extending `JavaRDDLike` trait, because it will add a method to the
associated interface. One option is to just do it and ask users not to write
code that directly accepts or extends `JavaRDDLike`, and maybe we could
document that in the JavaDoc. Another option is just to add this to the
concrete implementations in JavaRDD and JavaPairRDD. @JoshRosen, any thoughts
one way or the other?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4074#discussion_r23129645
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -436,6 +436,12 @@ trait JavaRDDLike[T, This : JavaRDDLike[T, This]]
extends Serializable {
def first(): T = rdd.first()
/**
+ * @return true if and only if the RDD contains no elements at all. Note
that an RDD
+ * may be empty even when it has at least 1 partition.
+ */
+ def isEmpty(): Boolean = rdd.isEmpty()
--- End diff --
I've seen code that _accepts_ JavaRDDLike (e.g. as a parameter type), but I
can't think of a good reason to extend JavaRDDLike in user code (I couldn't
find any non-Spark-internal examples on GitHub code search). Therefore, I'd be
in favor of just adding a MiMa override and updating the documentation to
clarify that this interface isn't really designed for end-users.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4074#discussion_r23129767
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -436,6 +436,12 @@ trait JavaRDDLike[T, This : JavaRDDLike[T, This]]
extends Serializable {
def first(): T = rdd.first()
/**
+ * @return true if and only if the RDD contains no elements at all. Note
that an RDD
+ * may be empty even when it has at least 1 partition.
+ */
+ def isEmpty(): Boolean = rdd.isEmpty()
--- End diff --
Yea I don't think this should be an interface for users.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/4074 SPARK-5270 [CORE] Elegantly check if RDD is empty Pretty minor, but submitted for consideration -- this would at least help people make this check in the most efficient way I know. You can merge this pull request into a Git repository by running: $ git pull https://github.com/srowen/spark SPARK-5270 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/4074.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4074 commit de6b95e97908fb7d81ac831849085c65001c96f8 Author: Sean Owen so...@cloudera.com Date: 2015-01-16T16:49:31Z Add efficient RDD.isEmpty() --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70284499 [Test build #25667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25667/consoleFull) for PR 4074 at commit [`de6b95e`](https://github.com/apache/spark/commit/de6b95e97908fb7d81ac831849085c65001c96f8). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70285268 (Oh of course, if this looks good I can add this to Java / Python too) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70294498 [Test build #25667 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25667/consoleFull) for PR 4074 at commit [`de6b95e`](https://github.com/apache/spark/commit/de6b95e97908fb7d81ac831849085c65001c96f8). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70294506 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25667/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user ksakellis commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70296530 LTGM. What is the use case? is this part of a bigger pr? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70301808 This is all there is to it. It's just a convenience method that implements the check efficiently. Given several questions on the list, it seems that people do want to test for an empty RDD and there hasn't been an accepted way to do it that is faster than `count() == 0`: http://apache-spark-user-list.1001560.n3.nabble.com/Testing-if-an-RDD-is-empty-td1678.html#a1679 ... and of course http://issues.apache.org/jira/browse/SPARK-5270 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70310222 Seems reasonable to have since it's non obvious how to do it - @srowen could you add this in Java and Python? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70329801 [Test build #25682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25682/consoleFull) for PR 4074 at commit [`d76f8e3`](https://github.com/apache/spark/commit/d76f8e3cbe10f2ed5239281d6098d619640368d5). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70338533 [Test build #25682 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25682/consoleFull) for PR 4074 at commit [`d76f8e3`](https://github.com/apache/spark/commit/d76f8e3cbe10f2ed5239281d6098d619640368d5). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70338544 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25682/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
