[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r62260803 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +64,22 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]])

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread rdblue
Github user rdblue commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217288532 Thank you @andrewor14 and all the reviewers! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11242 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217288054 Merging into master 2.0. Thanks for your work! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217283424 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217283422 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217283166 **[Test build #57913 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57913/consoleFull)** for PR 11242 at commit [`9b183cb`](https://g

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217263457 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217263452 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217263203 **[Test build #57907 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57907/consoleFull)** for PR 11242 at commit [`002353a`](https://g

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217263232 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217252564 **[Test build #57913 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57913/consoleFull)** for PR 11242 at commit [`9b183cb`](https://gi

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread rdblue
Github user rdblue commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217251801 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217251212 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217251188 **[Test build #57911 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57911/consoleFull)** for PR 11242 at commit [`9b183cb`](https://g

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217251208 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r62238592 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +64,22 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217248027 **[Test build #57911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57911/consoleFull)** for PR 11242 at commit [`9b183cb`](https://gi

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r62237486 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +64,22 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]])

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217245132 By the way, I just tried this out in the REPL and it still blocks behind user code: ``` import scala.concurrent.ExecutionContext.Implicits.global import s

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217237871 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217237865 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217237543 **[Test build #57902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57902/consoleFull)** for PR 11242 at commit [`2e90c5a`](https://g

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217230427 **[Test build #57907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57907/consoleFull)** for PR 11242 at commit [`002353a`](https://gi

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r62228103 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +64,21 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r62225084 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +64,21 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]])

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-217212503 **[Test build #57902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57902/consoleFull)** for PR 11242 at commit [`2e90c5a`](https://gi

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r62215672 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -56,14 +58,30 @@ private[spark] class UnionPartition[T: ClassTag]( } }

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r62215696 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -56,14 +58,30 @@ private[spark] class UnionPartition[T: ClassTag]( } }

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-05 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r62215656 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -56,14 +58,30 @@ private[spark] class UnionPartition[T: ClassTag]( } }

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-216710971 @rdblue Looks great! Much simpler than earlier patches. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r61979292 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -56,14 +58,30 @@ private[spark] class UnionPartition[T: ClassTag]( } }

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r61979221 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -56,14 +58,30 @@ private[spark] class UnionPartition[T: ClassTag]( } }

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r61979094 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]])

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r61978707 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -56,14 +58,30 @@ private[spark] class UnionPartition[T: ClassTag]( } }

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r61978625 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -56,14 +58,30 @@ private[spark] class UnionPartition[T: ClassTag]( } }

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-05-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r61978567 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -56,14 +58,30 @@ private[spark] class UnionPartition[T: ClassTag]( } }

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-19 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r60199645 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]])

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-211700178 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-211700176 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-211700015 **[Test build #56161 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56161/consoleFull)** for PR 11242 at commit [`96b188d`](https://g

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-211652005 **[Test build #56161 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56161/consoleFull)** for PR 11242 at commit [`96b188d`](https://gi

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-211629764 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-211629761 **[Test build #56155 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56155/consoleFull)** for PR 11242 at commit [`4fc9be4`](https://g

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-211629766 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-211629524 **[Test build #56155 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56155/consoleFull)** for PR 11242 at commit [`4fc9be4`](https://gi

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-18 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r60151772 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59111476 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) exte

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59062676 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59061019 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59059521 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59058588 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59058645 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59058248 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59057410 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59053387 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) exte

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59050309 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59048405 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59010483 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r59010259 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r58991517 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) exte

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r58977765 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207184221 LGTM. This PR dramatically improves our s3 performance at Netflix. @andrewor14 @srowen @JoshRosen @davies @marmbrus @yhuai, any further feedback? Thanks. ---

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207181095 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207181096 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207180917 **[Test build #55278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55278/consoleFull)** for PR 11242 at commit [`1633bd6`](https://g

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207141392 **[Test build #55278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55278/consoleFull)** for PR 11242 at commit [`1633bd6`](https://gi

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207140497 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207063759 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread rdblue
Github user rdblue commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207062268 I've updated this PR and rebased on the current master. As Sean suggested, I used the default thread pool and a parallel collection to do the work, so the commit is cons

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r58932013 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-23 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r57210970 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r57203700 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-23 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r57203120 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r57021251 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r57014360 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56968395 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56918938 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56894879 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56893742 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56892490 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56891305 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56890135 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56886896 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r5688 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56882466 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56880928 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56879711 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56878058 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-20 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56727449 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56432610 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,21 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread rdblue
Github user rdblue commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-197591766 @JoshRosen, I've added the test. Sorry for taking so long, my e-mail notifications weren't getting through for a little while. --- If your project is set up for it, you

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-198572978 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56719728 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56432618 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,21 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-198535461 **[Test build #53562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53562/consoleFull)** for PR 11242 at commit [`79d618e`](https://gi

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56747209 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56747153 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56746051 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,23 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-198572980 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

  1   2   >