[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3249#issuecomment-68677425 [Test build #25049 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25049/consoleFull) for PR 3249 at commit

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3249#issuecomment-68677427 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-04 Thread jongyoul
GitHub user jongyoul opened a pull request: https://github.com/apache/spark/pull/3897 [SPARK-5088] Use spark-class for running executors directly You can merge this pull request into a Git repository by running: $ git pull https://github.com/jongyoul/spark SPARK-5088

[GitHub] spark pull request: SPARK-5087. [YARN] Merge yarn.Client and yarn....

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3896#issuecomment-68677294 [Test build #25048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25048/consoleFull) for PR 3896 at commit

[GitHub] spark pull request: SPARK-5087. [YARN] Merge yarn.Client and yarn....

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3896#issuecomment-68677297 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-1600] Refactor FileInputStream tests to...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3801#discussion_r22449971 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala --- @@ -281,102 +279,130 @@ class CheckpointSuite extends TestSuiteBase

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-68676178 [Test build #25051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25051/consoleFull) for PR 3897 at commit

[GitHub] spark pull request: [SPARK-5084][SQL]add if not exists after creat...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3895#issuecomment-68676566 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-1507][YARN]specify num of cores for AM

2015-01-04 Thread XuTingjun
Github user XuTingjun commented on the pull request: https://github.com/apache/spark/pull/3806#issuecomment-68676616 @sryza, I have splited spark.driver.memory into spark.driver.memory and spark.yarn.am.memory. Please have a look. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-5084][SQL]add if not exists after creat...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3895#issuecomment-68676564 [Test build #25050 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25050/consoleFull) for PR 3895 at commit

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-04 Thread jeanlyn
Github user jeanlyn commented on the pull request: https://github.com/apache/spark/pull/3891#issuecomment-68676770 hi, @marmbrus ,can you please take a look and give some suggestions?thx. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-5049: ParquetTableScan always prepends t...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3870#issuecomment-68632040 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-5049: ParquetTableScan always prepends t...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3870#issuecomment-68632038 [Test build #25031 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25031/consoleFull) for PR 3870 at commit

[GitHub] spark pull request: SPARK-5049: ParquetTableScan always prepends t...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3870#issuecomment-68630113 [Test build #25031 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25031/consoleFull) for PR 3870 at commit

[GitHub] spark pull request: [SPARK-5074][Core] Fix a non-deterministic tes...

2015-01-04 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/3889 [SPARK-5074][Core] Fix a non-deterministic test failure Add `assert(sc.listenerBus.waitUntilEmpty(WAIT_TIMEOUT_MILLIS))` to make sure `sparkListener` receive the message. You can merge this pull

[GitHub] spark pull request: [SPARK-5074][Core] Fix a non-deterministic tes...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3889#issuecomment-68630520 [Test build #25032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25032/consoleFull) for PR 3889 at commit

[GitHub] spark pull request: [SPARK-4229] Create hadoop configuration in a ...

2015-01-04 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3543#discussion_r22438855 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala --- @@ -789,7 +790,7 @@ class JavaPairDStream[K, V](val dstream:

[GitHub] spark pull request: [SPARK-5074][Core] Fix a non-deterministic tes...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3889#issuecomment-68632859 [Test build #25032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25032/consoleFull) for PR 3889 at commit

[GitHub] spark pull request: [SPARK-5074][Core] Fix a non-deterministic tes...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3889#issuecomment-68632862 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: add SVMWithLBFGS interface in MLLIB

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3890#issuecomment-68633831 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5062][Graphx] replace mapReduceTriplets...

2015-01-04 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3883#issuecomment-68638026 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-5036][Graphx]Better support sending par...

2015-01-04 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3866#issuecomment-68638018 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-5036][Graphx]Better support sending par...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3866#issuecomment-68638067 [Test build #25034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25034/consoleFull) for PR 3866 at commit

[GitHub] spark pull request: [SPARK-5062][Graphx] replace mapReduceTriplets...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3883#issuecomment-68638068 [Test build #25033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25033/consoleFull) for PR 3883 at commit

[GitHub] spark pull request: [SPARK-5036][Graphx]Better support sending par...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3866#issuecomment-68638104 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5062][Graphx] replace mapReduceTriplets...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3883#issuecomment-68638103 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5062][Graphx] replace mapReduceTriplets...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3883#issuecomment-68638101 [Test build #25033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25033/consoleFull) for PR 3883 at commit

[GitHub] spark pull request: [SPARK-5036][Graphx]Better support sending par...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3866#issuecomment-68638102 [Test build #25034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25034/consoleFull) for PR 3866 at commit

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-04 Thread jeanlyn
GitHub user jeanlyn opened a pull request: https://github.com/apache/spark/pull/3891 [SPARK-5068][SQL]fix bug query data when path doesn't exists the issue is descript on [SPARK-5068] (https://issues.apache.org/jira/browse/SPARK-5068) the purpose of this pull request is to

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3891#issuecomment-68639582 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-5059 - create README.md

2015-01-04 Thread xhudik
GitHub user xhudik opened a pull request: https://github.com/apache/spark/pull/3892 SPARK-5059 - create README.md Let's create a short README.md file to get a basic understanding of Spark REPL subproject You can merge this pull request into a Git repository by running: $ git

[GitHub] spark pull request: SPARK-5059 - create README.md

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3892#issuecomment-68640674 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-5059 - create README.md

2015-01-04 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/3892#issuecomment-68641442 Duplicating documentation is a good way for things to get out of sync. If we really need a README.md here, I'd prefer that it be little more than a link to where

[GitHub] spark pull request: SPARK-5059 - create README.md

2015-01-04 Thread xhudik
Github user xhudik commented on the pull request: https://github.com/apache/spark/pull/3892#issuecomment-68641889 since Spark REPL is just a mirror/clone I don't think it is a good idea to have some real documentation. I do think simple information that it is actually clone with a

[GitHub] spark pull request: [SPARK-4987] [SQL] parquet timestamp type supp...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3820#issuecomment-68624719 [Test build #25027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25027/consoleFull) for PR 3820 at commit

[GitHub] spark pull request: [SPARK-5062][Graphx] replace mapReduceTriplets...

2015-01-04 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3883#issuecomment-68624865 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5050][Mllib] Add unit test for sqdist

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3869#issuecomment-68624964 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5036][Graphx]Better support sending par...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3866#issuecomment-68624957 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5050][Mllib] Add unit test for sqdist

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3869#issuecomment-68624960 [Test build #25026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25026/consoleFull) for PR 3869 at commit

[GitHub] spark pull request: [SPARK-5036][Graphx]Better support sending par...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3866#issuecomment-68624956 [Test build #25029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25029/consoleFull) for PR 3866 at commit

[GitHub] spark pull request: [SPARK-5036][Graphx]Better support sending par...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3866#issuecomment-68624939 [Test build #25029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25029/consoleFull) for PR 3866 at commit

[GitHub] spark pull request: [SPARK-5069][Core] Fix the race condition of T...

2015-01-04 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/3887 [SPARK-5069][Core] Fix the race condition of TaskSchedulerImpl.dagScheduler It's not necessary to set `TaskSchedulerImpl.dagScheduler` in preStart. It's safe to set it after

[GitHub] spark pull request: [SPARK-5069][Core] Fix the race condition of T...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3887#issuecomment-68625684 [Test build #25030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25030/consoleFull) for PR 3887 at commit

[GitHub] spark pull request: [SPARK-4987] [SQL] parquet timestamp type supp...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3820#issuecomment-68626346 [Test build #25027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25027/consoleFull) for PR 3820 at commit

[GitHub] spark pull request: [SPARK-4987] [SQL] parquet timestamp type supp...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3820#issuecomment-68626348 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-4226: Add support for subqueries in wher...

2015-01-04 Thread saucam
GitHub user saucam opened a pull request: https://github.com/apache/spark/pull/3888 SPARK-4226: Add support for subqueries in where in clause this PR adds support for subquery in where in clause by adding a dynamic filter class that will compute the values list from the subquery

[GitHub] spark pull request: SPARK-4226: Add support for subqueries in wher...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3888#issuecomment-68626506 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-4226: Add support for subqueries in wher...

2015-01-04 Thread saucam
Github user saucam commented on the pull request: https://github.com/apache/spark/pull/3888#issuecomment-68626500 Hi, @marmbrus can you please take a look and suggest changes ; Have tested for a few queries and this approach looks simpler than an already existing PR. --- If

[GitHub] spark pull request: [SPARK-5069][Core] Fix the race condition of T...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3887#issuecomment-68627759 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5069][Core] Fix the race condition of T...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3887#issuecomment-68627756 [Test build #25030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25030/consoleFull) for PR 3887 at commit

[GitHub] spark pull request: SPARK-5059 - create README.md

2015-01-04 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3892#issuecomment-68646381 This is also not related to SPARK-5059. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4631] unit test for MQTT

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3844#issuecomment-68646513 [Test build #25035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25035/consoleFull) for PR 3844 at commit

[GitHub] spark pull request: [SPARK-4631] unit test for MQTT

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3844#issuecomment-68646517 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-04 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68646490 Will this work for broadcast variables as well? One thing I often see is users trying to directly broadcast an RDD without collecting it. --- If your project is set up

[GitHub] spark pull request: SPARK-1143: refactor TaskSchedulerImplSuite

2015-01-04 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/339#issuecomment-68647110 @CodingCat @kayousterhout What's the status of this PR? It hasn't been updated in several months. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68647489 [Test build #25036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25036/consoleFull) for PR 3884 at commit

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68647672 @sryza Good idea; I've added a new check which prevents RDDs from being directly broadcasted. I should probably add these checks to PySpark, too. I'm not

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-68647850 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-68647896 [Test build #25037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25037/consoleFull) for PR 3868 at commit

[GitHub] spark pull request: [SPARK-794][Core] Remove sleep() in ClusterSch...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3851#issuecomment-68647997 Alright, I'm going to merge this into `master` (1.3.0); after I do that, let's wait a bit and see if it adds any flakiness. If this does serve some purpose, then

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-04 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3871#issuecomment-68648254 @tgaloppo 2 more small comments, but after those, I believe this will be ready. Thanks! --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-4991][CORE] Worker should reconnect to ...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3825#issuecomment-68648457 @markhamstra From that page: Depending on the nature of the work to be supervised and the nature of the failure, the supervisor has a choice of the following

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-04 Thread tgaloppo
Github user tgaloppo commented on the pull request: https://github.com/apache/spark/pull/3871#issuecomment-68649845 @jkbradley Good call on the test suite; I have added some non-center points to the tests. I also added the brackets to the in-comment link. --- If your project is

[GitHub] spark pull request: spark-core - [SPARK-4787] - Stop sparkcontext ...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3809#issuecomment-68649791 Alright, I've re-scoped SPARK-4787 to only cover the changes here and made it into a subtask of the broader SPARK-4194 issue. So as not to let the perfect be the

[GitHub] spark pull request: spark-core - [SPARK-4787] - Stop sparkcontext ...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3809#issuecomment-68649927 Merged into `master` (1.3.0), `branch-1.2` (1.2.1), `branch-1.1` (1.1.2), and `branch-1.0` (1.0.3). --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: spark-core - [SPARK-4787] - Stop sparkcontext ...

2015-01-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3809 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68650048 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4991][CORE] Worker should reconnect to ...

2015-01-04 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/3825#issuecomment-68650323 @JoshRosen your thinking is that Master will be in good shape even though an exception has been thrown? If you can guarantee that, then resuming the actor while

[GitHub] spark pull request: [SPARK-5050][Mllib] Add unit test for sqdist

2015-01-04 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3869#issuecomment-68650535 @viirya LGTM Thanks! CC: @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-04 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3871#issuecomment-68650576 @tgaloppo Thanks for the updates! LGTM after tests pass CC: @mengxr --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-68652450 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-68652443 [Test build #25037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25037/consoleFull) for PR 3868 at commit

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3871#issuecomment-68652965 [Test build #25038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25038/consoleFull) for PR 3871 at commit

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3871#issuecomment-68652967 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-68654184 Could you add an unit test that tests this functionality. Something like this is easy undergo regression silently. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread cleaton
Github user cleaton commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-68656608 @tdas yes I think I can, I will take a look at it. Thanks for the feedback. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3868#discussion_r22444552 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -223,22 +227,40 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3868#discussion_r22444547 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -18,6 +18,8 @@ package

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3868#discussion_r22444560 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -285,16 +307,18 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3868#discussion_r22444563 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/TimeoutUtils.scala --- @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3868#discussion_r22444565 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/TimeoutUtils.scala --- @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3868#discussion_r22444576 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/TimeoutUtils.scala --- @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3868#discussion_r22444579 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/TimeoutUtils.scala --- @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3868#discussion_r22444573 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/TimeoutUtils.scala --- @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3868#discussion_r22444597 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -223,22 +227,40 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-68656939 As long as you're planning to do more work on this, I left a round of (kind of nitpicky) style comments

[GitHub] spark pull request: [SPARK-4140] Document dynamic allocation

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3731#issuecomment-68657238 @pwendell @andrewor14 Can we publish updated docs to the website? It looks like these changes aren't live yet:

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread cleaton
Github user cleaton commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-68657548 @JoshRosen Thank you for all the comments. I will update accordingly :). Regarding the waitUntilDone function, yes I assume there must be something like this in

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-68658085 Yeah, we can't / shouldn't use ScalaTest's `eventually` here; I was just noting the similarity. Maybe there's a library that has something like this, but I don't know

[GitHub] spark pull request: [SPARK-5078] Optionally read from SPARK_LOCAL_...

2015-01-04 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/3893 [SPARK-5078] Optionally read from SPARK_LOCAL_HOSTNAME Current spark lets you set the ip address using SPARK_LOCAL_IP, but then this is given to akka after doing a reverse DNS lookup. This makes

[GitHub] spark pull request: [SPARK-5078] Optionally read from SPARK_LOCAL_...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3893#issuecomment-68658688 [Test build #25039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25039/consoleFull) for PR 3893 at commit

[GitHub] spark pull request: [SPARK-4835] Disable validateOutputSpecs for S...

2015-01-04 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3832#issuecomment-68659162 Alright, I've pushed one more commit that explains the confusing `transform()` use and refers to https://issues.apache.org/jira/browse/SPARK-5079. Feel free to fix up

[GitHub] spark pull request: [SPARK-4835] Disable validateOutputSpecs for S...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3832#issuecomment-68659288 [Test build #25040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25040/consoleFull) for PR 3832 at commit

[GitHub] spark pull request: [SPARK-5062][Graphx] replace mapReduceTriplets...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3883#issuecomment-68661168 [Test build #25041 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25041/consoleFull) for PR 3883 at commit

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-68661860 [Test build #25043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25043/consoleFull) for PR 3247 at commit

[GitHub] spark pull request: [SPARK-4835] Disable validateOutputSpecs for S...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3832#issuecomment-68662296 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4835] Disable validateOutputSpecs for S...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3832#issuecomment-68662294 [Test build #25040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25040/consoleFull) for PR 3832 at commit

[GitHub] spark pull request: [SPARK-4229] Create hadoop configuration in a ...

2015-01-04 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/3543#discussion_r22446683 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala --- @@ -789,7 +790,7 @@ class JavaPairDStream[K, V](val

[GitHub] spark pull request: [SPARK-5062][Graphx] replace mapReduceTriplets...

2015-01-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3883#issuecomment-68664498 [Test build #25041 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25041/consoleFull) for PR 3883 at commit

[GitHub] spark pull request: [SPARK-5062][Graphx] replace mapReduceTriplets...

2015-01-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3883#issuecomment-68664502 Test FAILed. Refer to this link for build results (access rights to CI server needed):

  1   2   >