[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/2843 [SPARK-3791][SQL][WIP] Provides Spark version and Hive version in HiveThriftServer2 This PR overrides the `GetInfo` Hive Thrift API to provide correct version information. Another property

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2843#discussion_r19058163 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLEnv.scala --- @@ -33,8 +33,10 @@ private[hive] object

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2843#discussion_r19058178 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala --- @@ -306,7 +306,9 @@ class HiveContext(sc: SparkContext) extends

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59640634 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21881/consoleFull) for PR 2843 at commit

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59640641 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2843#discussion_r19058181 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suite.scala --- @@ -37,35 +43,81 @@ import

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59640639 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21881/consoleFull) for PR 2843 at commit

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2576#issuecomment-59640730 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59641162 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/399/consoleFull) for PR 2843 at commit

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59641199 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/400/consoleFull) for PR 2843 at commit

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/2844 [SPARK-3958] TorrentBroadcast cleanup / debugging improvements. This PR makes several changes to TorrentBroadcast in order to make it easier to reason about, which should help when debugging

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59641699 /cc @rxin for review. I'd like to apply this to `branch-1.1` as well, since I believe that it's also affected by current TorrentBroadcast bugs. --- If your project

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59641733 Also, /cc @davies, who helped me to spot the local mode might deblockify an empty array bug and who's been working on TorrentBroadcast optimizations. --- If your

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59641757 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21882/consoleFull) for PR 2844 at commit

[GitHub] spark pull request: [SPARK-3902] [SPARK-3590] Stabilize AsynRDDAct...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2760#issuecomment-59641916 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21883/consoleFull) for PR 2760 at commit

[GitHub] spark pull request: [SPARK-3902] [SPARK-3590] Stabilize AsynRDDAct...

2014-10-19 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2760#issuecomment-59641956 @rxin Did you have any other feedback here? If not, I'd like to merge this. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-2546] Clone JobConf for each task (bran...

2014-10-19 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2684#issuecomment-59642012 I'm going to merge this and cherry-pick it into all maintenance branches. We'll probably turn on cloning by default in 1.2 and we'll be sure to clearly document this

[GitHub] spark pull request: [SPARK-2546] Clone JobConf for each task (bran...

2014-10-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2684 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/2845 [SPARK-4000][BUILD] Sends archived unit tests logs to Jenkins master This PR sends archived unit tests logs to the build history directory in Jenkins master, so that we can serve it via HTTP

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2845#issuecomment-59642454 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21884/consoleFull) for PR 2845 at commit

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59642510 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/399/consoleFull) for PR 2843 at commit

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59642634 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21882/consoleFull) for PR 2844 at commit

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59642636 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59643009 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21885/consoleFull) for PR 2844 at commit

[GitHub] spark pull request: [SPARK-3997][Build]scalastyle should output th...

2014-10-19 Thread witgo
GitHub user witgo opened a pull request: https://github.com/apache/spark/pull/2846 [SPARK-3997][Build]scalastyle should output the error location You can merge this pull request into a Git repository by running: $ git pull https://github.com/witgo/spark SPARK-3997

[GitHub] spark pull request: [SPARK-3997][Build]scalastyle should output th...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2846#issuecomment-59643139 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21886/consoleFull) for PR 2846 at commit

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59643215 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21887/consoleFull) for PR 2843 at commit

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59643247 Hm, 3 consecutive build failures, embarrassing... For the first one, unit tests are not started at all, seems that the build process is interrupted somehow.

[GitHub] spark pull request: [SPARK-3902] [SPARK-3590] Stabilize AsynRDDAct...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2760#issuecomment-59643261 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21883/consoleFull) for PR 2760 at commit

[GitHub] spark pull request: [SPARK-3902] [SPARK-3590] Stabilize AsynRDDAct...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2760#issuecomment-59643262 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59643445 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/400/consoleFull)** for PR 2843 at commit

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2845#issuecomment-59643839 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21884/consoleFull) for PR 2845 at commit

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59643838 It looks like this build is going to fail a ReplSuite test: ```scala test(broadcast vars) { // Test that the value that a broadcast var had when it

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2845#issuecomment-59643840 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59643971 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21885/consoleFull) for PR 2844 at commit

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59643974 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59644064 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21888/consoleFull) for PR 2844 at commit

[GitHub] spark pull request: [SPARK-3997][Build]scalastyle should output th...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2846#issuecomment-59644562 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3997][Build]scalastyle should output th...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2846#issuecomment-59644561 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21886/consoleFull) for PR 2846 at commit

[GitHub] spark pull request: [SPARK-4001][MLlib] adding apriori algorithm f...

2014-10-19 Thread jackylk
GitHub user jackylk opened a pull request: https://github.com/apache/spark/pull/2847 [SPARK-4001][MLlib] adding apriori algorithm for frequent item set mining in Spark Apriori is the classic algorithm for frequent item set mining in a transactional data set. It will be useful if

[GitHub] spark pull request: [SPARK-4001][MLlib] adding apriori algorithm f...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2847#issuecomment-59644841 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59645000 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2843#issuecomment-59644997 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21887/consoleFull) for PR 2843 at commit

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59645141 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59645137 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21888/consoleFull) for PR 2844 at commit

[GitHub] spark pull request: SPARK-3926 [CORE] Result of JavaRDD.collectAsM...

2014-10-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2805#issuecomment-59645175 Yes, all SGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-3989]Added possibility to directly inst...

2014-10-19 Thread ziky90
Github user ziky90 commented on the pull request: https://github.com/apache/spark/pull/2836#issuecomment-59645599 Ok, currently I'm using EMR instead of the spark-ec2 script, because it seems to me more convenient then connecting EC2 cluster from my own bash script, but you're right

[GitHub] spark pull request: [SPARK-3989]Added possibility to directly inst...

2014-10-19 Thread ziky90
Github user ziky90 closed the pull request at: https://github.com/apache/spark/pull/2836 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2845#discussion_r19059750 --- Diff: dev/run-tests-jenkins --- @@ -92,12 +92,39 @@ function post_message () { echo api_response: ${api_response} 2 echo data:

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2845#issuecomment-59647627 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21889/consoleFull) for PR 2845 at commit

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2014-10-19 Thread viper-kun
Github user viper-kun commented on the pull request: https://github.com/apache/spark/pull/2471#issuecomment-59648648 @vanzin. is it ok to go? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2845#issuecomment-59649466 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21889/consoleFull) for PR 2845 at commit

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2845#issuecomment-59649470 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3959][SPARK-3960][SQL] SqlParser fails ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2816#issuecomment-59649582 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21890/consoleFull) for PR 2816 at commit

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2845#issuecomment-59649597 Verified that the log archive was uploaded to the correct location in Jenkins master node. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-3677] [BUILD] [YARN] pom.xml and SparkB...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2520#issuecomment-59649721 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21891/consoleFull) for PR 2520 at commit

[GitHub] spark pull request: [SPARK-3959][SPARK-3960][SQL] SqlParser fails ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2816#issuecomment-59650495 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21890/consoleFull) for PR 2816 at commit

[GitHub] spark pull request: [SPARK-3959][SPARK-3960][SQL] SqlParser fails ...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2816#issuecomment-59650498 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3791][SQL][WIP] Provides Spark version ...

2014-10-19 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2843#discussion_r19060362 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -753,44 +753,19 @@ class HiveQuerySuite extends

[GitHub] spark pull request: [SPARK-3959][SPARK-3960][SQL] SqlParser fails ...

2014-10-19 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2816#issuecomment-59650806 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3959][SPARK-3960][SQL] SqlParser fails ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2816#issuecomment-59650907 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21892/consoleFull) for PR 2816 at commit

[GitHub] spark pull request: [SPARK-3677] [BUILD] [YARN] pom.xml and SparkB...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2520#issuecomment-59651589 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21891/consoleFull) for PR 2520 at commit

[GitHub] spark pull request: [SPARK-3677] [BUILD] [YARN] pom.xml and SparkB...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2520#issuecomment-59651592 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3959][SPARK-3960][SQL] SqlParser fails ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2816#issuecomment-59652376 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21892/consoleFull) for PR 2816 at commit

[GitHub] spark pull request: [SPARK-3959][SPARK-3960][SQL] SqlParser fails ...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2816#issuecomment-59652379 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59653126 This most recent test-failure is another side-effect of removing TorrentBroadcast's optimizations for local mode: ``` [info] - Unpersisting

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59653640 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21893/consoleFull) for PR 2844 at commit

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59655954 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21893/consoleFull) for PR 2844 at commit

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2844#issuecomment-59655958 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-19 Thread scwf
Github user scwf commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r19061946 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcTableOperations.scala --- @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2576#issuecomment-59660023 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21894/consoleFull) for PR 2576 at commit

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-19 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-59661579 @chouqin Thanks for the updates! The updates look good. One more small comment: Could you please add explicit checks in the unit tests to make sure the

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2576#issuecomment-59663743 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2844#discussion_r19063222 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -76,23 +87,20 @@ private[spark] class TorrentBroadcast[T: ClassTag](

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2844#discussion_r19063253 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -62,6 +59,20 @@ private[spark] class TorrentBroadcast[T: ClassTag](

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-19 Thread manishamde
Github user manishamde commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-59664449 @chouqin LGTM. :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2844#discussion_r19063271 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -156,6 +158,7 @@ private[spark] class TorrentBroadcast[T: ClassTag](

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/2844#discussion_r19063287 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -156,6 +158,7 @@ private[spark] class TorrentBroadcast[T: ClassTag](

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/2844#discussion_r19063336 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -76,23 +87,20 @@ private[spark] class TorrentBroadcast[T: ClassTag](

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-19 Thread manishamde
Github user manishamde commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-59664847 @jkbradley I read the paper by Sanku et al and other papers but they required a custom implementation. The sort method has worked OK so far but I was hoping somebody

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/2844#discussion_r19063363 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -62,6 +59,20 @@ private[spark] class TorrentBroadcast[T: ClassTag](

[GitHub] spark pull request: [SPARK-3958] TorrentBroadcast cleanup / debugg...

2014-10-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2844#discussion_r19063455 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -179,43 +183,29 @@ private[spark] class TorrentBroadcast[T: ClassTag](

[GitHub] spark pull request: Minor change in the comment of spark-defaults....

2014-10-19 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/2709#issuecomment-59667207 @andrewor14 Sorry for late reply since I was on vacation in Europe last week. I can continue work on this after I finish my talk in IOTA conf tomorrow. --- If your

[GitHub] spark pull request: SPARK-1209 [CORE] SparkHadoop{MapRed,MapReduce...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2814#issuecomment-59669342 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21895/consoleFull) for PR 2814 at commit

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-10-19 Thread ryan-williams
GitHub user ryan-williams opened a pull request: https://github.com/apache/spark/pull/2848 [SPARK-3967] don’t redundantly overwrite executor JAR deps You can merge this pull request into a Git repository by running: $ git pull https://github.com/ryan-williams/spark

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2848#issuecomment-59669916 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3888] [PySpark] limit the memory used b...

2014-10-19 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2743#discussion_r19064542 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -63,9 +64,12 @@ private[spark] class PythonRDD( val localdir =

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2845#discussion_r19064621 --- Diff: dev/run-tests-jenkins --- @@ -92,12 +92,39 @@ function post_message () { echo api_response: ${api_response} 2 echo data:

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2845#discussion_r19064633 --- Diff: dev/run-tests-jenkins --- @@ -92,12 +92,39 @@ function post_message () { echo api_response: ${api_response} 2 echo data:

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2845#discussion_r19064648 --- Diff: dev/run-tests-jenkins --- @@ -92,12 +92,39 @@ function post_message () { echo api_response: ${api_response} 2 echo data:

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2845#issuecomment-59671283 Overall this looks good, had a few minor questions. One thing we can do next is that we could scp the logs to a web server that we control (e.g. something under

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2845#discussion_r19064710 --- Diff: dev/run-tests-jenkins --- @@ -92,12 +92,39 @@ function post_message () { echo api_response: ${api_response} 2 echo data:

[GitHub] spark pull request: [SPARK-3902] [SPARK-3590] Stabilize AsynRDDAct...

2014-10-19 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2760#issuecomment-59672002 LGTM - we discussed some details of this offline last week. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: SPARK-1209 [CORE] SparkHadoop{MapRed,MapReduce...

2014-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2814#issuecomment-59672188 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2845#discussion_r19064925 --- Diff: dev/run-tests-jenkins --- @@ -92,12 +92,39 @@ function post_message () { echo api_response: ${api_response} 2 echo data:

[GitHub] spark pull request: SPARK-1209 [CORE] SparkHadoop{MapRed,MapReduce...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2814#issuecomment-59672186 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21895/consoleFull) for PR 2814 at commit

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2845#discussion_r19064927 --- Diff: dev/run-tests-jenkins --- @@ -92,12 +92,39 @@ function post_message () { echo api_response: ${api_response} 2 echo data:

[GitHub] spark pull request: [spark-3907][sql] add truncate table support

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2770#issuecomment-59672228 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/402/consoleFull) for PR 2770 at commit

[GitHub] spark pull request: [SPARK-3904] [SQL] add constant objectinspecto...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2762#issuecomment-59672239 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/403/consoleFull) for PR 2762 at commit

[GitHub] spark pull request: [MLLIB] [WIP] SPARK-1547: Adding Gradient Boos...

2014-10-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2607#issuecomment-59672538 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21896/consoleFull) for PR 2607 at commit

  1   2   >