[GitHub] spark pull request: [YARN] Delete confusion configurations

2014-12-23 Thread XuTingjun
GitHub user XuTingjun opened a pull request: https://github.com/apache/spark/pull/3776 [YARN] Delete confusion configurations First, I think put master/work config into yarn mode is confusion. Second, it will add the config twice. For example, if SPARK_WORKER_INSTANCES is null

[GitHub] spark pull request: [YARN] Delete confusion configurations

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3776#issuecomment-67929626 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4928][SQL] Fix: Operator ',,=,=' wi...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3767#issuecomment-67930166 [Test build #24733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24733/consoleFull) for PR 3767 at commit

[GitHub] spark pull request: [SPARK-4931][Yarn][Docs] Fix the format of run...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3774#issuecomment-67930240 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4931][Yarn][Docs] Fix the format of run...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3774#issuecomment-67930237 [Test build #24731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24731/consoleFull) for PR 3774 at commit

[GitHub] spark pull request: [SPARK-4933][CORE] eventlog file not found

2014-12-23 Thread liyezhang556520
GitHub user liyezhang556520 opened a pull request: https://github.com/apache/spark/pull/3777 [SPARK-4933][CORE] eventlog file not found enent log file not found exception will be thrown after making eventLog into a single file. And web ui will show No event logs found for

[GitHub] spark pull request: [SPARK-4933][CORE] eventlog file not found

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3777#issuecomment-67930447 [Test build #24734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24734/consoleFull) for PR 3777 at commit

[GitHub] spark pull request: [YARN] Delete confusion configurations

2014-12-23 Thread SaintBacchus
Github user SaintBacchus commented on the pull request: https://github.com/apache/spark/pull/3776#issuecomment-67930492 I think `SPARK_WORKER_INSTANCES` was used for compatibility of previous version such as *spark-0.9.0*. Did we need remove it now @XuTingjun ? --- If your

[GitHub] spark pull request: [SPARK-3586][streaming]Support nested director...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2765#issuecomment-67931020 [Test build #24735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24735/consoleFull) for PR 2765 at commit

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-67931131 [Test build #24732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24732/consoleFull) for PR 3755 at commit

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-67931138 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4094][CORE] checkpoint should still be ...

2014-12-23 Thread dding3
Github user dding3 commented on the pull request: https://github.com/apache/spark/pull/2956#issuecomment-67931182 We have tested the patch in below senarios and find it works : 1. Apply checkpoint. RDD has been flush to disk as expected 2. Doesn't apply checkpoint. There is no

[GitHub] spark pull request: [SPARK-4928][SQL] Fix: Operator ',,=,=' wi...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3767#issuecomment-67932261 [Test build #24736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24736/consoleFull) for PR 3767 at commit

[GitHub] spark pull request: [SPARK-4928][SQL] Fix: Operator ',,=,=' wi...

2014-12-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3767#issuecomment-67932445 LGTM as long as Jenkins says yes. Thanks for working on this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-1442] [SQL] window function implement

2014-12-23 Thread guowei2
Github user guowei2 closed the pull request at: https://github.com/apache/spark/pull/2953 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SQL] spark-sql aborted if passed in a wrong s...

2014-12-23 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/3761#issuecomment-67933179 Hi @liancheng, this is a very interesting bug i think, you place a hive-site.xml into spark/conf, this will happen. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-4928][SQL] Fix: Operator ',,=,=' wi...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3767#issuecomment-67934519 [Test build #24733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24733/consoleFull) for PR 3767 at commit

[GitHub] spark pull request: [SPARK-4928][SQL] Fix: Operator ',,=,=' wi...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3767#issuecomment-67934528 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SQL] spark-sql aborted if passed in a wrong s...

2014-12-23 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/3761#issuecomment-67934561 If we config ``` property namehive.cli.print.header/name valuetrue/value /property ``` spark-sql will abort. --- If your project is set up

[GitHub] spark pull request: [SQL] spark-sql aborted if passed in a wrong s...

2014-12-23 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/3761#issuecomment-67934889 This is because it always return ResponseCode(0), even if we pass a invalid sql. So ``` if (ret != 0) {

[GitHub] spark pull request: [SPARK-4860][pyspark][sql] speeding up `sample...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3764#issuecomment-67935475 [Test build #24737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24737/consoleFull) for PR 3764 at commit

[GitHub] spark pull request: [SPARK-4933][CORE] eventlog file not found

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3777#issuecomment-67935702 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4933][CORE] eventlog file not found

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3777#issuecomment-67935695 [Test build #24734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24734/consoleFull) for PR 3777 at commit

[GitHub] spark pull request: [SPARK-3586][streaming]Support nested director...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2765#issuecomment-67936460 [Test build #24735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24735/consoleFull) for PR 2765 at commit

[GitHub] spark pull request: [SPARK-4928][SQL] Fix: Operator ',,=,=' wi...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3767#issuecomment-67936918 [Test build #24736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24736/consoleFull) for PR 3767 at commit

[GitHub] spark pull request: [SPARK-4928][SQL] Fix: Operator ',,=,=' wi...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3767#issuecomment-67936922 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4797] Replace breezeSquaredDistance

2014-12-23 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/3643#issuecomment-67938403 I have not run pyspark tests. Fixed in update. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4797] Replace breezeSquaredDistance

2014-12-23 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/3643#issuecomment-67938482 Please test it again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4860][pyspark][sql] speeding up `sample...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3764#issuecomment-67942526 [Test build #24737 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24737/consoleFull) for PR 3764 at commit

[GitHub] spark pull request: [SPARK-4386] Improve performance when writing ...

2014-12-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3254#issuecomment-67948280 @MickDavies Thanks for pinning down the hotspot, looking forward to your PR :) --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-4935][SQL] When hive.cli.print.header c...

2014-12-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3761#issuecomment-67949598 @scwf Thanks for the explanation! Then this change makes sense and LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-4938][SQL] Adding optimization to simpl...

2014-12-23 Thread scwf
GitHub user scwf opened a pull request: https://github.com/apache/spark/pull/3778 [SPARK-4938][SQL] Adding optimization to simplify the filter condition Adding optimization to simplify the filter condition(https://issues.apache.org/jira/browse/SPARK-4938) You can merge this pull

[GitHub] spark pull request: [SPARK-4938][SQL] Adding optimization to simpl...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-67950891 [Test build #24738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24738/consoleFull) for PR 3778 at commit

[GitHub] spark pull request: [SPARK-4938][SQL] Adding optimization to simpl...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-67950961 [Test build #24738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24738/consoleFull) for PR 3778 at commit

[GitHub] spark pull request: [SPARK-4938][SQL] Adding optimization to simpl...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-67950963 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4938][SQL] Adding optimization to simpl...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-67951710 [Test build #24739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24739/consoleFull) for PR 3778 at commit

[GitHub] spark pull request: [SPARK-4938][SQL] Adding optimization to simpl...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-67952573 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4938][SQL] Adding optimization to simpl...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-67952570 [Test build #24739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24739/consoleFull) for PR 3778 at commit

[GitHub] spark pull request: [SPARK-4730][YARN] Warn against deprecated YAR...

2014-12-23 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/3590#issuecomment-67955599 +1 looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4386] Improve performance when writing ...

2014-12-23 Thread jimfcarroll
Github user jimfcarroll commented on the pull request: https://github.com/apache/spark/pull/3254#issuecomment-67960092 Thanks @MickDavies. I was JUST about to start profiling it again. This is the same scala class I originally had issues with. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-2883][SQL] Orc support through datasour...

2014-12-23 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/3753#issuecomment-67961227 Thanks @liancheng, so when we have this interface to support partitioned tables, or anyone is work on it? Now the partitioned table support in Orc referred to parquet

[GitHub] spark pull request: [SPARK-4935][SQL] When hive.cli.print.header c...

2014-12-23 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3761#discussion_r3165 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -278,7 +278,6 @@ private[hive] class

[GitHub] spark pull request: [SPARK-4860][pyspark][sql] speeding up `sample...

2014-12-23 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3764#issuecomment-67973120 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4860][pyspark][sql] speeding up `sample...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3764#issuecomment-67973506 [Test build #24740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24740/consoleFull) for PR 3764 at commit

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-67978058 [Test build #24741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24741/consoleFull) for PR 3571 at commit

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-67978159 [Test build #24741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24741/consoleFull) for PR 3571 at commit

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-67978163 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-23 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-67984238 LGTM. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-4860][pyspark][sql] speeding up `sample...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3764#issuecomment-67984392 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4860][pyspark][sql] speeding up `sample...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3764#issuecomment-67984383 [Test build #24740 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24740/consoleFull) for PR 3764 at commit

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-67984571 @jacek-lewandowski my understanding is that you don't need the master to know the public key of every executor. The master can either disable cert validation, or have a

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-67985287 About style checks - I know... I've pushed it too fast --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-67985290 [Test build #24743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24743/consoleFull) for PR 3571 at commit

[GitHub] spark pull request: [SPARK-4890] Ignore downloaded EC2 libs

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3770#issuecomment-67987338 LGTM, so I've merged this into `master` (1.3.0). Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4890] Ignore downloaded EC2 libs

2014-12-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3770 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4932] Add help comments in Analytics

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3775#issuecomment-67987463 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4932] Add help comments in Analytics

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3775#issuecomment-67987563 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4937][SQL] Adding optimization to simpl...

2014-12-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-67987903 Hey @scwf, I'm kinda lost in all the predicates provided in the PR description... Would you mind to provide several simpler cases to illustrate what kind of

[GitHub] spark pull request: [SPARK-4932] Add help comments in Analytics

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3775#issuecomment-67987852 [Test build #24744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24744/consoleFull) for PR 3775 at commit

[GitHub] spark pull request: [SPARK-4931][Yarn][Docs] Fix the format of run...

2014-12-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3774 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-67988310 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-67988843 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24745/consoleFull) for PR 3768 at commit

[GitHub] spark pull request: [SPARK-4813][Streaming] Fix the issue that Con...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3661#discussion_r22230703 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ContextWaiter.scala --- @@ -17,30 +17,63 @@ package org.apache.spark.streaming

[GitHub] spark pull request: [SPARK-4920][UI] add version on master and wor...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3769#issuecomment-67990199 Here's a screenshot of what this looks like on various pages: ## Master:

[GitHub] spark pull request: [SPARK-4857] [CORE] Adds Executor membership e...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3711#discussion_r22231207 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -19,6 +19,8 @@ package org.apache.spark.scheduler import

[GitHub] spark pull request: [SPARK-4857] [CORE] Adds Executor membership e...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3711#discussion_r22231341 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -84,6 +86,14 @@ case class SparkListenerBlockManagerRemoved(time: Long,

[GitHub] spark pull request: [SPARK-4857] [CORE] Adds Executor membership e...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3711#discussion_r22231382 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -109,7 +119,8 @@ private[spark] case object SparkListenerShutdown

[GitHub] spark pull request: [SPARK-4857] [CORE] Adds Executor membership e...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3711#discussion_r22231531 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorInfo.scala --- @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-67991523 [Test build #24743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24743/consoleFull) for PR 3571 at commit

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-67991531 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4834] [standalone] Clean up application...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3705#issuecomment-67991646 [Test build #24742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24742/consoleFull) for PR 3705 at commit

[GitHub] spark pull request: [SPARK-4834] [standalone] Clean up application...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3705#issuecomment-67991653 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4797] Replace breezeSquaredDistance

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3643#issuecomment-67992209 [Test build #553 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/553/consoleFull) for PR 3643 at commit

[GitHub] spark pull request: [SPARK-4501][Core] - Create build/mvn to autom...

2014-12-23 Thread brennonyork
Github user brennonyork commented on the pull request: https://github.com/apache/spark/pull/3707#issuecomment-67992254 @pwendell updated the script per the discussion, updated all places where `sbt/sbt` was referenced and replaced it with `build/sbt`, added the `sbt/sbt` stub, and

[GitHub] spark pull request: [SPARK-4834] [standalone] Clean up application...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3705#issuecomment-67992288 That most recent failure is due to a known flaky test, _org.apache.spark.streaming.CheckpointSuite.recovery with file input stream_, so I'm not going to make Jenkins

[GitHub] spark pull request: [SPARK-4834] [standalone] Clean up application...

2014-12-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3705 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4939] move to next locality when no pen...

2014-12-23 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/3779 [SPARK-4939] move to next locality when no pending tasks for executors Currently, if there are different locality in a task set, the tasks with NODE_LOCAL only get scheduled after all the

[GitHub] spark pull request: [SPARK-4939] move to next locality when no pen...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3779#issuecomment-67992975 [Test build #24746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24746/consoleFull) for PR 3779 at commit

[GitHub] spark pull request: [SPARK-4501][Core] - Create build/mvn to autom...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3707#issuecomment-67993395 It looks like there's a merge conflict in `dev/scalastyle`; mind fixing that? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: SPARK-4547 [MLLIB] OOM when making bins in Bin...

2014-12-23 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3702#discussion_r22232433 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -28,9 +28,28 @@ import

[GitHub] spark pull request: [SPARK-4501][Core] - Create build/mvn to autom...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3707#discussion_r22232479 --- Diff: docs/building-spark.md --- @@ -189,22 +200,22 @@ compilation. More advanced developers may wish to use SBT. The SBT build is derived from

[GitHub] spark pull request: [SPARK-4501][Core] - Create build/mvn to autom...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3707#issuecomment-67993676 I'm testing this out on EC2 right now and it's pretty nice, especially since it's kind of a pain to install Maven since it's not in any of the default EC2 package

[GitHub] spark pull request: SPARK-4547 [MLLIB] OOM when making bins in Bin...

2014-12-23 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3702#issuecomment-67993599 @srowen I added one last comment; sorry, I should have thought of it earlier. This took longer to do that I expected, but I think it's a good solution. Thanks for

[GitHub] spark pull request: [SPARK-4932] Add help comments in Analytics

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3775#issuecomment-67995955 [Test build #24744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24744/consoleFull) for PR 3775 at commit

[GitHub] spark pull request: [SPARK-4932] Add help comments in Analytics

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3775#issuecomment-67995964 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-67996167 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24745/consoleFull) for PR 3768 at commit

[GitHub] spark pull request: [SPARK-4932] Add help comments in Analytics

2014-12-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3775 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-67996175 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4932] Add help comments in Analytics

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3775#issuecomment-67996178 I've merged this into `master` (1.3.0) and `branch-1.2` (1.2.1). Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-67996364 I agree that the test failure seems unrelated to this patch, but we should definitely investigate it (maybe separately): ``` OK (skipped=1) Traceback

[GitHub] spark pull request: [SPARK-4914][Build] Cleans lib_managed before ...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3756#issuecomment-67997413 LGTM, so I'll merge this into `master` (1.3.0) and `branch-1.2` (1.2.1). Thanks for the fix! --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-4914][Build] Cleans lib_managed before ...

2014-12-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3756 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-67998860 @vanzin what about this failure? This what was thrown before and I have no idea what is the reason because I cannot replicate it locally. --- If your project

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2014-12-23 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-6720 Not sure without digging through the code and trying it out myself. Make sure you don't have a configuration file lying around in your `$SPARK_HOME/conf` that is not

[GitHub] spark pull request: [SPARK-4797] Replace breezeSquaredDistance

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3643#issuecomment-6769 [Test build #553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/553/consoleFull) for PR 3643 at commit

[GitHub] spark pull request: [SPARK-4939] move to next locality when no pen...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3779#issuecomment-68000315 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4939] move to next locality when no pen...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3779#issuecomment-68000310 [Test build #24746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24746/consoleFull) for PR 3779 at commit

[GitHub] spark pull request: SPARK-4843 [YARN] Squash ExecutorRunnableUtil ...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3696#discussion_r22237217 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala --- @@ -50,11 +51,12 @@ class ExecutorRunnable( executorCores:

[GitHub] spark pull request: SPARK-4843 [YARN] Squash ExecutorRunnableUtil ...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3696#issuecomment-68001717 I downloaded this and used vimdiff to confirm that this is just code movement. Left one super-minor comment about the re-ordering of two statements in the

[GitHub] spark pull request: SPARK-2450 Adds exeuctor log links to Web UI

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3486#issuecomment-68002119 It looks like this PR is now blocked on #3711, your PR to add the executor added/removed events to SparkListener. That one looks like it should be just about ready to

  1   2   3   4   >