[GitHub] spark pull request #15154: [SPARK-17494] [SQL] changePrecision() on compact ...

2016-09-19 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/15154 [SPARK-17494] [SQL] changePrecision() on compact decimal should respect rounding mode ## What changes were proposed in this pull request? Floor()/Ceil() of decimal is implemented using

[GitHub] spark issue #15154: [SPARK-17494] [SQL] changePrecision() on compact decimal...

2016-09-19 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15154 cc @JoshRosen @chenghao-intel --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15090: [SPARK-17073] [SQL] generate column-level statist...

2016-09-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15090#discussion_r79496929 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -0,0 +1,159 @@ +/* + * Licensed to

[GitHub] spark pull request #15090: [SPARK-17073] [SQL] generate column-level statist...

2016-09-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15090#discussion_r79496528 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -0,0 +1,159 @@ +/* + * Licensed to

[GitHub] spark issue #15146: [SPARK-17590][SQL] Analyze CTE definitions at once and a...

2016-09-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15146 @viirya does this improve performance? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15146: [SPARK-17590][SQL] Analyze CTE definitions at onc...

2016-09-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15146#discussion_r79487809 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -76,6 +76,23 @@ class SubquerySuite extends QueryTest with

[GitHub] spark pull request #15090: [SPARK-17073] [SQL] generate column-level statist...

2016-09-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15090#discussion_r79492362 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/StatisticsSuite.scala --- @@ -101,4 +101,47 @@ class StatisticsSuite extends QueryTest with

[GitHub] spark pull request #15090: [SPARK-17073] [SQL] generate column-level statist...

2016-09-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15090#discussion_r79492145 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -0,0 +1,159 @@ +/* + * Licensed to

[GitHub] spark pull request #15090: [SPARK-17073] [SQL] generate column-level statist...

2016-09-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15090#discussion_r79491476 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -0,0 +1,159 @@ +/* + * Licensed to

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-19 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15149 Why does the sum need to be less than one? That is not the case for R's randomForest "cutoff" parameter. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #15047: [SPARK-17495] [SQL] Add Hash capability semantically equ...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15047 **[Test build #65615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65615/consoleFull)** for PR 15047 at commit

[GitHub] spark issue #15046: [SPARK-17492] [SQL] Fix Reading Cataloged Data Sources w...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15046 **[Test build #65616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65616/consoleFull)** for PR 15046 at commit

[GitHub] spark pull request #15090: [SPARK-17073] [SQL] generate column-level statist...

2016-09-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15090#discussion_r79489925 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/StatisticsSuite.scala --- @@ -101,4 +101,47 @@ class StatisticsSuite extends QueryTest with

[GitHub] spark issue #15132: [SPARK-17510][STREAMING][KAFKA] config max rate on a per...

2016-09-19 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15132 I personally wouldn't encourage people to use receivers unless they had a very specific reason to. The kafka-0-10 submodule doesn't even have a way to create receiver based streams, so I'm

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

2016-09-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15147 @nbeyer I manually checked the method call before and it seems that is currently only called when reading/inferring CSV/JSON (but I'd like you confirm this to double check in case). --- If

[GitHub] spark pull request #14803: [SPARK-17153][SQL] Should read partition data whe...

2016-09-19 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/14803#discussion_r79488089 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -197,10 +197,13 @@ case class DataSource(

[GitHub] spark pull request #14803: [SPARK-17153][SQL] Should read partition data whe...

2016-09-19 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/14803#discussion_r79488934 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala --- @@ -608,6 +608,34 @@ class FileStreamSourceSuite extends

[GitHub] spark issue #15047: [SPARK-17495] [SQL] Add Hash capability semantically equ...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15047 **[Test build #65614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65614/consoleFull)** for PR 15047 at commit

[GitHub] spark pull request #15071: [SPARK-17517][SQL]Improve generated Code for Broa...

2016-09-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15071#discussion_r79489276 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/BroadcastHashJoinBenchmark.scala --- @@ -0,0 +1,84 @@ +/* + *

[GitHub] spark issue #15047: [SPARK-17495] [SQL] Add Hash capability semantically equ...

2016-09-19 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/15047 @hvanhovell Done with all changes. Ready for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #15071: [SPARK-17517][SQL]Improve generated Code for Broa...

2016-09-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15071#discussion_r79488945 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/BroadcastHashJoinBenchmark.scala --- @@ -0,0 +1,84 @@ +/* + *

[GitHub] spark issue #15152: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15152 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15152: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15152 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65609/ Test FAILed. ---

[GitHub] spark issue #15152: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15152 **[Test build #65609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65609/consoleFull)** for PR 15152 at commit

[GitHub] spark issue #15132: [SPARK-17510][STREAMING][KAFKA] config max rate on a per...

2016-09-19 Thread jnadler
Github user jnadler commented on the issue: https://github.com/apache/spark/pull/15132 Are you comfortable with the inconsistency this creates between Direct Stream and Receivers? I don't need a receiver-based solution personally. It does occur to me that it might be desirable

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #65613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65613/consoleFull)** for PR 1 at commit

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/1 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #11490: [SPARK-13117] [Web UI] WebUI should use the local ip not...

2016-09-19 Thread l15k4
Github user l15k4 commented on the issue: https://github.com/apache/spark/pull/11490 @zsxwing Done, thanks : https://issues.apache.org/jira/browse/SPARK-17600 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15082: [SPARK-17528][SQL] MutableProjection should not cache co...

2016-09-19 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15082 What's the status of this issue? I see that it's currently targeted for 2.0.1 in JIRA and wanted to ping to make sure that this doesn't miss the cut in case we prepare an RC soon. --- If your

[GitHub] spark issue #14634: [SPARK-17051][SQL] we should use hadoopConf in InsertInt...

2016-09-19 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/14634 @yhuai, @cloud-fan What's the status of this issue? Should this still be targeted for 2.0.1 (which it is currently in JIRA)? --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #15064: [SPARK-17509][SQL]When wrapping catalyst datatype to Hiv...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15064 **[Test build #65612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65612/consoleFull)** for PR 15064 at commit

[GitHub] spark issue #15034: [SPARK-16240][ML] ML persistence backward compatibility ...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15034 **[Test build #3279 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3279/consoleFull)** for PR 15034 at commit

[GitHub] spark pull request #15034: [SPARK-16240][ML] ML persistence backward compati...

2016-09-19 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15034#discussion_r79485256 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -354,6 +361,39 @@ private[clustering] trait LDAParams extends Params with

[GitHub] spark issue #15034: [SPARK-16240][ML] ML persistence backward compatibility ...

2016-09-19 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15034 Thanks @hhbyyh ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15064: [SPARK-17509][SQL]When wrapping catalyst datatype to Hiv...

2016-09-19 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/15064 Rebased with latest master. cc - @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #11490: [SPARK-13117] [Web UI] WebUI should use the local ip not...

2016-09-19 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/11490 @l15k4 could you create a JIRA to discuss the issue? A closed PR is not a good place to discuss. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15153: [SPARK-17599] Prevent ListingFileCatalog from failing if...

2016-09-19 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15153 cc @marmbrus @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15153: [SPARK-17599] Prevent ListingFileCatalog from failing if...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15153 **[Test build #65611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65611/consoleFull)** for PR 15153 at commit

[GitHub] spark pull request #15153: [SPARK-17599] Prevent ListingFileCatalog from fai...

2016-09-19 Thread brkyvz
GitHub user brkyvz opened a pull request: https://github.com/apache/spark/pull/15153 [SPARK-17599] Prevent ListingFileCatalog from failing if path doesn't exist ## What changes were proposed in this pull request? The `ListingFileCatalog` lists files given a set of resolved

[GitHub] spark pull request #15051: [SPARK-17499][SparkR][ML][MLLib] make the default...

2016-09-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15051#discussion_r79482668 --- Diff: R/pkg/R/mllib.R --- @@ -695,17 +695,15 @@ setMethod("predict", signature(object = "KMeansModel"), #' @note spark.mlp since 2.1.0

[GitHub] spark issue #15151: [CORE][DOC]: Fix errors in comments

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15151 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15151: [CORE][DOC]: Fix errors in comments

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15151 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65606/ Test PASSed. ---

[GitHub] spark issue #15151: [CORE][DOC]: Fix errors in comments

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15151 **[Test build #65606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65606/consoleFull)** for PR 15151 at commit

[GitHub] spark pull request #15046: [SPARK-17492] [SQL] Fix Reading Cataloged Data So...

2016-09-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15046#discussion_r79481517 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -318,24 +319,28 @@ case class DataSource(

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost

2016-09-19 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14547 So, taking a look at the current patch, the API for this "loss-based" impurity feels clunky and a bit confusing. To enumerate, we have the following scenarios: **1. Completely decoupled

[GitHub] spark pull request #15103: [SPARK-17100] [SQL] fix Python udf in filter on t...

2016-09-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15103 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15103: [SPARK-17100] [SQL] fix Python udf in filter on top of o...

2016-09-19 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15103 Merging into master and 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15103: [SPARK-17100] [SQL] fix Python udf in filter on top of o...

2016-09-19 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15103 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15054: [SPARK-17502] [SQL] Fix Multiple Bugs in DDL Statements ...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15054 **[Test build #65610 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65610/consoleFull)** for PR 15054 at commit

[GitHub] spark issue #14926: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14926 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65607/ Test FAILed. ---

[GitHub] spark pull request #15103: [SPARK-17100] [SQL] fix Python udf in filter on t...

2016-09-19 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15103#discussion_r79479009 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -109,7 +109,9 @@ object EliminateOuterJoin extends

[GitHub] spark issue #14926: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14926 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14926: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14926 **[Test build #65607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65607/consoleFull)** for PR 14926 at commit

[GitHub] spark pull request #15054: [SPARK-17502] [SQL] Fix Multiple Bugs in DDL Stat...

2016-09-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15054#discussion_r79478675 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -37,7 +38,9 @@ case class

[GitHub] spark pull request #15054: [SPARK-17502] [SQL] Fix Multiple Bugs in DDL Stat...

2016-09-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15054#discussion_r79478609 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -246,30 +246,42 @@ class SessionCatalog(

[GitHub] spark pull request #15103: [SPARK-17100] [SQL] fix Python udf in filter on t...

2016-09-19 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15103#discussion_r79477760 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -109,7 +109,9 @@ object EliminateOuterJoin extends

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15122 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15122 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65605/ Test PASSed. ---

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15122 **[Test build #65605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65605/consoleFull)** for PR 15122 at commit

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15122 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65604/ Test PASSed. ---

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15122 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15122 **[Test build #65604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65604/consoleFull)** for PR 15122 at commit

[GitHub] spark pull request #15150: [SPARK-17595] [MLLib] Use a bounded priority queu...

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15150#discussion_r79468787 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -580,7 +581,11 @@ class Word2VecModel private[spark] ( ind +=

[GitHub] spark issue #15144: [SPARK-17587][PYTHON][MLLIB] SparseVector __getitem__ sh...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15144 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65608/ Test PASSed. ---

[GitHub] spark issue #15144: [SPARK-17587][PYTHON][MLLIB] SparseVector __getitem__ sh...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15144 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15144: [SPARK-17587][PYTHON][MLLIB] SparseVector __getitem__ sh...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15144 **[Test build #65608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65608/consoleFull)** for PR 15144 at commit

[GitHub] spark issue #15152: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15152 **[Test build #65609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65609/consoleFull)** for PR 15152 at commit

[GitHub] spark pull request #15150: [SPARK-17595] [MLLib] Use a bounded priority queu...

2016-09-19 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/15150#discussion_r79466262 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -580,7 +581,11 @@ class Word2VecModel private[spark] ( ind +=

[GitHub] spark issue #15152: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15152 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15152: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15152 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #15151: [CORE][DOC]: Fix errors in comments

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15151#discussion_r79465795 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -937,7 +938,7 @@ object SparkSession { } /** - *

[GitHub] spark pull request #15151: [CORE][DOC]: Fix errors in comments

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15151#discussion_r79465827 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -359,6 +359,7 @@ class SparkSession private( * * WARNING:

[GitHub] spark issue #15152: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15152 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15152: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-19 Thread dhruve
GitHub user dhruve opened a pull request: https://github.com/apache/spark/pull/15152 [SPARK-17365][Core] Remove/Kill multiple executors together to reduce RPC call time. ## What changes were proposed in this pull request? We are killing multiple executors together instead of

[GitHub] spark pull request #14959: [SPARK-17387][PYSPARK] Creating SparkContext() fr...

2016-09-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14959#discussion_r79461443 --- Diff: python/pyspark/java_gateway.py --- @@ -50,13 +50,18 @@ def launch_gateway(): # proper classpath and settings from spark-env.sh

[GitHub] spark issue #15144: [SPARK-17587][PYTHON][MLLIB] SparseVector __getitem__ sh...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15144 **[Test build #65608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65608/consoleFull)** for PR 15144 at commit

[GitHub] spark pull request #15106: [SPARK-16439] [SQL] bring back the separator in S...

2016-09-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15106 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15144: [SPARK-17587][PYTHON][MLLIB] SparseVector __getitem__ sh...

2016-09-19 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/15144 @BryanCutler Sure, here you are. Actually both `Sparse` and `Dense` had to be fixed. Note: Technically speaking this can is a breaking change if someone expects `ValueError`. On

[GitHub] spark issue #15106: [SPARK-16439] [SQL] bring back the separator in SQL UI

2016-09-19 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15106 Merging this into master and 2.0, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11490: [SPARK-13117] [Web UI] WebUI should use the local ip not...

2016-09-19 Thread l15k4
Github user l15k4 commented on the issue: https://github.com/apache/spark/pull/11490 This is still causing problems in `2.0.0` standalone, `SPARK_PUBLIC_DNS` works well for Driver WebUI, it doesn't seem to have any effect on Worker or Master UI. I tried to set all of these :

[GitHub] spark issue #15150: [SPARK-17595] [MLLib] Use a bounded priority queue to fi...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15150 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15150: [SPARK-17595] [MLLib] Use a bounded priority queue to fi...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15150 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65603/ Test PASSed. ---

[GitHub] spark issue #15150: [SPARK-17595] [MLLib] Use a bounded priority queue to fi...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15150 **[Test build #65603 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65603/consoleFull)** for PR 15150 at commit

[GitHub] spark issue #15106: [SPARK-16439] [SQL] bring back the separator in SQL UI

2016-09-19 Thread maver1ck
Github user maver1ck commented on the issue: https://github.com/apache/spark/pull/15106 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15106: [SPARK-16439] [SQL] bring back the separator in SQL UI

2016-09-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15106 If @maver1ck is OK with it I am, sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15120: [SPARK-4563][core] Allow driver to advertise a different...

2016-09-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15120 /cc @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15106: [SPARK-16439] [SQL] bring back the separator in SQL UI

2016-09-19 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15106 @srowen Could you left an `lgtm` here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14659#discussion_r79457356 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -196,8 +198,13 @@ private[spark] class ApplicationMaster(

[GitHub] spark issue #14926: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/14926 I rebased to get changes and the PR has the additional commits. Will close this one and open a new one. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #14926: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14926 **[Test build #65607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65607/consoleFull)** for PR 14926 at commit

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-19 Thread dhruve
Github user dhruve closed the pull request at: https://github.com/apache/spark/pull/14926 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14659#discussion_r79457084 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging { } }

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-19 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/1 @cloud-fan can you please review the fix and retest? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13863: [SPARK-16158][CORE][WIP] Make dynamic allocation heurist...

2016-09-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13863 What do we need to do to push this forward? I probably wouldn't create a new top level package just for this api. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #15140: [SPARK-17585][PySpark][Core] PySpark SparkContext.addFil...

2016-09-19 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15140 Just one minor suggestion, otherwise LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15140: [SPARK-17585][PySpark][Core] PySpark SparkContext...

2016-09-19 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/15140#discussion_r79454288 --- Diff: python/pyspark/tests.py --- @@ -409,13 +409,22 @@ def func(x): self.assertEqual("Hello World!", res) def

[GitHub] spark issue #15151: [CORE][DOC]: Fix errors in comments

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15151 **[Test build #65606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65606/consoleFull)** for PR 15151 at commit

[GitHub] spark pull request #15151: [CORE][DOC]: Fix errors in comments

2016-09-19 Thread wangmiao1981
GitHub user wangmiao1981 opened a pull request: https://github.com/apache/spark/pull/15151 [CORE][DOC]: Fix errors in comments ## What changes were proposed in this pull request? While reading source code of CORE and SQL core, I found some minor errors in comments such as extra

[GitHub] spark pull request #15001: [SPARK-17438][WebUI] Show Application.executorLim...

2016-09-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15001 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15001: [SPARK-17438][WebUI] Show Application.executorLimit in t...

2016-09-19 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/15001 LGTM, merging into master 2.0 thanks @zsxwing! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

<    1   2   3   4   5   >