[GitHub] spark issue #13736: [SPARK-12113][SQL] Add some timing metrics for blocking ...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13736 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64051/ Test PASSed. ---

[GitHub] spark issue #13320: [SPARK-13184][SQL] Add a datasource-specific option minP...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13320 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13736: [SPARK-12113][SQL] Add some timing metrics for blocking ...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13736 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13320: [SPARK-13184][SQL] Add a datasource-specific option minP...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13320 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64052/ Test FAILed. ---

[GitHub] spark issue #13320: [SPARK-13184][SQL] Add a datasource-specific option minP...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13320 **[Test build #64052 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64052/consoleFull)** for PR 13320 at commit [`6440370`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13736: [SPARK-12113][SQL] Add some timing metrics for blocking ...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13736 **[Test build #64051 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64051/consoleFull)** for PR 13736 at commit [`7ccd981`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14716: [SPARK-17141] [ML] MinMaxScaler should remain NaN value.

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14716 **[Test build #64057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64057/consoleFull)** for PR 14716 at commit [`19cb3ad`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #14716: [SPARK-17141] [ML] MinMaxScaler should remain NaN...

2016-08-19 Thread yanboliang
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/14716 [SPARK-17141] [ML] MinMaxScaler should remain NaN value. ## What changes were proposed in this pull request? ```MinMaxScaler``` should remain ```NaN``` value. ## How was this pa

[GitHub] spark issue #14715: [SPARK-17085] [Streaming] [Documentation and actual code...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14715 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64056/ Test PASSed. ---

[GitHub] spark issue #14715: [SPARK-17085] [Streaming] [Documentation and actual code...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14715 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14715: [SPARK-17085] [Streaming] [Documentation and actual code...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14715 **[Test build #64056 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64056/consoleFull)** for PR 14715 at commit [`6d1c52f`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r75446103 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -200,22 +375,77 @@ private[spark] class HiveExternalCatalog(cl

[GitHub] spark pull request #14181: [SPARK-15382][SQL] Fix a rule to push down projec...

2016-08-19 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/14181#discussion_r75446046 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -148,6 +148,21 @@ class SimpleTestOptimizer extends Opti

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r75445428 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -584,13 +579,8 @@ case class AlterTableSetLocationCommand(

[GitHub] spark issue #14714: paged jdbcRDD for like mysql limit start,pageSize

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14714 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #14715: [SPARK-17085] [Streaming] [Documentation and actual code...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14715 **[Test build #64056 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64056/consoleFull)** for PR 14715 at commit [`6d1c52f`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r75445214 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -264,10 +261,8 @@ case class AlterTableUnsetPropertiesCommand(

[GitHub] spark pull request #14666: [SPARK-16578][SparkR] Enable SparkR to connect to...

2016-08-19 Thread zjffdu
Github user zjffdu commented on a diff in the pull request: https://github.com/apache/spark/pull/14666#discussion_r75444995 --- Diff: R/pkg/R/utils.R --- @@ -689,3 +689,33 @@ getSparkContext <- function() { sc <- get(".sparkRjsc", envir = .sparkREnv) sc } +

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r75444916 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -233,226 +229,21 @@ case class CreateDataSour

[GitHub] spark pull request #14715: [SPARK-17085] [Streaming] [Documentation and actu...

2016-08-19 Thread jagadeesanas2
GitHub user jagadeesanas2 opened a pull request: https://github.com/apache/spark/pull/14715 [SPARK-17085] [Streaming] [Documentation and actual code differs - Unsupported Operations] You can merge this pull request into a Git repository by running: $ git pull https://github.c

[GitHub] spark pull request #14714: paged jdbcRDD for like mysql limit start,pageSize

2016-08-19 Thread jianran
GitHub user jianran opened a pull request: https://github.com/apache/spark/pull/14714 paged jdbcRDD for like mysql limit start,pageSize ## What changes were proposed in this pull request? new feature for jdbcRDD with mysql limit query ## How was this patch tested?

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r75444695 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -97,16 +92,17 @@ case class CreateDataSourceT

[GitHub] spark issue #14038: [SPARK-16317][SQL] Add a new interface to filter files i...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14038 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14038: [SPARK-16317][SQL] Add a new interface to filter files i...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14038 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64049/ Test FAILed. ---

[GitHub] spark issue #14038: [SPARK-16317][SQL] Add a new interface to filter files i...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14038 **[Test build #64049 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64049/consoleFull)** for PR 14038 at commit [`d53ad8e`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14674: [SPARK-17002][CORE]: Document that spark.ssl.prot...

2016-08-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14674#discussion_r75444301 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -282,6 +282,9 @@ private[spark] class SecurityManager(sparkConf: SparkConf)

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64045/ Test PASSed. ---

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #64045 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64045/consoleFull)** for PR 14452 at commit [`e094c14`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14181: [SPARK-15382][SQL] Fix a rule to push down projects bene...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14181 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64048/ Test FAILed. ---

[GitHub] spark issue #14181: [SPARK-15382][SQL] Fix a rule to push down projects bene...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14181 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14181: [SPARK-15382][SQL] Fix a rule to push down projects bene...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14181 **[Test build #64048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64048/consoleFull)** for PR 14181 at commit [`c947583`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14181: [SPARK-15382][SQL] Fix a rule to push down projects bene...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14181 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64046/ Test FAILed. ---

[GitHub] spark issue #14181: [SPARK-15382][SQL] Fix a rule to push down projects bene...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14181 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14181: [SPARK-15382][SQL] Fix a rule to push down projects bene...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14181 **[Test build #64046 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64046/consoleFull)** for PR 14181 at commit [`b0f5dd5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14118 I believe we can change the default vale of `nullValue` to `'\u'.toString` in order to express any value is not `null`. I remember this matches with no empty string nor any other string alth

[GitHub] spark issue #14700: [SPARK-17127]Make unaligned access in unsafe available f...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14700 **[Test build #3226 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3226/consoleFull)** for PR 14700 at commit [`24bcf05`](https://github.com/apache/spark/commit

[GitHub] spark pull request #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/dupli...

2016-08-19 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/14705#discussion_r75441096 --- Diff: R/pkg/R/DataFrame.R --- @@ -932,7 +932,7 @@ setMethod("sample_frac", #' @param x a SparkDataFrame. #' @family SparkDataFrame functions

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/10896 Could you update? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14118 @rxin Please let me leave my though why I thought it looks good to me in case it is helpful. Yes, but we should set `nullValue` for writing `null`. So, I think, setting `""` for `nullVa

[GitHub] spark issue #14666: [SPARK-16578][SparkR] Enable SparkR to connect to a remo...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14666 **[Test build #64055 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64055/consoleFull)** for PR 14666 at commit [`54fe8a9`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #14475: [SPARK-16862] Configurable buffer size in `UnsafeSorterS...

2016-08-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14475 The change looks simple & good. Left couple minor comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14475: [SPARK-16862] Configurable buffer size in `Unsafe...

2016-08-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14475#discussion_r75440822 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java --- @@ -31,6 +34,9 @@ * of the file format).

[GitHub] spark issue #14709: [SPARK-17150][SQL] Support SQL generation for inline tab...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14709 **[Test build #64054 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64054/consoleFull)** for PR 14709 at commit [`442918f`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #14666: [SPARK-16578][SparkR] Enable SparkR to connect to...

2016-08-19 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/14666#discussion_r75440848 --- Diff: R/pkg/R/utils.R --- @@ -689,3 +689,33 @@ getSparkContext <- function() { sc <- get(".sparkRjsc", envir = .sparkREnv) sc } +

[GitHub] spark pull request #14475: [SPARK-16862] Configurable buffer size in `Unsafe...

2016-08-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14475#discussion_r75440799 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java --- @@ -50,7 +56,21 @@ public UnsafeSorterSpillReader(

[GitHub] spark pull request #14384: [Spark-16443][SparkR] Alternating Least Squares (...

2016-08-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14384#discussion_r75440356 --- Diff: R/pkg/R/mllib.R --- @@ -632,3 +642,146 @@ setMethod("predict", signature(object = "AFTSurvivalRegressionModel"), function(object

[GitHub] spark pull request #14709: [SPARK-17150][SQL] Support SQL generation for inl...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14709#discussion_r75440152 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LocalRelation.scala --- @@ -75,4 +76,16 @@ case class LocalRelation(outp

[GitHub] spark pull request #14709: [SPARK-17150][SQL] Support SQL generation for inl...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14709#discussion_r75440135 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LocalRelation.scala --- @@ -75,4 +76,16 @@ case class LocalRelation(outp

[GitHub] spark issue #14713: [SPARK-16994][SQL] Whitelist operators for predicate pus...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14713 **[Test build #64053 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64053/consoleFull)** for PR 14713 at commit [`82935a7`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/duplicated a...

2016-08-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14705 Hmm, I'm beginning to think we could do this: ``` generics.R setGeneric("spark.naiveBayes", function(data, formula, ...) { standardGeneric("spark.naiveBayes") }) ``` ```

[GitHub] spark pull request #14713: [SPARK-16994][SQL] Whitelist operators for predic...

2016-08-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14713#discussion_r75439649 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1208,17 +1208,27 @@ object PushDownPredicate extends Rule

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75439413 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -95,6 +95,12 @@ abstract class LogicalPlan extends

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10896 **[Test build #3228 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3228/consoleFull)** for PR 10896 at commit [`572db4c`](https://github.com/apache/spark/commit

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75438886 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala --- @@ -32,5 +32,11 @@ package org.apache.spark.sql.catalys

[GitHub] spark pull request #14713: [SPARK-16994][SQL] Whitelist operators for predic...

2016-08-19 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/14713#discussion_r75438847 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1208,17 +1208,27 @@ object PushDownPredicate extends Ru

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10896 **[Test build #3228 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3228/consoleFull)** for PR 10896 at commit [`572db4c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-19 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/10896 okay, thanks!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #14713: [SPARK-16994][SQL] Whitelist operators for predic...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14713#discussion_r75438681 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1208,17 +1208,27 @@ object PushDownPredicate extends

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/10896 Yeah, lets pick this up again. Thanks for the ping. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75438629 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -95,6 +95,12 @@ abstract class LogicalPlan extends

[GitHub] spark pull request #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/dupli...

2016-08-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14705#discussion_r75438540 --- Diff: R/pkg/R/DataFrame.R --- @@ -932,7 +932,7 @@ setMethod("sample_frac", #' @param x a SparkDataFrame. #' @family SparkDataFrame functions

[GitHub] spark pull request #14713: [SPARK-16994][SQL] Whitelist operators for predic...

2016-08-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14713#discussion_r75438304 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1208,17 +1208,27 @@ object PushDownPredicate extends Rule

[GitHub] spark issue #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-cluster...

2016-08-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14639 but to keep in mind, currently even spark.master can change while in-flight - doesn't seem like Spark Scala code prevents that - we could get some very wrong values. I'm not sure that is super r

[GitHub] spark issue #12790: [SPARK-15018][PYSPARK][ML] Fixed bug causing error if Py...

2016-08-19 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/12790 @BryanCutler @MechCoder The current fix of removing the default value for the ```stages``` param is OK for me. But we also should discuss the behavior of ```stages=[]``` which is inconsistent bet

[GitHub] spark issue #14384: [Spark-16443][SparkR] Alternating Least Squares (ALS) wr...

2016-08-19 Thread junyangq
Github user junyangq commented on the issue: https://github.com/apache/spark/pull/14384 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #14713: [SPARK-16994][SQL] Whitelist operators for predic...

2016-08-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14713#discussion_r75437875 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1208,17 +1208,27 @@ object PushDownPredicate extend

[GitHub] spark issue #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-cluster...

2016-08-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14639 It's all of the Runtime Config from the current active SparkSession which includes all SparkConf. http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.SparkSess

[GitHub] spark pull request #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/dupli...

2016-08-19 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/14705#discussion_r75437776 --- Diff: R/pkg/R/functions.R --- @@ -2276,9 +2276,8 @@ setMethod("n_distinct", signature(x = "Column"), countDistinct(x, ...)

[GitHub] spark issue #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-cluster...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14639 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64043/ Test FAILed. ---

[GitHub] spark issue #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-cluster...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14639 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-cluster...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14639 **[Test build #64043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64043/consoleFull)** for PR 14639 at commit [`fef88cd`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14713: [SPARK-16994][SQL] Whitelist operators for predic...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14713#discussion_r75437433 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1208,17 +1208,27 @@ object PushDownPredicate extends

[GitHub] spark pull request #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/dupli...

2016-08-19 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/14705#discussion_r75437423 --- Diff: R/pkg/R/functions.R --- @@ -1335,7 +1336,7 @@ setMethod("rtrim", #' @note sd since 1.6.0 setMethod("sd", signature(x = "Co

[GitHub] spark issue #14713: [SPARK-16994][SQL] Whitelist operators for predicate pus...

2016-08-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14713 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/dupli...

2016-08-19 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/14705#discussion_r75437149 --- Diff: R/pkg/R/mllib.R --- @@ -917,14 +922,14 @@ setMethod("spark.lda", signature(data = "SparkDataFrame"), # Returns a summary of the AFT survival

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75436918 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -95,6 +95,12 @@ abstract class LogicalPlan extends

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-19 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/10896 @hvanhovell ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark issue #13320: [SPARK-13184][SQL] Add a datasource-specific option minP...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13320 **[Test build #64052 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64052/consoleFull)** for PR 13320 at commit [`6440370`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75436727 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -95,6 +95,12 @@ abstract class LogicalPlan ext

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75436746 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala --- @@ -32,5 +32,11 @@ package org.apache.spark.sql.catalyst

[GitHub] spark issue #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/duplicated a...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64047/ Test PASSed. ---

[GitHub] spark issue #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/duplicated a...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14705 **[Test build #64047 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64047/consoleFull)** for PR 14705 at commit [`870279a`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/duplicated a...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14705 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13736: [SPARK-12113][SQL] Add some timing metrics for blocking ...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13736 **[Test build #64051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64051/consoleFull)** for PR 13736 at commit [`7ccd981`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14712 **[Test build #3227 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3227/consoleFull)** for PR 14712 at commit [`4375e76`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14118 What if I am writing explicitly an empty string out? Does it become just 1,,2? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark issue #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-cluster...

2016-08-19 Thread sun-rui
Github user sun-rui commented on the issue: https://github.com/apache/spark/pull/14639 Does this API get only the Spark SQL configurations or including SparkConf? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark issue #14713: [SPARK-16994][SQL] Whitelist operators for predicate pus...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14713 **[Test build #64050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64050/consoleFull)** for PR 14713 at commit [`9be428a`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #14583: [SPARK-16994][SQL] PushDownPredicate should not ignore l...

2016-08-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14583 I'm fixing this differently here: https://github.com/apache/spark/pull/14713 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request #14713: [SPARK-16994][SQL] Whitelist operators for predic...

2016-08-19 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/14713 [SPARK-16994][SQL] Whitelist operators for predicate push down ## What changes were proposed in this pull request? This patch changes predicate push down optimization rule (PushDownPredicate) from

<    2   3   4   5   6   7