[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 ``` Quitting from lines 65-67 (sparkr-vignettes.Rmd) Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics: Java version check failed Execution halted ```

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21278 **[Test build #90399 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90399/testReport)** for PR 21278 at commit [`7d1894c`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90399/ Test FAILed. ---

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21255: [SPARK-24186][R][SQL]change reverse and concat to...

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r186942863 --- Diff: R/pkg/R/functions.R --- @@ -2047,17 +2049,15 @@ setMethod("countDistinct", #' \code{concat}: Concatenates multiple input columns together

[GitHub] spark pull request #21255: [SPARK-24186][R][SQL]change reverse and concat to...

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r186942776 --- Diff: R/pkg/R/functions.R --- @@ -2047,17 +2049,15 @@ setMethod("countDistinct", #' \code{concat}: Concatenates multiple input columns together

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21268 Thanks for your comment @jerryshao. Unfortunately I don't think so. Since through the SHS you can access all the other pages, all of them have to support the proper detection of the proxy base as w

[GitHub] spark issue #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwrite a p...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21257 **[Test build #90400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90400/testReport)** for PR 21257 at commit [`a51620b`](https://github.com/apache/spark/commit/a5

[GitHub] spark issue #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwrite a p...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21257 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spa

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186940485 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,39 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedArgs

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186940268 --- Diff: R/pkg/R/sparkR.R --- @@ -163,6 +163,10 @@ sparkR.sparkContext <- function( submitOps <- getClientModeSparkSubmitOpts( Sys

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186939722 --- Diff: R/pkg/R/utils.R --- @@ -756,7 +756,7 @@ launchScript <- function(script, combinedArgs, wait = FALSE) { # stdout = F means discard out

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186940416 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,39 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedArgs

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186939913 --- Diff: R/pkg/R/sparkR.R --- @@ -163,6 +163,10 @@ sparkR.sparkContext <- function( submitOps <- getClientModeSparkSubmitOpts( Sys

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186940332 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,39 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedArgs

[GitHub] spark issue #21279: [SPARK-24219][k8s] Improve the docker building script to...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21279 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/2980/ ---

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3065/ Tes

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 yea, I don't think Spark builds on Java 9 (at least from what I've seen) I see the package is gone from CRAN so the test results are brief but maybe related to https://cran.r-project

[GitHub] spark issue #21279: [SPARK-24219][k8s] Improve the docker building script to...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21279 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21279: [SPARK-24219][k8s] Improve the docker building script to...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21279 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3064/ Tes

[GitHub] spark issue #21279: [SPARK-24219][k8s] Improve the docker building script to...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21279 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/2980/ --- --

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21278 **[Test build #90399 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90399/testReport)** for PR 21278 at commit [`7d1894c`](https://github.com/apache/spark/commit/7d

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/21278 Ah got it - Thanks @HyukjinKwon . I'll check if `== 1.8` is supported by R syntax @felixcheung I moved the logic into a `checkJavaVersion` function now. Let me know if this looks better.

[GitHub] spark issue #21182: [SPARK-24068] Propagating DataFrameReader's options to T...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21182 I tried but it had conflict. @MaxGekk mind opening a backport pr? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apa

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-05-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20636 ping @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21278 (I think Spark itself doesn't properly support Java 9 yet if I haven't missed a discussion or thread) --- - To unsubscribe,

[GitHub] spark issue #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression uses g...

2018-05-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21276 @gatorsmile @hvanhovell could you trigger tests? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21279: [SPARK-24219][k8s] Improve the docker building script to...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21279 **[Test build #90398 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90398/testReport)** for PR 21279 at commit [`e9ea7e5`](https://github.com/apache/spark/commit/e9

[GitHub] spark issue #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression uses g...

2018-05-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21276 How about adding a `sql` method in `Aggregator` and printing it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or

[GitHub] spark pull request #21279: [SPARK-24219][k8s] Improve the docker building sc...

2018-05-08 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/21279 [SPARK-24219][k8s] Improve the docker building script to avoid copying everything under examples to docker image ## What changes were proposed in this pull request? Current docker build

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-08 Thread zheh12
Github user zheh12 commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186936032 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class Ins

[GitHub] spark issue #21182: [SPARK-24068] Propagating DataFrameReader's options to T...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21182 shall we backport it to 2.3? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21144: [SPARK-24043][SQL] Interpreted Predicate should initiali...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21144 shall we backport it to 2.3? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21266: [SPARK-24206][SQL] Improve DataSource read benchm...

2018-05-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21266#discussion_r186934756 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadBenchmark.scala --- @@ -1,339 +0,0 @@ -/* --- End di

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/21278 Thats a fair question -- I initially created a script was to handle Windows calls but I think we can do some of the split stuff inside R. Let me try that out. Regarding Java 9, do you kn

[GitHub] spark issue #21064: [SPARK-23976][Core] Detect length overflow in UTF8String...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21064 shall we backport it to 2.3? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 also I think test fails on Java 9 - is there a way to exclude - doc sounds like it's a minimal version --- - To unsubscribe,

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3063/ Tes

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21278 **[Test build #90397 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90397/testReport)** for PR 21278 at commit [`b4e1039`](https://github.com/apache/spark/commit/b4

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/21278 The need for both the Requirements field and the runtime check is documented at https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Writing-portable-packages (Search for `Make sure that a

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/21278 [SPARKR] Require Java 8 for SparkR This change updates the SystemRequirements and also includes a runtime check if the JVM is being launched by R. The runtime check is done by querying `java -ver

[GitHub] spark pull request #20973: [SPARK-20114][ML] spark.ml parity for sequential ...

2018-05-08 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/20973#discussion_r186931806 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/PrefixSpan.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #21251: [SPARK-10878][core] Fix race condition when multiple cli...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21251 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21251: [SPARK-10878][core] Fix race condition when multiple cli...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21251 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3062/ Tes

[GitHub] spark issue #21251: [SPARK-10878][core] Fix race condition when multiple cli...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21251 **[Test build #90396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90396/testReport)** for PR 21251 at commit [`9a6377b`](https://github.com/apache/spark/commit/9a

[GitHub] spark pull request #21165: [Spark-20087][CORE] Attach accumulators / metrics...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21165#discussion_r186930874 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -212,9 +212,15 @@ case object TaskResultLost extends TaskFailedReason { * T

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186929981 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -18,15 +18,50 @@ package org.apache.spa

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21235 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark issue #21186: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by defau...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21186 can you resolve the conflicts? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark pull request #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10....

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21070 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21070 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21204: [SPARK-24132][ML] Instrumentation improvement for...

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21204 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to InputPar...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21145 any other suggestions about naming? we are going to rename `DataReaderFactory` to `InputPartition`. cc @rxin @marmbrus @jose-torres --- -

[GitHub] spark pull request #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to I...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21145#discussion_r186927701 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/DataSourceReader.java --- @@ -76,5 +76,5 @@ * If this method fails (by th

[GitHub] spark issue #21204: [SPARK-24132][ML] Instrumentation improvement for classi...

2018-05-08 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/21204 LGTM. Merged into master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21235 I'll merge into master once the test passes. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additi

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186927477 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21235 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21235 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21235 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3061/ Tes

[GitHub] spark pull request #21218: [SPARK-24155][ML] Instrumentation improvements fo...

2018-05-08 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21218#discussion_r186926118 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/BisectingKMeans.scala --- @@ -278,6 +279,7 @@ class BisectingKMeans @Since("2.0.0") (

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21235 **[Test build #90395 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90395/testReport)** for PR 21235 at commit [`7b64a97`](https://github.com/apache/spark/commit/7b

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21231 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r186925349 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,40 @@ case class SortPrefix(child: SortOr

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21153 **[Test build #90394 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90394/testReport)** for PR 21153 at commit [`b47beab`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90394/ Test PASSed. ---

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3060/ Tes

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21153 **[Test build #90394 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90394/testReport)** for PR 21153 at commit [`b47beab`](https://github.com/apache/spark/commit/b4

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90391/ Test PASSed. ---

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21231 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21231 **[Test build #90391 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90391/testReport)** for PR 21231 at commit [`28f1b70`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #21277: [ML]Power Iteration Clustering is not displaying cluster...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21277 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3059/ Tes

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21153 **[Test build #90393 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90393/testReport)** for PR 21153 at commit [`86fa433`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90393/ Test FAILed. ---

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21277: [ML]Power Iteration Clustering is not displaying cluster...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21277 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21277: [ML]Power Iteration Clustering is not displaying ...

2018-05-08 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/21277 [ML]Power Iteration Clustering is not displaying cluster indices corresponding to some nodes. ## What changes were proposed in this pull request? 1) Currently PIC in ML displays cluster indi

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21153 **[Test build #90393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90393/testReport)** for PR 21153 at commit [`86fa433`](https://github.com/apache/spark/commit/86

[GitHub] spark pull request #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work ...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21267#discussion_r186920316 --- Diff: python/pyspark/context.py --- @@ -211,9 +211,23 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize,

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21267 Hm .. @jerryshao, seems it's a bit difficult to do so. The simplest way should be just to copy files into the directories in `SparkFiles.getRootDirectory`; however, `SparkEnv` is inaccessible at

[GitHub] spark pull request #21243: [SPARK-24182][yarn] Improve error message when cl...

2018-05-08 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21243#discussion_r186918972 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -346,7 +346,7 @@ private[spark] class Applica

[GitHub] spark pull request #21266: [SPARK-24206][SQL] Improve DataSource read benchm...

2018-05-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21266#discussion_r186918634 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadBenchmark.scala --- @@ -1,339 +0,0 @@ -/* ---

[GitHub] spark issue #21165: [Spark-20087][CORE] Attach accumulators / metrics to 'Ta...

2018-05-08 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/21165 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21269: [SPARK-23355][SQL][DOC][FOLLOWUP] Add migration doc for ...

2018-05-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21269 Thank you, @HyukjinKwon and @cloud-fan . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21266 **[Test build #90392 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90392/testReport)** for PR 21266 at commit [`2813706`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90392/ Test FAILed. ---

[GitHub] spark issue #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in SQL fun...

2018-05-08 Thread mmolimar
Github user mmolimar commented on the issue: https://github.com/apache/spark/pull/18447 @felixcheung I think it should be fine now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional co

[GitHub] spark issue #21273: [WIP][SPARK-17916][SQL] Fix empty string being parsed as...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21273 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90388/ Test PASSed. ---

[GitHub] spark issue #21273: [WIP][SPARK-17916][SQL] Fix empty string being parsed as...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21273 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21273: [WIP][SPARK-17916][SQL] Fix empty string being parsed as...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21273 **[Test build #90388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90388/testReport)** for PR 21273 at commit [`db18345`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3058/ Tes

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-08 Thread zheh12
Github user zheh12 commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186915569 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class Ins

  1   2   3   4   5   >