[GitHub] spark pull request #21379: [SPARK-24327][SQL] Add an option to quote a parti...

2018-05-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21379#discussion_r191117129 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala --- @@ -78,7 +79,12 @@ private[sql] object

[GitHub] spark issue #21379: [SPARK-24327][SQL] Add an option to quote a partition co...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21379 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21379: [SPARK-24327][SQL] Add an option to quote a partition co...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21379 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3630/

[GitHub] spark issue #21379: [SPARK-24327][SQL] Add an option to quote a partition co...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21379 **[Test build #91213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91213/testReport)** for PR 21379 at commit

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3629/

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21416 **[Test build #91212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91212/testReport)** for PR 21416 at commit

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-27 Thread e-dorigatti
Github user e-dorigatti commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191113977 --- Diff: python/pyspark/util.py --- @@ -89,6 +93,33 @@ def majorMinorVersion(sparkVersion): " version numbers.")

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 @cloud-fan unfortunately, scala vararg can not be overloaded, and scala will return the following error. ```scala Error:(410, 32) ambiguous reference to overloaded definition, both

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-27 Thread e-dorigatti
Github user e-dorigatti commented on the issue: https://github.com/apache/spark/pull/21383 Yes, the problem was that the signature is lost when the function is wrapped, and the worker needs the signature to know whether the function needs keys together with values or not.

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16478 @metasim Thanks for comment. This patch takes the approach to use Scala data types on UDT and let SparkSQL's encoder to convert user data to internal format. I'm not sure which part the

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21435 Merged to master and branch-2.3. There were minor conflicts but I just resolved it by myself. --- - To unsubscribe,

[GitHub] spark pull request #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experim...

2018-05-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21435 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21435 Thanks @BryanCutler. still LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21288 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21288 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3628/

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91209/ Test PASSed. ---

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21435 **[Test build #91209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91209/testReport)** for PR 21435 at commit

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21288 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3627/

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21288 **[Test build #91211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91211/testReport)** for PR 21288 at commit

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21288 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21288 **[Test build #91210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91210/testReport)** for PR 21288 at commit

[GitHub] spark pull request #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmar...

2018-05-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21288#discussion_r191109472 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala --- @@ -0,0 +1,437 @@ +/* + * Licensed to

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3626/

[GitHub] spark pull request #21432: [SPARK-24373][SQL] Add AnalysisBarrier to Relatio...

2018-05-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21432 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21435 **[Test build #91209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91209/testReport)** for PR 21435 at commit

[GitHub] spark pull request #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experim...

2018-05-27 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21435#discussion_r191107787 --- Diff: docs/sql-programming-guide.md --- @@ -1827,6 +1827,10 @@ working with timestamps in `pandas_udf`s to get the best performance, see -

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21432 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21424: [SPARK-24379] BroadcastExchangeExec should catch ...

2018-05-27 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/21424#discussion_r191107018 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -106,11 +108,20 @@ private[execution] object

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3625/

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21389 **[Test build #91208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91208/testReport)** for PR 21389 at commit

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21416 Yes. The design of PySpark API seems to be a bit different than Scala/Java API at beginning. If we are going to make them consistent, either we break Scala queries like

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21319 Anyway I think moving statistics to physical plan is the ultimate solution, all others are workarounds, we should pick the simplest workaround. I'm glad to take your stats visitor approach if

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21319 > but in the first pass I see that it breaks equality by relying on a user-supplied reader instance. Can you say more about this? I explicitly mentioned it in the PR description that

[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

2018-05-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r191104615 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -657,18 +656,46 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #21424: [SPARK-24379] BroadcastExchangeExec should catch ...

2018-05-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21424#discussion_r191104413 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -106,11 +108,20 @@ private[execution] object

[GitHub] spark pull request #21397: [SPARK-24334] Fix race condition in ArrowPythonRu...

2018-05-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21397 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21416 @viirya good point. One thing I'm not sure is, does `isin(collection: Iterable)` conflict with `isin(list: Any*)`? if they don't conflict, they we can follow pyspark. ---

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21397 Merged to master and branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21416: [SPARK-24371] [SQL] Added isInCollection in DataF...

2018-05-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21416#discussion_r191101660 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -219,7 +218,14 @@ object

[GitHub] spark issue #18390: [SPARK-21178][ML] Add support for label specific metrics...

2018-05-27 Thread thesuperzapper
Github user thesuperzapper commented on the issue: https://github.com/apache/spark/pull/18390 @MLnick @WeichenXu123 sorry to ping again, but what is currently stopping this from merging? --- - To unsubscribe,

[GitHub] spark issue #21420: [SPARK-24377][Spark Submit] make --py-files work in non ...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21420 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3624/

[GitHub] spark issue #21420: [SPARK-24377][Spark Submit] make --py-files work in non ...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21420 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21420: [SPARK-24377][Spark Submit] make --py-files work in non ...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21420 **[Test build #91207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91207/testReport)** for PR 21420 at commit

[GitHub] spark pull request #21317: [SPARK-24232][k8s] Add support for secret env var...

2018-05-27 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21317#discussion_r191092872 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/SecretEnvUtils.scala --- @@ -0,0 +1,28 @@ +/* + * Licensed

[GitHub] spark pull request #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap,...

2018-05-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21434#discussion_r191090953 --- Diff: R/pkg/R/functions.R --- @@ -3062,6 +3077,21 @@ setMethod("array_sort", column(jc) }) +#' @details

[GitHub] spark pull request #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap,...

2018-05-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21434#discussion_r191090821 --- Diff: R/pkg/R/functions.R --- @@ -207,7 +208,7 @@ NULL #' tmp <- mutate(df, v1 = create_array(df$mpg, df$cyl, df$hp)) #' head(select(tmp,

[GitHub] spark pull request #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap,...

2018-05-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21434#discussion_r191090619 --- Diff: R/pkg/R/functions.R --- @@ -207,7 +208,7 @@ NULL #' tmp <- mutate(df, v1 = create_array(df$mpg, df$cyl, df$hp)) #' head(select(tmp,

[GitHub] spark pull request #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap,...

2018-05-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21434#discussion_r191090863 --- Diff: R/pkg/R/functions.R --- @@ -3048,6 +3048,26 @@ setMethod("array_position", column(jc) }) +#'

[GitHub] spark pull request #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap,...

2018-05-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21434#discussion_r191090725 --- Diff: R/pkg/R/functions.R --- @@ -3048,6 +3048,26 @@ setMethod("array_position", column(jc) }) +#'

[GitHub] spark pull request #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap,...

2018-05-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21434#discussion_r191090696 --- Diff: R/pkg/R/functions.R --- @@ -3048,6 +3048,26 @@ setMethod("array_position", column(jc) }) +#'

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 @cloud-fan Let me know if the new API looks good to you. Thanks. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91206/ Test PASSed. ---

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21370 **[Test build #91206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91206/testReport)** for PR 21370 at commit

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21424 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21424 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21424 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91203/ Test PASSed. ---

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21424 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91204/ Test PASSed. ---

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21424 **[Test build #91204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91204/testReport)** for PR 21424 at commit

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21424 **[Test build #91203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91203/testReport)** for PR 21424 at commit

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91202/ Test PASSed. ---

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16478 **[Test build #91202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91202/testReport)** for PR 16478 at commit

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21397 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21397 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91201/ Test PASSed. ---

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21397 **[Test build #91201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91201/testReport)** for PR 21397 at commit

[GitHub] spark pull request #21397: [SPARK-24334] Fix race condition in ArrowPythonRu...

2018-05-27 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21397#discussion_r191082371 --- Diff: python/pyspark/sql/tests.py --- @@ -4931,6 +4931,30 @@ def foo3(key, pdf): expected4 = udf3.func((), pdf)

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-27 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/21397 Sure! Added. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3623/

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21370 **[Test build #91206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91206/testReport)** for PR 21370 at commit

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191080316 --- Diff: docs/configuration.md --- @@ -456,6 +456,29 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3622/

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191080194 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,9 +238,13 @@ class Dataset[T] private[sql]( * @param truncate

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91205/ Test FAILed. ---

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21370 **[Test build #91205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91205/testReport)** for PR 21370 at commit

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191080082 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -358,6 +357,43 @@ class Dataset[T] private[sql]( sb.toString()

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21370 **[Test build #91205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91205/testReport)** for PR 21370 at commit

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191080049 --- Diff: python/pyspark/sql/dataframe.py --- @@ -347,13 +347,30 @@ def show(self, n=20, truncate=True, vertical=False): name | Bob

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191080066 --- Diff: python/pyspark/sql/dataframe.py --- @@ -347,13 +347,30 @@ def show(self, n=20, truncate=True, vertical=False): name | Bob

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191080057 --- Diff: python/pyspark/sql/tests.py --- @@ -3040,6 +3040,50 @@ def test_csv_sampling_ratio(self): .csv(rdd,

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191080044 --- Diff: python/pyspark/sql/dataframe.py --- @@ -347,13 +347,30 @@ def show(self, n=20, truncate=True, vertical=False): name | Bob

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191080037 --- Diff: python/pyspark/sql/dataframe.py --- @@ -347,13 +347,30 @@ def show(self, n=20, truncate=True, vertical=False): name | Bob

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191080026 --- Diff: docs/configuration.md --- @@ -456,6 +456,29 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21424 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3621/

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21424 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21424 **[Test build #91204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91204/testReport)** for PR 21424 at commit

[GitHub] spark issue #21424: [SPARK-24379] BroadcastExchangeExec should catch SparkOu...

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21424 **[Test build #91203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91203/testReport)** for PR 21424 at commit

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3620/

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

2018-05-27 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r191078137 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -657,18 +656,46 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16478 **[Test build #91202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91202/testReport)** for PR 16478 at commit

[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

2018-05-27 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r191078043 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala --- @@ -53,7 +52,7 @@ class HiveClientSuite(version: String)

[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

2018-05-27 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r191078009 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -657,18 +656,46 @@ private[client] class Shim_v0_13 extends

  1   2   >